The practice of collecting customer feedback is a bad thing.

Customer satisfaction surveys are extremely common. Each time you visit a bank, send a request to an Internet provider, go on a trip, or hire a freelancer for a task, you then receive an email request with an offer to evaluate the interaction.



At first glance, this is a good idea. Given the practical ubiquity of the Internet and e-mail, combined with the low cost of sending these letters, the ability to understand the essence of the matter "from the inside" that your company can receive from customers - what might not like in this practice?



Feedback store / rentals -> evaluation -> reward has become just an integral part of some systems. Many interactive trading platforms incorporate these feedback operations into ratings, which then become an assessment that affects the prospects of the person being assessed.

')

And this is where the problems begin.



There is a point where this approach ceases to be useful and the direct effects of an evaluation system based on feedback results become counterproductive. Such a point is the situation when the rating becomes the goal.



When a company proclaims: “All feedback ratings from a client must be at least X, otherwise ...”, the company thus begins the process of canceling and destroying all the useful information that could be obtained from this feedback process / surveys.



The formulation of this concept is the “Goodhart Law” (named after the economist Charles Goodhart), which, in fact, states that “when a measurement result becomes a goal, the measurement becomes useless”. He was referring to the economic context, criticizing Margaret Thatcher’s attempts to stimulate the economy through measurement, which set targets in monetary policy (sounds familiar? This is exactly what the data-managed Federal Reserve System has been doing for almost 10 years years old). This is what Superinvestor Howard Marks described in detail in the chapter “Thinking of the Second Order” in his book “The Most Important” .



Goodhart's law, in general, relates to the area of ​​"unintended consequences":

“The best-known examples of the Goodhart law were the enterprises of the Soviet Union, which, when they set a goal in the form of a number of nails, produced a lot of tiny useless nails, and with a goal that was formulated as weight, they produced some huge nails. The quantity and weight correlated well in the central scenario. After they were made targets (at different times and for different periods), they lost this value. ”

This excerpt is taken from the article-review of this phenomenon , in which the author formulates the Goodhart law as: “People and institutions try to achieve their exact goals in the simplest possible way, often observing only the formal norms of the law” , and gives an excellent example of the result obtained in the work "guess the teacher's keyword" :

“Assume that the teacher puts a tricky task in front of you, in which condition the metal plate is located next to the radiator and its far side is warmer than the side near the radiator. The teacher asks: "Why?". If you say, “I don't know,” you will have no chance of a prize; moreover, you will not be marked at least as an active participant in the lesson. But in the current semester, this teacher repeatedly used the phrases “due to thermal convection”, “due to thermal conductivity” and “due to thermal radiation”. One of them, perhaps, is the one that the teacher wants to hear. You say: "Well, maybe due to thermal conductivity?".

This is not a hypothesis regarding a metal plate.

This is not even some kind of conjecture in the full sense. It's just an attempt to guess the teacher's keyword. ”

And this is what happens on a massive scale, as soon as feedback results turn into ratings, which then become targets.



When this process passes to the end in the systems of feedback and ratings, the same dynamics takes effect in the work. Instead of data carrying useful information that can be used to improve interaction with customers, ratings become a goal in themselves, and then they lose their usefulness. Participants focus their efforts on optimizing the actual rating responses, rather than the underlying actions for which the measurement system was built.



“I have satisfactorily dealt with the reason you called today, Mr. Yeslovich?”



"Yes thank you."



“Then you can give me the highest rating if the company chooses you for our customer satisfaction research?”



"Uh, well, I think so."



I do not know about you, but I am always embarrassed at such a conversation. If not outwardly, then in the shower. The whole conversation is built in a begging / imploring construction, which, if it makes an embarrassing situation for me, should be - at least subconsciously - many times humiliating for a person whose livelihood depends on my assessment of 10/10 in some research. This changes the nature and mood of the conversation, taking it out of the useful field, where I would feel comfortable enough to convey useful information.



Example: I rated my communication at 8 points out of 10, because ...

• The robot told me that the waiting time will be 3 minutes, and I waited 7 and a half.
• The agent was polite, but did not know the firmware version of my product.

Such feedback contains useful information.



Actuality:



“Is everything all right now, Mr. Yeftovich?”



"Yes thank you".



"Wonderful. Listen, when will the survey, I need you to give me 10 out of 10, otherwise I will suck, OK? "



"Sure, no problem".



And here I give an assessment of the quality of customer service:



10/10



The amount of useful information received by the company: zero .



What happened when I tried sincerely, for ideological reasons, to honestly evaluate the quality of my service?



I was confronted with real harassment, howling, and different shades of drama and hysteria when I did this in the context of an open market interactive trading platform. I had such a bad experience of interaction on the trading floors when I left sincerely written reviews, which I now usually write in my job specifications: “Any attempt to influence my rating in feedback will lead to a decrease in points.” This, in fact, leads to a decrease in the number of applications that I get for any particular job, but I believe that thanks to this I will hire people, ultimately, at a higher level.



God forbid what happens if I leave a sincere assessment in the corporate feedback scenario.



Indeed, this week I had the opportunity to evaluate a technician who, I felt, was damn good . He just hit me. He worked in a big stupid corporation and was well versed in my problem. He lingered after work (and did not demand payment for this) until my work was done, and put everything in order.



He was so good that I even decided to call this company to see if his manager knew how great this specialist was. Then an online survey took place, and I did what any normal person would do, who was absolutely fascinated by a specialist who had done much more than expected to correct the situation that was tangled up first by the company; I rated the quality of service with 9 points out of 10 .



Is that good and right? Not. In fact, as I learned later, 9/10 is the required rating for such specialists, which must be obtained from the feedback survey in order not to fail. It represents the absolute minimum. The guy was new to this job, and therefore did not say anything to me. He has not yet learned the established order to ask me "when I have a survey, I need you to give me 10 out of 10, OK?".



The logic (or lack thereof), operating in a company with a rating feedback system, in which the minimum number of points required to “pass” an employee is 9 out of 10, is simply ridiculous. There is no place for useful information. This is the logic of the "typical" town of Lake Wabigon in Minnesota , invented by American writer Harrison Keillor , "in which all women are strong, all men look good, and all children are developed above average."



Let us return to the Goodhart law. If you want to have honest feedback from your customers, to have material that can be used to improve business processes and improve business, then you need to separate these feedbacks from employee ratings.



Your employees will serve customers more diligently if they do not worry about retrieving 10 out of 10 on a follow-up survey.



Your clients will provide more honest, accurate and useful information if they do not worry that their opinion will lead to undesirable measures against the unlucky employee.



This may seem counterintuitive, but associating an employee score or — even worse — associating payment with these feedback mechanisms is akin to an old joke: “The beatings will continue until the morale improves.”



You will not get anything of value from those polls, apart from a rather dubious hierarchy, who is the best at teasing your clients to participate in manipulated and useless feedback. In this sense, such a procedure can be useful for determining the future CEO of a company, but outside of this, it only gives information noise.