Almost fired. How I built the Yandex analytics department

My name is Alexey Dolotov, I haven’t written 10 years on Habr. Partly, the fact is that when I was 22, I began to build the Yandex analytics department, then I ran it for seven years, and now I have invented and built the Yandex.Talents service. Profession analyst gives a lot of opportunities. The main thing is to start correctly - for example, in the School of Managers , a recruiting in the direction of analytics is currently underway.

I decided to tell how my career developed, and to give some advice to those who want to “ignite” in this profession. I hope my original experience will be useful to someone.

The only semester of the university and the beginning of a career

By the time I entered college, I programmed well, even wrote my shareware product (a word from the past). It was a laser disc cataloger. Winchesters were still small, they did not fit everything, so people often used CDs and DVDs. The cataloger reads the disk file system, indexes it and collects meta-information from files, writes all this into the database and allows it to be searched for. On the first day, the product was downloaded by 50 thousand Chinese, on the second day, a crack appeared on Altavista. And I thought I did a cool defense.
')
I entered the ITMO University in St. Petersburg, but after one semester I decided that I could program as much, I learned faster in the process, and so I left for Norway to freelance. When I returned, I organized a web-studio for two with a partner. He was responsible rather for business and documents, I - for everything else, including the technical part. At various times we have worked up to 10 people.

Yandex in those years conducted the so-called client seminars, one of which I allegedly entered as a journalist. There were including Andrei Sebrant, Zhenya Lomize, Lena Kolmanovskaya. After listening to what they say, I was impressed with their out-of-the-box thinking. The best way to approach someone in terms of professionalism is to start working with them. Therefore, at that moment - I was 19 or 20 years old - rethought all my life, I decided to quit a not very successful web-studio and move from St. Petersburg to Moscow to get into Yandex.

I could not do it immediately after the move. The department, which I, for some reason, persistently tried to settle in, knew that I had penetrated the above-mentioned seminar with cunning and even tried to get a certificate of the completed course on Yandex.Direct. By the way, this certificate could not be given to me for a long time. Nobody expected that the course would be held by anyone besides the main audience of the seminar. This story seemed strange to my future colleagues, and they didn’t take me to Yandex.

But in Mail.Ru they quickly took five interviews in two days. It was the way - after the move I was running out of money. I was responsible for all search services, including GoGo and go.mail.ru. But after a year and a half, I still went to Yandex as a manager for koldunshchik (issuing elements that answer a user's question right on the page with search results). It was the end of 2008, about 400 people worked at Mail.Ru, and about 1,500 in Yandex.

Yandex

It must be admitted that at first Yandex failed. Four months later, I was offered to look for other options in the company. In fact - fired. I had some time to search, but if I hadn’t found anything, I would have had to leave. Until then, I did not work in a really big company with a complex project management structure. Not oriented, not enough experience.

I stayed, got a job as an analyst of communication services: Photos, Ya.ru, but the most important thing is Mail. And here I was very useful combination of managerial skills (walk around people, agree), grocery (to understand where the benefits are, what users want) and technical (to apply programming experience, independently process the data).

We were the first in the company to start building cohorts - to study the dependence of the outflow of users on the month in which they registered. First, we fairly accurately predicted the size of the audience on the basis of the model obtained. Secondly, and more importantly, it was possible to predict how different changes will affect the main indicators of the service. Previously, Yandex did not do this.

Once, Andrey Sebrant came to me and said, “You are good at it, now we need the same thing on the scale of all Yandex. "Do the department." I replied: "Good."

Department

Andrei helped me a lot, sometimes he said, you are an adult guy, figure it out. No typos, this is also help. I really needed more independence, and I started doing everything myself. When there was a question for the management, I tried to think first: what would the manager have answered this question for me? This approach helped develop faster. Sometimes, because of the big responsibility, it was just scary. The turning point came: I am from a person solving a problem, became responsible for the development of a large piece of processes. The number of services and the services themselves grew, they needed analysts. I was actively engaged in two things: hiring and mentoring.

They often came to me with questions that I did not know the answers to. Thus, I learned to solve almost any problem with some accuracy, based on a very limited amount of data. It's like in “What? Where? When? ”, Only there it is necessary to give the correct answer, and here the correct answer may not be at all, but it is enough to understand in which direction to dig. I began to struggle with many cognitive distortions (perhaps the most popular among researchers and analysts — confirmation bias, the tendency to confirm my point of view); I developed “invariant”, “quantum” thinking. It works like this: you hear the condition of the problem and immediately imagine all possible and impossible solutions, automatically “resolving” these branches and understanding what minimum hypotheses you need to check in order to “solve” as many of the most likely branches as possible.

I also taught the guys what I did not know myself. I learned the first basics of statistics through interviews that I conducted. Then he began to teach leadership, although he just became a leader. It seems that there is no greater incentive in something how to figure out how to explain it to someone else.

Partisans

I began to help analysts grow: I told everyone that I would work with him, and he should work independently with the service team. At the same time I asked uncomfortable questions. An analyst comes to me and talks about the tasks he is doing now. Next dialogue:

- Why are you doing such tasks?
- Because they asked me.
- What are the most important tasks for the team now?
- I do not know.
- Let's not do what was asked for, but what the service needs.

The following dialogue:

- They do this.
- Why don't they do it? What they did not consider, what they forgot to think?

I taught the guys not to take on tasks until they understand what really hurts the customer. It is important together with the customer to “rehearse” the scenario of how the analytics result will be used. It often turned out that the customer did not need what he initially asked. To understand this is the responsibility of the analyst.

Such is the philosophy of "good partisanism" or "partisan product management." Yes, you're just an analyst. But you have the opportunity to influence the course of the entire service - for example, through the correct formulation of metrics. To formulate metrics and goals on them is perhaps the main instrument of influence of the analyst. A clear and transparent goal, decomposed into metrics, each of which is clear how to improve, is the best way to send the team the right course and help it not to stray from this course. I promoted the idea that all my guys should interact cross-service and thereby form “hydrogen bonds” inside Yandex, which tended to be glued together at other points.

Search share

In 2011, we investigated the reasons for the change in the search share of Yandex - it was difficult to prove the influence of each specific factor, and there were a lot of them. One Friday, I showed Arkady Volozh the schedule, which I could not make up for a long time and finally made it. Then I came up with the “factor freezing method”, which made it possible to isolate the influence of browsers with a predefined alternative search. It read well that the proportion is changing precisely because of them. Such a conclusion at that time did not seem obvious: people have not used such browsers very often. And yet it turned out that the preset search greatly influences the situation.

In those days, the phase of my active communication with Volozh began: I began to devote more time to the search share. The concept itself appeared - the analyst of a share or “fakapochnaya” (abrupt changes in the share were often caused by someone's factories). It was then that Seryozha Linev appeared in the team, in the future one of the key analysts of Yandex. Together with Lyosha Tikhonov, another excellent analyst and author of Avtopoet, we helped Seryozha to grow and create around himself invaluable expertise in detecting and analyzing complex anomalies. Now, if any incident affecting a share occurs, the administrator on duty will immediately know about it with all the details. It is no longer necessary, as then, to be gathered by a dozen analysts and spend several days investigating the causes. We can say that now in this regard, we have the era of spacecraft, and then we dragged carts.

Arcadia was always very interested in the share. He began to call and write to me often when there were anomalies on the search share devices - even if I had no relation to the reasons for these anomalies. Perhaps he continued to call me because it helped. And I just knew who to call next.

By the way, Yandex has a mailing list for non-working questions, and when I asked someone to lend me a badminton racket, it was Arkady who was the first to respond.

Ilya

Probably, it is here that it is appropriate to tell how I, though not for long, worked with Ilya Segalovich. According to chronology, it was necessary to tell about this earlier: oddly enough, I worked with him, while still in Mail.Ru.

The fact is that the go.mail.ru search at that moment worked on the Yandex engine (only GoGo, another Mail.Ru project, had its own engine). Therefore, as the manager of search services, I was given contacts of several Yandex. For technical questions, I called either Tole Orlov or Ilya Segalovich. To my shame, I had no idea then who these people were. After hours, it was easier to get a phone call to Ilya, but during the day, vice versa. I wondered why he so rarely happens in the workplace, I thought - what kind of developer? But when he answered, he helped me very politely and meaningfully in the shortest possible time. Therefore, first of all I called him.

Later I found out who Ilya was, I even played badminton with him as part of a large group of colleagues. Getting settled in Yandex, I tried to remember what I managed to tell him. Ilya, by all appearances, was an ordinary good man without any star disease.

There was a case, we ran into Elijah in the elevator. Ilya in a wild excitement makes me elevator pitch, showing the screen of his phone: “This is the future!” During the time in the elevator, it’s impossible to figure out what he means. But you notice how much a person burns, and you do not understand whether this is madness or genius. Probably both.

There are people whose ideas live in me and make me better. Ilya is one of them.

Talents

Many of the current analytical departments in Yandex are now headed by my guys. There were several reasons why I decided to switch to something else after seven years of managing the department.

First, Yandex has become a group of companies, and the need for centralized analytics has disappeared. Secondly, there was too much administrative work with such a large department. And thirdly, I wanted to make decisions and take full responsibility for them. I wanted to come home one day and tell my wife: "This is what I did."

Therefore, I created the Yandex.Talents service. We try to rethink job search and hiring employees. We are now taking only the first steps, but I see great potential in us. The classic idea of ​​a job board in an era when machine learning everywhere and drones drive through the streets seems outdated. It's time to start using smart algorithms to help both job seekers and employers.

I used to always explain to people in services how to do their work, considering that these arguments are based on analytics and my expert opinion. But the work on Yandex.Talents has shown that I am very often wrong. The truth is born between people - a simple statement, which, however, must be felt. In addition, the creation of a startup required to immerse themselves in the business, and now I believe that first of all the product analyst should study the financial model of his product. If you do not understand what your core business indicators are made of, how can you help the team achieve them?

What you need cool analyst

The analyst should be able to do a lot, but there are two main skills that allow you to really “ignite”.

First of all, a phenomenal ability to deal with cognitive distortions is required. I advise you to read the article "List of cognitive distortions" on Wikipedia, an interesting and useful reading. All the time you catch yourself thinking how this list is about us.

And secondly, no authority can be recognized. Analytics - it's about arguing. You first prove to yourself that you are wrong in your conclusions, and then you learn to prove that someone else is wrong. On one of the days of August 2011, the Yandex portal worked intermittently for a while. It was Friday, and next Monday it was Hural , who led me. Arkady came, cursed for a long time. Then I took the floor: "Arkady, now I will begin to hural, perhaps." He says - no, the hural will not, let everyone go to work. I replied that I would not let the company work all week in this mood. He immediately agreed. And we spent hural.

These qualities will be useful in other areas, especially if you are a leader.