Machine learning for managers: the sacrament of separation

Introduction

Once again working with a company doing a project related to machine learning (ML), I noticed that managers use terms from the field of ML, without understanding their essence. Although the words are pronounced grammatically correctly and in the right places of sentences, however, their meaning is no clearer to them than the purpose of sepules , which, as is well known, are used in sepulkariah for sepulenia. At the same time, it seems to timlids and simple developers that they speak the same language with the management, which leads to conflict situations that complicate the work on the project. So, this article is devoted to facilitation techniques (from Latin: simplification or facilitation) of communication between developers and management, or how to explain the basic ML terms in a simple and intelligible way, thereby leading your project to success. If this subject is close to you - welcome under cat.

Esthetu note: Sepulki, sepulkarii and sepulenie - terms used by the genius Stanislav Lem on the 14 journey of Iion the Quiet.

Project start

The ML project should start with the validation of the validation metric. Sounds scary, doesn't it? Let's start the explanation. Legitimation (in Russian from Latin it is legitimization) is simply the coming to an agreement of the parties, recorded in writing and endorsed - preferably, of course, also in writing. Parties are both donor and project management, as well as its performers.

We now turn to validation . The ML programmer usually has experience in writing validation code and, when tracing, sees true and false returned to it. But how to explain this concept to a manager who does not deal with code? Let's use this simple life example.

Imagine that you pass by the market and see: sell peaches. The seller says to you: “Bury! Haroshy parsik, svezhiy, juicy such, you will not regret it ”. However, you look and see: in one place it is spoiled. You say: “Well, where is he good? that's rotten. ” Seller offers half price. If you think: “It is possible to cut the spoiled, it is only a quarter, it seems to be profitable” - and you buy it, then in the ML language validation occurs and the peach (in the ML slang - sample) is considered valid. In case, if you think that it is possible to find the best instead of spoiled elsewhere, then there is a disability, and you recognize the peach as not valid.

It turns out that there is nothing difficult in validation, and we all do validation every day, recognizing one thing as good, suitable for oneself or disability, and recognizing something else as bad, worthless.

Estetu note: Suddenly, Jourdain was surprised to learn that all his life expressed prose (s). Moliere, a bourgeois in the nobility.

Finally, it remains to explain what a validation metric is . Let's think about why we decided to buy a peach from the previous example?

• it is cheap enough (price <threshold value)
• it is quite ripe (ripeness> threshold value), but not overripe (ripeness below the 2nd threshold value)
• it is of normal size, that is, its value is in the category “normal” (all categories: too small, small, normal, large, huge)
• it is not spoiled enough (the area of ​​the rotted and spoiled areas is less than the threshold value)

All this listed above is an example of a validation metric consisting of 4 categories in this example. In the simplest case, when a peach satisfies all the criteria at once, it will be recognized as valid and purchased.

Now it becomes obvious why it is so important to agree from the very beginning how exactly validation will take place, according to what number of parameters and what threshold values ​​all interested parties will arrange. A special section may take descriptions of actions in case of partial compliance with the conditions.

Naturally, each ML project, depending on its subject area, will have its own validation metric. The document fixing the validation metric is as important for the ML project as the constitution is for the state.

Only after the project finally has a fixed document regulating the validation metric and is available to all project participants, does it make sense to write its code. The validation code is the heart of the project and its quality must be immaculate, any error in this part is very likely to lead to the collapse of the entire ML project as a whole.

Mystery computing accuracy

The most important indicator of the current situation in the project for management is accuracy . How can a manager explain in a simple way what it is and what actions need to be performed to calculate it?

First we need to explain what a validated sample is. In our example, this is when we bought not one peach, but a ton. We sit down ourselves or hire workers and they sort out peaches in 2 containers. On the containers of the inscription: X (good) and P (bad). The work done on peach bulkheads is the creation of a validated sample.

How to explain why a validated sample is needed? Imagine that you have a younger sister and you want to teach her to choose peaches. You take it to the market and say: "Learn, see how I do." When it seems to you that she has already learned, you want to test her skills. How to do it? You create a control sample, i.e. take from containers, for example, 100 already peeled peaches from each container and quietly stick secret stickers on them to know from which container they were taken, but it would not be known to her sister and offer her to decompose them into new empty containers. The percentage of your sister's choices with the secret stickers matches the indicator accuracy. In other words, accuracy is the objective meaning of how much your sister can be trusted with the choice of peaches for you. 100% means that it is your copy and it does everything exactly as you do. 0% - that her opinion is exactly the opposite of yours.

Estetu note: Yes, you are right, over time, peaches may begin to deteriorate and you need to consider that their suitability will have to be reviewed from time to time. And in computer data this also happens, for example, with such a characteristic as “relevance”.

And now let's look at 4 ML performance indicators, which can be confused. These are true-positive (TP), false-positive (FP), true-negative (TN) and false-negative (FN). The first half of a word means a match (true) or a mismatch (false) of your sister's opinion with a secret peach sticker. The second half simply means the container in which your sister threw a peach (X-good - positive, P-bad - negative). And two words together is just the number of peaches in such a category.

In addition to accuracy, 3 auxiliary indicators are also used, these are precision (accuracy), recall (sensitivity) and f1_score.

Precision shows% of matches with your opinion of peaches thrown into container X (good). 100% means that all the peaches that you recognized as suitable are recognized as such by your sister. A lower value means that those that you have declared unfit have fallen into container X. The indicator is important when it is critical for the business that the unsuitable peaches do not fall into the suitable ones, but if the fitter is found to be mistakenly unfit - that’s okay.

Recall shows the relationship between the correctly selected fit peaches (TP) and to the sum of this value with good peaches mistakenly considered unfit (TP + FN). 100% means that your sister never throws fit peaches in a basket with bad ones and is the opposite of Precision. This indicator is important when it is necessary for business that suitable peaches fall into a container as unfit as possible.

F1 score is a synthetic indicator that combines the benefits of precision and recall. Its great importance indicates the balance of learning and says that just as good peaches do not fall into the basket of bad ones, the bad ones do not rush to good ones.

Note: This indicator is a harmonic average between precisions and recall and is calculated by the formula:

f1_score = 2*(recall*precision) / (recall + precision) 

Often the question arises: why should an ML project manager know and understand all these indicators so deeply? Answer: this is important for business. As a manager of a dairy farm, you need to know what milk yield is and what formula they are considered in. As a farm manager, you need to know what yield is and how it is calculated. Yes, the manager may not understand how exactly cows are milked, how they calve and how to treat them, but to understand the main business indicators of the project is the key to business success.

Results

All of us, participants of ML projects, are doing good and necessary work. Which of us, being a student, did not dream, turning over potatoes, tomatoes and cabbage on a collective farm, so that robots would do it for him, and not man (s). We make a fairy tale come true and let our projects be successful. I will be glad if this article will help to make a small contribution to the successful start of ML projects.

If this article seems useful to you, write in the comments and I will do the 2nd article on how to explain the additivity and generalization to management, these are the pillars of a proper, useful ML project.