Code Review and Probability Theory

Not all programmers are familiar with the theory of probability. It would seem - well, what is the trouble here? Who studied what, universal geniuses do not happen. Teorver at a good level you need to know except in game dev, cryptography, well, maybe in any financial and statistical software. But no! Failure to understand some things can lead to poor results even in projects where it does not even smell. There is no magic, just the human brain incorrectly evaluates certain probabilities and, as a result, makes the wrong decisions.

So, imagine a certain programmer Vasya. He writes code. Well, because he is a programmer. Suppose Vasya is a good programmer and 75% of his code is error free. In fact, I, of course, am lying and hardly Vasya is such a guru, but again - imagine. For simplicity, we assume that for every 100 lines of its code 75 do not require revisions, and in 25 of them errors should be found and corrected. And here we decide how to do it. As always, there are a lot of opinions, someone for general unit tests, someone for expanding the QA department, someone for the Code Review of each commit, ignoring the budget and deadlines, perfectionists for everything at once, self-confident "pro" and greedy managers against all at once. Well, as always. But something must be solved. And here comes the idea to evaluate the resources for each type of activity and profit from it. And let's estimate the effectiveness, for example, Code Review.

An inexperienced calculation might look something like this: Vasya wrote 100 lines of code, he usually has about 75% of the correct code, 25% of the bugs. Let's make him spend the same amount of time on the Code Review - and get 2 times fewer bugs (12.5%). Well, because once spent 2 times more resources - it means 2 times less bugs.

But nothing of the sort! Vasya is Vasya. He wrote this code yesterday, and if he will conduct a Code Review of his code today, he will not find anything. Why? Because.
')
Well, OK. So, if Petya (about the same qualification) scans his code, he will reduce the number of bugs by half (to 12.5%). Well, because they are about the same qualifications, but still different people and it turns out we spend 2 times more resources on this piece of work, which means we get 2 times fewer bugs. Now let's see how it really is.

Probability theory says that if we have a system of two devices with reliability X and Y and in order for it to fail, the failure of both devices is necessary, then the overall reliability of the system is 1 - (1-X) * (1-Y). Those. for “devices” Vasya and Petya with the same reliability 75% total reliability will be 93.75%! See, the number of bugs decreased not to 12.5%, but to 6.25%! Those. by spending on the Code Review the resources of another person, even of a similar qualification, we reduce the number of bugs not twice, but four times! Cool. In addition, given that the Code Review is usually done by the best developers in the team (roughly speaking, having a reliability higher than the "average" 75%) and they spend much less time on it than the author of writing the code, we get even more impressive results.

Why am I doing this?

Now, better imagining what we can get at the output, having invested a number of resources in the Code Review, we can more easily make decisions on the use of this and / or other code quality enhancement mechanisms. We can already operate with numbers, and not unfounded "they do everything this way, this is right!" Against "yes, well, and so there is no time!" For an adequate guideline, a numerical argument will always have more weight than just throwing out loud phrases.

A short conclusion for those who are too lazy to read the whole article.

Applying a Code Review (if you don’t use it yet) will help more than it first seems.