The line between the cargo cult and the evolution in learning success

Entrepreneurs are very enthusiastic about the verification tools for future profits. The idea of ​​A / B testing fell into fertile soil. You no longer need to puzzle over the color of the button, you can immediately test and choose the best.

Let me remind you how this is done: you create two pages, the difference of which will consist only in the color of the “checkout” button. Then you, randomly, half of the visitors to the site show page number 1, and the other half - page number 2. As a result, on one of the pages, users will click the checkout button more than the same number of other users on another page.

Does it follow from this that the color of the button affects the quantity of completed orders? Let's do a mental experiment. We will have two teams, five people each. One team will be in red T-shirts, the other - in blue. Each member of the team will throw up a coin and write down what has happened: heads or tails. Let everyone throw a coin, say, three times. After, we calculate the number of received “eagles” for each of the teams.
')
We will see that one of the teams scored more “eagles” than the other. Is it possible to conclude that the color of the T-shirt determined the winner? Should we wear a red (blue) T-shirt when we are going to earn more “eagles”?

We all know very well that if our teams would throw a coin an infinite number of times, the teams would have the same results. We know this only because of the understanding of the model, since there was no immortal and, at the same time, a great fan, everything is checked in practice.

The conclusion here suggests itself, but let's not be in a hurry with the conclusions. Imagine that you received a message stating that a group of scientists had developed an excellent technology for predicting the results of football matches. And in order to prove its revolutionism, the message to you is the outcome of the match, which will take place tomorrow. Of course, the result is correct. A week later, you again receive a message from the same “group of scientists” who, to finally convince you to purchase their product, again send you the correct outcome of the next match.

If you are surprised by their insight, then it has a simple basis. The first message with the text “Brazil will win Mexico” is sent to one group of people, and with the text “Mexico will win Brazil” to the second group of people. After the results of the Mexico-Brazil match become known, the second message with the next “prediction” is sent only to the group that was previously accidentally given the correct prediction.

Maybe this is a good way to persuade, but obviously a bad way to predict. Is there something similar in how we are now trying to verify the success of one strategy over another? Is it really worth looking for the relationship between the color of the button and the number of completed orders?

We can make an assumption: the color of the button can influence the decision to purchase. If this assumption is correct, the results of A / B testing are valid, and if not, then the test results are useless. On the face of asymmetry: we essentially gain, if we are right, and lose nothing, if not.

Under conditions of uncertainty, when we cannot be sure whether event A is connected with outcome B, and our testing costs are insignificantly small compared to the possible loss of profit, we, of course, will be right to test our system.

Can the same principle be transferred to learning success? What if we scrupulously record all the decisions and events that entrepreneurs take and are experiencing, then we will identify the characteristic patterns, and then apply them in similar conditions?

To understand whether this idea will work, you must first decide: can actions that have ever preceded success be repeated in a similar situation and also lead to a successful outcome? Isn't this the same mechanic that allowed a “group of scientists” to unmistakably guess the outcomes of football matches?

The actions of entrepreneurs who have brought them success have changed the market. In the literal sense of the word, if someone began to regularly receive money that he had not previously received, then someone began to receive less. The market favors one business model at the expense of others (not necessarily direct competitors). Yes, sometimes such a load on the market is distributed among many enterprises and it is difficult to feel the burden of someone else’s success, but it is.

Consequently, by selecting the key of profitable decisions for the lock to the distribution of goods on the market, if successful, we change the lock for our followers. But how then to explain the phenomenon of success of copycat, does this mean that every successful local analogue of a foreign service survived and brought profit to its founder only by accidental coincidence of circumstances?

Let us mentally isolate the minimal patterns of decision making from all the histories of the foundation and development of a company; let us present them as a chain of genes that form the bodies of companies in a changing market environment. Highlighting the same “genes” of successful companies and the same “genes” of bankrupt companies, we will be tempted to draw a simple conclusion about the recipe for success and failure. But, if we make a search for “successful genes” among companies-losers, then we will definitely find them there. Yes, some companies losers will carry the genes of "winners". Therefore, simply recreating “successful patterns” is not enough for success.

If the computability of a successful sequence of actions is impossible and it is solved only by looking at the available combinations, then how can we explain the success of copied businesses? Include in the calculations failed copycat-projects and recognize the results as random?

Copycat-projects are the use of an already found whole and integral key of solutions, joint exploitation of the winning combination with the discoverer. Similarly, in nature, we see groups, populations, species. The evolution of living systems did not come up with a “successful gene” that could be pasted into any arbitrary genotype and, thus, increase the success of the future organism.

So it may be worthwhile to apply the same principle that I suggested in the case of the color of the buttons? Can a pattern occurring in successful startups clearly have a loss-benefit asymmetry? Is it possible to say that once in a similar situation, someone successful once made a certain decision, then I, having done the same thing, will either gain or not suffer a loss?

There is no doubt the assumption that the decision can be fraught with both positive outcomes and negative ones. The decision affecting the future well-being of the project cannot be subjected to A / B testing, since we have only one project and it cannot be divided into two, checked on the market and returned to its former state. Does this mean that the patterns of other people's experience are useless for us?

Far from it, we just need to look for not the philosopher's stone, but its opposite: something that turns gold into dust. Those. negative decisions, immediately followed by failure. So, at the decision-making level, based on the common patterns of failed startups, we will make a statistically sound choice between death and survival. And this is an adaptive, profitable strategy.