How I implemented A / B testing in PHP
There was a task to conduct a split test of our online store. Googled on this topic. I found a couple of services, but for one reason or another they did not suit us.
For example, the use of testing through google analytics did not fit, because We use Yandex. Metrics for statistics and do not want to embed another analytical system into the site. And besides, I understand that there is a problem with Google when testing dynamic pages or some of their elements.
Other services, such as abtest.ru (which, by the way, for some reason does not work) or similar ones, are also not suitable, since there you can change only the appearance of something. And we, for example, need to test the site for the withdrawal / non-withdrawal of this or that information from the database (well, for example, whether to display a brief description of the goods in the list of goods or else to display specifications, etc.).
I decided to try to implement A / B testing on my own, by adding a little php-code to the site (the online store is self-written, in php).
It is necessary to check how the change of one or another element of the site will act on the conversion. Those. we divide all visitors to the site conditionally into “visitors-A” and “visitors-B”. Divide naturally equally, i.e. every first is “A”, every second is “B.” Depending on whether "A" or "B", we show him the corresponding version of the site element. At the end we fix whether the visitor has made an order or not
')
I will not describe the code, but only briefly explain what and how I did:
1. At the entrance to the site, each user in the session (say $ _SESSION ['split']) records the value "A" or "B". Every first - “A”, every second - “B”. For this purpose, the database has a special “last visitor counter”, which contains data on the last visitor to the site. That is, the visitor came to the site, they recorded the value “A” in $ _SESSION ['split'] and the counter in the database also changed to “A.” When the next visitor visits the site, the script takes the counter value from the database, and if it is = “A”, then the “B” is recorded in the session for the visitor and vice versa. Those. thus, alternation of users is provided. By the way, the question is: can someone tell me how to implement the alternation of visitors without referring to the database, how do you know what value to assign to the visitor "A" or "B"? After all, you need to know what value the previous one had. Or you can not bathe and just assign randomly the 1st or 2nd option and this randomly? Will the visitors split equally?
2. After the user has recorded the value “A” or “B” in the session, now depending on this value, the user is shown either the first or the second version of the element under test. Those. A person came to the page where the element under test is located; immediately a check is made that is written in $ _SESSION ['split'] and depending on the result, the first or second option is shown.
3. Saving results. To do this, there are counters in the database for counting the results (for counting the number of orders by “visitors-A” and “visitors-B”. Ie, each of the two options “A” or “B” has its own counter. If a visitor has in $ _SESSION ['split'] the value “A” is placed with the order, then the counter “A” increases by 1. If the visitor “B” places an order, then the counter “B” increases by 1.
In fact, it turns out quite simple code. But it is scary. It seems that I did not take into account some subtleties and nuances of the A / B testing process and everything is much more complicated and deeper. Indeed, various disputable situations are possible. For example, how to account for visitors who visited the site, then left the site and returned again? Whether to show them when they visit the site again the version of the element under test that was shown to them during their first visit, or to re-consider this visitor in the queue of alternation. And here the question arises of using not the session mechanism, but the mechanism of cookies.
I wrote this post to share my implementation of the simplest A / B testing and at the same time listen to the opinions of experts on this issue. Maybe there will be some hints, comments, etc.
For example, the use of testing through google analytics did not fit, because We use Yandex. Metrics for statistics and do not want to embed another analytical system into the site. And besides, I understand that there is a problem with Google when testing dynamic pages or some of their elements.
Other services, such as abtest.ru (which, by the way, for some reason does not work) or similar ones, are also not suitable, since there you can change only the appearance of something. And we, for example, need to test the site for the withdrawal / non-withdrawal of this or that information from the database (well, for example, whether to display a brief description of the goods in the list of goods or else to display specifications, etc.).
I decided to try to implement A / B testing on my own, by adding a little php-code to the site (the online store is self-written, in php).
It is necessary to check how the change of one or another element of the site will act on the conversion. Those. we divide all visitors to the site conditionally into “visitors-A” and “visitors-B”. Divide naturally equally, i.e. every first is “A”, every second is “B.” Depending on whether "A" or "B", we show him the corresponding version of the site element. At the end we fix whether the visitor has made an order or not
')
I will not describe the code, but only briefly explain what and how I did:
1. At the entrance to the site, each user in the session (say $ _SESSION ['split']) records the value "A" or "B". Every first - “A”, every second - “B”. For this purpose, the database has a special “last visitor counter”, which contains data on the last visitor to the site. That is, the visitor came to the site, they recorded the value “A” in $ _SESSION ['split'] and the counter in the database also changed to “A.” When the next visitor visits the site, the script takes the counter value from the database, and if it is = “A”, then the “B” is recorded in the session for the visitor and vice versa. Those. thus, alternation of users is provided. By the way, the question is: can someone tell me how to implement the alternation of visitors without referring to the database, how do you know what value to assign to the visitor "A" or "B"? After all, you need to know what value the previous one had. Or you can not bathe and just assign randomly the 1st or 2nd option and this randomly? Will the visitors split equally?
2. After the user has recorded the value “A” or “B” in the session, now depending on this value, the user is shown either the first or the second version of the element under test. Those. A person came to the page where the element under test is located; immediately a check is made that is written in $ _SESSION ['split'] and depending on the result, the first or second option is shown.
3. Saving results. To do this, there are counters in the database for counting the results (for counting the number of orders by “visitors-A” and “visitors-B”. Ie, each of the two options “A” or “B” has its own counter. If a visitor has in $ _SESSION ['split'] the value “A” is placed with the order, then the counter “A” increases by 1. If the visitor “B” places an order, then the counter “B” increases by 1.
In fact, it turns out quite simple code. But it is scary. It seems that I did not take into account some subtleties and nuances of the A / B testing process and everything is much more complicated and deeper. Indeed, various disputable situations are possible. For example, how to account for visitors who visited the site, then left the site and returned again? Whether to show them when they visit the site again the version of the element under test that was shown to them during their first visit, or to re-consider this visitor in the queue of alternation. And here the question arises of using not the session mechanism, but the mechanism of cookies.
I wrote this post to share my implementation of the simplest A / B testing and at the same time listen to the opinions of experts on this issue. Maybe there will be some hints, comments, etc.
Source: https://habr.com/ru/post/177959/
All Articles