📜 ⬆️ ⬇️

An example of convenient organization of split (A / B) testing on the server side

Hello, dear habravchane!
I would like to share with you some experience of organizing split testing on a project with which I had to work, I also ask you not to judge strictly, since this is my first article and you should not expect a really huge profit.
As we all know, split (A / B) testing on the site is an experimental approach with the aim of increasing the conversion of a process on a resource by displaying different content to different groups of users.

The goal of the task was to organize a fairly complex testing of different content options (both static and dynamic), with possible unlimited number of experiments and options on each resource page. After some analysis of the existing ready-made solutions to the task, we managed to find several options - these were GWO (no longer exists as a separate service, integrated into GA), a solution based on nginx and Visual Website Optimizer . But not one of them did not fit for a number of reasons: external services transfer rendering to the client side, which can be very uncomfortable with complex tests, and nginx is far from everywhere, plus developers do not always have access to the web server configuration and also have a problem smearing logic. Because I made the decision to implement my bike option, based on the example of the implementation of the nginx module.
I also didn’t like the standard implementation tips (the remainder of making the last digit of the client’s IP address, etc.), so I decided to use checksums from the test key to determine the options to use absolutely any alphanumeric key, to create a huge number of options testing with any percentage. The choice of the hash function fell on MurmurHash2 , since it is characterized by a very good distribution and, what to hide, is used for hashing in the nginx module. The project I'm working on is implemented in php, I recommend using this extension to implement the hash function.
Now let's touch on the implementation. To determine the value of the test variant for a specific experiment, I implemented the SplitTestModel class, using it as a singleton:
class SplitTestModel { public static function getInstance($key = ''){...} private function __construct( $key ){...} public function getSplitVariantByExperiment( $experimentName ){...} } 

I carried out the information about each experiment to the config, which is, in my implementation, an array of the form:
 array( 'Experiment1' => array( 'Variant1' => 50, 'Variant2' => 50 ), 'Experiment2' => array( 'Variant1' => 15, 'Variant2' => 35, 'Variant3' => 12, 'Variant4' => * ) ); 

Where the top level key is the name of the experiment, the enclosed keys are the names of the options, and the corresponding values ​​are the percentage of impressions, '*' means all remaining percentages.
As it may have already become clear, the key is hashed using MurmurHash2. In the example given in the first experiment, with hash values ​​from 0 to 50% of the maximum possible hash value, the user will receive a variant value equal to 'Variant1', from 50% of the maximum possible value up to 100% - respectively the value of 'Variant2'.
The convenience of the method lies in the fact that now, anywhere in the code, for a specific user we can get its variant values ​​for any of the experiments and, depending on the variant, give this or that content to any client of any part of the page. Usage example:
 $variant1 = SplitTestModel::getInstance($user_id)->getSplitVariantByExperiment('experiment1'); $variant2 = SplitTestModel::getInstance($remote_addr)->getSplitVariantByExperiment('experiment2'); 

Everything is extremely simple!
Thank you very much for your attention, and once again I will ask you not to judge strictly. If anyone would be interested to gladly send the source code of the class.

UPD:
reference to class implementation on github

')

Source: https://habr.com/ru/post/169777/


All Articles