Site testing: measure 7 times, cut once

Author: Ivan Krasyuk - Director of Client Network Development at Kokoc.com (Kokoc Group).

Visitors to any site are ordinary people who buy where the most convenient and attractive interface is. Some executives or marketers believe that they know users' preferences best of all. In fact, it is impossible to guess whether they will like the new site design or changing the color of the basket. See for yourself.
')
Suppose you need to choose how the elements will be located on one of the pages of the site. Which of the two options would you choose?


or


Which of the following options would you prefer here?



What kind of basket do you think is better?



or



In fact, whatever option you choose, you made a mistake at least once with a probability of more than 65%. Such an error for an online store may lead to a decrease in the number of orders. Therefore, in order for the site to work effectively, and its innovations do not accidentally scare users away, it is necessary to conduct testing.

What can be tested on the site? Everything:

• buttons (size, text, color);
• headings (size, text, location);
• calls to action;
• entry forms;
• default page options;
• personal recommendations;
• the location of the blocks;
• the mechanics of the elements on the site;
• photographs, drawings and other graphic elements;
• the presence and composition of individual blocks and elements;
• sorting goods and services;
• the order of the menu items;
• filters, search.

Almost anything can affect conversion: adding a product, changing a basket or an order form. You can not embed on the site any element without first conducting tests.

The principle of testing is as follows. For example, you have two options for pictures on the page. In this case, it is necessary to divide the site’s audience into two parts, each of which sees a page with one of the pictures. From which page the conversion will be higher, and that option should be taken. The ratio of parts in the division of the audience depends on the timing and risk preparedness. If results need to be obtained quickly, visitors are divided exactly in half. In this case, the conversion may fall. The risk is minimized if you show a new version of the page first 10% of users and gradually increase their share, watching the conversion. However, such a cautious approach will take longer.

The main types of tests:

A / A. This test is always carried out first in order to understand that the audience is divided evenly. You must create an identical copy of the page and separate 50% of the audience. Thus, when entering the site, half of the users see the old page, and the other half - the new one, which completely duplicates it. The conversion of both pages should be the same.

A / B. It is used when only one factor affecting conversion is considered, and there may be many options. For example, you decide which color of the buy button to choose: red, green, and blue. For testing, it is necessary to divide the site's audience into 3 equal groups and show them the pages with buttons of different colors, each group by color. Further, having studied the behavior of users, choose the most conversion option.

MVT. A multivariate test is performed if it is necessary to test different variants of several factors. Let's say you need to understand in which order it is better to place the “buy” button and a video clip. You have selected 3 button color options (red, green and blue) and 2 position options (video above button, video below button), that is, 6 different page options are obtained. Next, you need to divide the site visitors into 6 equal parts and see which of the combinations gives the greatest conversion. For example, the video above and the red button.

There are five stages of effective testing. Consider them on the example of an online store of mobile and digital technology.

Identify hypotheses and determine macro and micro conversions (business goals).
At this stage, analysts, along with business owners and marketers, find weak points on the site, that is, the places where the audience leaves. Experts identify hypotheses (reasons for low conversion), from which they choose the highest priority, that is, the problem, the solution of which will bring more profit.

Pre-test analysis with the economic potential of the test. Using Google Analytics data, assumptions are made about how much a conversion can grow if you change the selected item. The forecast is expressed as a percentage of conversion, cash profit and turnover.

Formation of variations in the representation of the influencing factor. Together with designers and programmers, analysts think through and create several variants of the element under test.

The testing process with a regular assessment of the significance of the results. Testing can be done using various software, for example, Google Content Experiments, Visual website optimizer or Optimizely. GA tester differs from other programs in that it is built into GA and is free. Visual website optimizer or Optimizely cost from $ 50 to $ 500 per month, depending on the amount of traffic and the number of tests per month.

Post-test analysis with economic justification and assessment of reliability. According to the test results, analysts identify the most conversion option and explain what results the introduction of this element will bring to the business.

Consider the stages of testing on the example of an online store of mobile and digital technology. The business goal of the test in this case is to increase the number of orders. According to Google Analytics, the conversion rate for site users who viewed reviews is 1.5 times higher than the average for the site. At the same time, the opinions of other buyers are very difficult to find on the product page and there is no clear indication that the diamonds from the rating are the result of user evaluation.



The priority hypothesis was the statement that a new product card page contributes to an increase in the number of orders. In order to draw attention to the reviews, it is necessary to modify the block with the rating, as well as its location. Using Google Analytics data, Kokoc.com analysts have suggested that this will result in a conversion increase of 0.06%. It was proposed to add a header to the block, indicating that it is formed based on the opinions of users, and next to post a link to them. For testing were developed 3 options.


Option number 1.


Option number 2.


Option number 3.

A multivariate test was conducted during the week using the Visual website optimizer (VWO) program. For completeness of the analysis of the results, the following objectives were additionally configured in VWO:

• Objective number 1 - any interaction with the page;
• Goal number 2 - click on the buy button;
• Goal number 3 - click on the button "Order in 1 click";
• Objective number 4 - the opening of the loan calculator;
• Goal number 5 - click on the button "Buy on credit";
• Goal number 6 - go to the page with product reviews;
• Objective number 7 - order online;
• Objective number 8 - click on the image of the goods.

According to the results of testing, the number 1 option won, the indicators of which were the highest by the number of orders, clicks on the “buy” button and transitions to the page with reviews. The VWO report showed that visitors who saw an alternative page opened up reviews on the product by 4.1% more often than those who saw a regular page (Fig. 12). Also, the indicators of option # 1 by clicks on the "buy" button above the control option by 0.23% (Fig. 13), and by order - by 0.16%.


Screenshot report Visual website optimizer review reviews.


Screen report of the Visual website optimizer by clicking on the "buy" button.


Screen report of the Visual website optimizer on orders.

According to VWO, 249,439 unique users visited the product page during the reporting week. Thus, when approving an alternative version of the page and an average order of 12,000 rubles, analysts predicted the following increase in turnover:

• minimum: +742 928 rubles per month (~ 8 915 143 rubles per year);
• expected: +19 322 687 rubles per month (~ 231 872 254 rubles per year);
• maximum: +37 902.447 rubles per month (~ 454 829 366 rubles per year).

What growth will be in the end depends on many factors: from the weather to the economic situation in the country.

In addition to the economic justification of the results, analysts give an assessment of their reliability and significance. For example, if you test two versions of pages on two users, the results will not be true. The more views, the more reliable testing.

When analyzing the results, it is also necessary to take into account that the conversion is different: macro and micro. Macro conversion is a buy or a call. Under the micro-conversion can be understood to view the item card, use the search, add it to the cart, click the button “buy on credit”. It is also important to consider indicators such as average check and total income. For example, the conversion could not increase, but the average bill increased and, as a result, the total income increased. The reverse situation may also be the case; an increase in conversion is accompanied by a decrease in the average check, as a result of which the total income may also decrease. A truly objective picture is shown by the cumulative analysis of all parameters, macro and micro goals using different segmentation.

Testing is an indispensable tool when introducing any innovations or changes to the site. Most of the 10 tests, only 4 give a positive result, and 6 do not give any result or reduce the conversion. In this regard, testing should be carried out continuously. It is best to create a calendar of tests, for example, a quarter. Moreover, if a test did not give any results today, it can be put at the end of the calendar and re-run. During this time, the site has changed, so the results may be different, the test may work. The more tests, the more effective your site.