Web analytics: analyze it! Part 6. Action
Part 1. Introduction.
Part 2. Data collection.
Part 3. Basic metrics.
Part 4. From statistics to analytics
Part 5. Divide and think
The most active web analyst Evangelist Avinash Koshik in his blog pays considerable attention to topics related to the organization of work: how to convince the authorities of the importance of analytics, and most importantly - how to get the company to use the data. The fact that they contain important information for the business seems to be beyond doubt, but even if the management agrees with this, it usually ignores the analyst’s conclusions. As a result, measurements on the site are not accompanied by an analysis of changes in indicators, the site itself is made “by the eye of the boss”, and the statistics are used only to draw beautiful graphs and general estimates, such as attendance for the month. Koshik calls it HiPPO, or “MSABC” - the opinion of the most highly paid employees. Taking into account the fact that every employee has an opinion on the site’s account, oh how hard it is for an activist.
In this situation, it is very easy to make a mistake: for example, by insisting on experimental measurements, to obtain changes for the worse and to dwell on this, being ashamed of the disillusioned views of the authorities. Excessive bluntness will also hurt (you shouldn’t start by swearing at the director who insisted on the changes that led to the problem) and, of course, lies (if the experiment weakened the site’s performance, the worst thing to do is falsify the results!). Stories about analytics-oriented companies (like Amazon) will not help the situation either: the usual answer in this case is “you understand, everything is different for us ...”. We need to start with the brightest and most reliable steps. Of course, everything is much simpler if the boss is your ally, or you are your own director. In any case, who is forewarned is armed, and difficulties along the way are not a reason to stop in the middle.
')
Before you rush into battle, you should make a plan. At the head of his set goals pursued by the site. In this case, it is necessary to focus not on what is declared by the creators of the site or the company's management, but on what users can actually see on the site. If the goal is declared “to inform users about new products”, and on the website information about them appears with a six-month delay, you can safely delete the goal. However, do not overwork: as long as you can safely list goals.
The next step is to choose numerical indicators for assessing the achievement of goals and create a chain of business requirements for the measured indicators of the site.
For example, for an online store, the chain might be:
Creating such chains is very important. They are needed for two purposes: firstly, you will be able to identify important metrics to be measured, and secondly, you will immediately establish their connection with the goals of the Great Management. Now you can speak his language!
When you drew this ladder (by the way, refine it in the course of work: who said that the first version should be good right away?), It's time to move on to measurements. Launch Google Analytics, look for familiar metrics, segment, think. The first time, if your experience with Analytics is small, you will have to get used to it, set it up, use it until it becomes familiar to you, and most importantly, you can quickly and correctly take the required readings.
After that, measure your metrics. Just measure, watch them, try to compare changes in the values of external circumstances. Once you become this super-guru, the real work will begin: you will have to offer, monitor and test on the site in order to improve these indicators. Without this, all your work does not make sense: the goal of analytics is not to make beautiful reports with numbers and graphs, but to improve the work of the site, increase profits, and make users happy. To do this, you need not only to be able to work with Analytics, but also to orient yourself in search engine promotion, interface development, web marketing - of course, you can also work in a team with other specialists.
All changes. which are made on the site, you need to check: they have to improve the performance of the site. The beauty of web analytics is that you can check any change quickly and cheaply: just offer it to users and see what changes! In most cases, this is highly recommended: there is usually room for improvement in the most unexpected places. In the same way, you can check other people's ideas (you can even try to implement the “strange” ideas of the authorities - but do not be surprised if they work well against your expectations!).
For testing, you can add a site engine or use third-party solutions. I recommend to learn and use Google Website Optimizer . It allows you to conduct comparative testing of old and new versions of pages, calculate changes in metrics and check the accuracy of measurements.
Reliability, by the way, should be checked for any measurements related to the comparison of values. The fact is that we are always dealing with a limited sample, which means that the measured value may differ from the true one. The smaller the sample, the greater the possible deviation. Therefore, if the sample is too small and the measured value has changed little, there is no certainty that it has actually improved. How to be?
There is a fairly simple rule to determine the sufficiency of the sample. The real value with a probability of 95% is in the range from X – 2√ (X) to X + 2√ (X) .
For example, we are trying to figure out which of the two ad banners is more clickable. One was shown 1500 times, of which 70 clicked. Another was shown 700 times, 30 users clicked on it. It would seem that the CTR of the first 4.7%, and the second 4.2%. The first is better? Is not a fact.
Calculate the possible real number of clicks. 70–2√ (70) = 53, 70 + 2√ (70) = 87. Possible CTR - from 3.5% to 5.8%! Confident to be too early. Calculating confidence intervals in a similar way with increasing impressions and clicks, you will see that they will decrease until they no longer overlap. Only then can we say with confidence that one of the options is better! Probability theory has always been among the best friends of the analyst.
This is where the first part of my cycle ends. In it, I tried to collect the most important milestones for the analyst, without pretending to complete disclosure of the topic. The next part will most likely be more specific. However, I strongly recommend that you continue researching this topic right now, and here is what I can recommend:
Part 2. Data collection.
Part 3. Basic metrics.
Part 4. From statistics to analytics
Part 5. Divide and think
Knockin 'on Heaven
The most active web analyst Evangelist Avinash Koshik in his blog pays considerable attention to topics related to the organization of work: how to convince the authorities of the importance of analytics, and most importantly - how to get the company to use the data. The fact that they contain important information for the business seems to be beyond doubt, but even if the management agrees with this, it usually ignores the analyst’s conclusions. As a result, measurements on the site are not accompanied by an analysis of changes in indicators, the site itself is made “by the eye of the boss”, and the statistics are used only to draw beautiful graphs and general estimates, such as attendance for the month. Koshik calls it HiPPO, or “MSABC” - the opinion of the most highly paid employees. Taking into account the fact that every employee has an opinion on the site’s account, oh how hard it is for an activist. In this situation, it is very easy to make a mistake: for example, by insisting on experimental measurements, to obtain changes for the worse and to dwell on this, being ashamed of the disillusioned views of the authorities. Excessive bluntness will also hurt (you shouldn’t start by swearing at the director who insisted on the changes that led to the problem) and, of course, lies (if the experiment weakened the site’s performance, the worst thing to do is falsify the results!). Stories about analytics-oriented companies (like Amazon) will not help the situation either: the usual answer in this case is “you understand, everything is different for us ...”. We need to start with the brightest and most reliable steps. Of course, everything is much simpler if the boss is your ally, or you are your own director. In any case, who is forewarned is armed, and difficulties along the way are not a reason to stop in the middle.
')
Plan to capture Paris
Before you rush into battle, you should make a plan. At the head of his set goals pursued by the site. In this case, it is necessary to focus not on what is declared by the creators of the site or the company's management, but on what users can actually see on the site. If the goal is declared “to inform users about new products”, and on the website information about them appears with a six-month delay, you can safely delete the goal. However, do not overwork: as long as you can safely list goals.
The next step is to choose numerical indicators for assessing the achievement of goals and create a chain of business requirements for the measured indicators of the site.
For example, for an online store, the chain might be:
For a social networking site, the goal may be “more registered users”, and the corresponding metrics will be traffic and the percentage of registered users.
Creating such chains is very important. They are needed for two purposes: firstly, you will be able to identify important metrics to be measured, and secondly, you will immediately establish their connection with the goals of the Great Management. Now you can speak his language!
To battle!
When you drew this ladder (by the way, refine it in the course of work: who said that the first version should be good right away?), It's time to move on to measurements. Launch Google Analytics, look for familiar metrics, segment, think. The first time, if your experience with Analytics is small, you will have to get used to it, set it up, use it until it becomes familiar to you, and most importantly, you can quickly and correctly take the required readings.
After that, measure your metrics. Just measure, watch them, try to compare changes in the values of external circumstances. Once you become this super-guru, the real work will begin: you will have to offer, monitor and test on the site in order to improve these indicators. Without this, all your work does not make sense: the goal of analytics is not to make beautiful reports with numbers and graphs, but to improve the work of the site, increase profits, and make users happy. To do this, you need not only to be able to work with Analytics, but also to orient yourself in search engine promotion, interface development, web marketing - of course, you can also work in a team with other specialists.
Check, do not trust
All changes. which are made on the site, you need to check: they have to improve the performance of the site. The beauty of web analytics is that you can check any change quickly and cheaply: just offer it to users and see what changes! In most cases, this is highly recommended: there is usually room for improvement in the most unexpected places. In the same way, you can check other people's ideas (you can even try to implement the “strange” ideas of the authorities - but do not be surprised if they work well against your expectations!).
For testing, you can add a site engine or use third-party solutions. I recommend to learn and use Google Website Optimizer . It allows you to conduct comparative testing of old and new versions of pages, calculate changes in metrics and check the accuracy of measurements.
Reliability, by the way, should be checked for any measurements related to the comparison of values. The fact is that we are always dealing with a limited sample, which means that the measured value may differ from the true one. The smaller the sample, the greater the possible deviation. Therefore, if the sample is too small and the measured value has changed little, there is no certainty that it has actually improved. How to be?
There is a fairly simple rule to determine the sufficiency of the sample. The real value with a probability of 95% is in the range from X – 2√ (X) to X + 2√ (X) .
For example, we are trying to figure out which of the two ad banners is more clickable. One was shown 1500 times, of which 70 clicked. Another was shown 700 times, 30 users clicked on it. It would seem that the CTR of the first 4.7%, and the second 4.2%. The first is better? Is not a fact.
Calculate the possible real number of clicks. 70–2√ (70) = 53, 70 + 2√ (70) = 87. Possible CTR - from 3.5% to 5.8%! Confident to be too early. Calculating confidence intervals in a similar way with increasing impressions and clicks, you will see that they will decrease until they no longer overlap. Only then can we say with confidence that one of the options is better! Probability theory has always been among the best friends of the analyst.
The end?
This is where the first part of my cycle ends. In it, I tried to collect the most important milestones for the analyst, without pretending to complete disclosure of the topic. The next part will most likely be more specific. However, I strongly recommend that you continue researching this topic right now, and here is what I can recommend:
Books
- Book Ashmanova "Optimization and website promotion in search engines . " Directly search engine optimization should also be borne in mind when conducting analytical work (for example, you need to understand that adding marker variables to a link to a page actually changes its address). However, the book describes not only optimization for search engines. Very systemic and well helps to build a chain of conclusions in your head.
- Of course, we should recommend Avinash Koshik’s book on web analytics , but he has already published a second book in which he pays more attention to work organization and details. We are waiting for translation into Russian.
- For those who can read English and communicate with Amazon (or with me ;) ), I recommend the book Landing Page Optimization Guide for Testing and Tuning for Conversions . It reveals many aspects of testing and provides many improvement tips that you can and should try to do on the site.
Blogs and resources
- Alexey Skobelev's blog . Interesting and detailed articles on aspects of analytics. Here, for example, is an article explaining how to interpret Alexa.com data.
- Google Conversion University . Training course on Google Analytics with online certification at the end (paid). Very useful, even if you are not going to pass on the certificate. It helped me a lot. Unfortunately, only in English.
- Blog Roman Zykov . Roman has a great deal of analytical experience (including at Ozon.ru) and writes, and sometimes talks at conferences, a lot of interesting things.
Source: https://habr.com/ru/post/79290/
All Articles