Web analytics: analyze it! Part 2. Data Collection
Before analyzing these statistics, you need to understand how they were collected, which of them may be inaccurate, and why.
The server on the Internet receives requests from the user's browser and sends the data. For each page view, the server receives one request (for the body of the page), and then several additional (pictures, scripts, style sheets and other additional data to display the page). Scripts on the page can also generate requests to the server - including, to a separate server statistics.
The web server connects requests for the same user using sessions. When a new user accesses the server, he creates a new session identifier, which the user informs the server with each new page load. Typically, the session ID is transmitted to the server from a cookie - a variable file that the browser can save for a particular site.
What can the server know about the user?
')
The second type of statistics collection system adds to the page shown by the user, additional code that queries the statistics system. When the Internet was young and the browsers were very stupid, they used pictures: to display a picture on the page, browsers made a request to the statistics server. This server has already made an entry in its journal, and then displayed a picture with numbers - it was precisely since then that the Runet tradition of “hanging counters” of 88x31 size went. This method is worse for everyone than the first, and was used only because site owners did not have access to their server logs.
When browsers everywhere learned to execute JavaScript scripts (mini-programs), the external counters became much smarter. In addition to the standard query, JavaScript was able to transmit a lot of new data to the statistics system: the resolution and color of the screen, the parameters of the operating system. They again got access to the referrer and were able to set cookies on behalf of the visited site, which made it easier to track user sessions. The most popular “fancy” counter of this kind is Google Analytics!
The external counter on JS also has flaws:
- with its help it is impossible to track file downloads from the site,
- it records only transitions to the pages loaded in full (since otherwise the code will not have time to be executed),
- it requires a modern browser and permission to operate the scripts,
- it does not work on mobile browsers (except Opera Mini and modern smartphones),
- in order to record the parameters of the internal “kitchen”, such as user account data, all this data needs to be distilled into the counter code, which is usually unsafe, difficult and, as a result, pointless. Imagine that this is a dating site, each of whose users has a profile with a bunch of parameters. To analyze the behavior of users with different parameters of the profiles, you need to link the questionnaires with the requests.
In such difficult cases, site developers themselves develop a system for recording statistics to which they add all the features they need. The advantages of such a system are its infinite flexibility. This also leads to a major drawback: the need to write data analysis tools manually for such a system. So developers, whose needs are met by ready-made systems, try to use them.
The first and most important thing to remember when working with statistics from the Internet: accurate and complete data is often very difficult to obtain. I will clarify inaccuracies as metrics are listed. A key analyst skill is the ability to distinguish important from unimportant constraints.
Standard, popular systems most often use the second method. Most of all, we’ll talk about Google Analytics, and in the next section we’ll look at the main metrics available to its users.
The server on the Internet receives requests from the user's browser and sends the data. For each page view, the server receives one request (for the body of the page), and then several additional (pictures, scripts, style sheets and other additional data to display the page). Scripts on the page can also generate requests to the server - including, to a separate server statistics.
The web server connects requests for the same user using sessions. When a new user accesses the server, he creates a new session identifier, which the user informs the server with each new page load. Typically, the session ID is transmitted to the server from a cookie - a variable file that the browser can save for a particular site.
What can the server know about the user?
- First of all, its ip-address. It is usually used for only one purpose: to find out the region from which the user goes to the Internet. Most providers are configured in such a way that each user addresses change all the time. In addition, many people leave the network with a single external address. Thus, the same address does not mean that the same user works with them, and vice versa.
- The address of the page that the user visits.
- Address of the page from which the user switched to the current, or "referrer". This data is very helpful. If the user, for example, came from a search engine, then the previous address saved the request, which he entered into the search engine.
- The exact time of the request.
- The “callsign” of the user's browser is his identification string. It allows you to understand which browsers are popular among users, and also to distinguish a query from a search robot.
- Cookies are data that the server previously “asked” the browser to remember. Cookies can be persistent (saved for a certain period of time) and session cookies (destroyed after the browser is closed). They contain arbitrary sets of variables and values.
- Variables that the browser passes.
')
The second type of statistics collection system adds to the page shown by the user, additional code that queries the statistics system. When the Internet was young and the browsers were very stupid, they used pictures: to display a picture on the page, browsers made a request to the statistics server. This server has already made an entry in its journal, and then displayed a picture with numbers - it was precisely since then that the Runet tradition of “hanging counters” of 88x31 size went. This method is worse for everyone than the first, and was used only because site owners did not have access to their server logs.
When browsers everywhere learned to execute JavaScript scripts (mini-programs), the external counters became much smarter. In addition to the standard query, JavaScript was able to transmit a lot of new data to the statistics system: the resolution and color of the screen, the parameters of the operating system. They again got access to the referrer and were able to set cookies on behalf of the visited site, which made it easier to track user sessions. The most popular “fancy” counter of this kind is Google Analytics!
The external counter on JS also has flaws:
- with its help it is impossible to track file downloads from the site,
- it records only transitions to the pages loaded in full (since otherwise the code will not have time to be executed),
- it requires a modern browser and permission to operate the scripts,
- it does not work on mobile browsers (except Opera Mini and modern smartphones),
- in order to record the parameters of the internal “kitchen”, such as user account data, all this data needs to be distilled into the counter code, which is usually unsafe, difficult and, as a result, pointless. Imagine that this is a dating site, each of whose users has a profile with a bunch of parameters. To analyze the behavior of users with different parameters of the profiles, you need to link the questionnaires with the requests.
In such difficult cases, site developers themselves develop a system for recording statistics to which they add all the features they need. The advantages of such a system are its infinite flexibility. This also leads to a major drawback: the need to write data analysis tools manually for such a system. So developers, whose needs are met by ready-made systems, try to use them.
| Server logs | External statistics | Record statistics | |
| Session tracking | - (difficult enough realizable) | + | + |
| Record all visits | + | - (only browsers with JS enabled and full pages) | + |
| Tracking uploaded files | + | - | + |
| Tracking search and other bots | + | - | + |
| Communication visits with internal data site | - | - | + |
| Comparison with data from other sites | - | + | - |
| Ability to track transactions and sales funnels | - | + | + |
| Tracking events that do not lead to requests to the server | - | + | + |
The first and most important thing to remember when working with statistics from the Internet: accurate and complete data is often very difficult to obtain. I will clarify inaccuracies as metrics are listed. A key analyst skill is the ability to distinguish important from unimportant constraints.
For example, external statistics systems based on JavaScript will not work for users with very old browsers or scripts disabled for security reasons. In most cases, this is acceptable: the share of such users is small (less than a percent). However, if you collect data on the corporate intranet in a company that disables JS from its employees, or you want to measure the percentage of users with disabled scripts, this method is no longer suitable.General limitations of statistics collection systems:
- It is impossible to track who is sitting at the computer. You can only distinguish between browsers.
- It is impossible to accurately determine the region - just try to guess based on the user's IP (although it can go online via remote proxy, VPN, etc.). For example, all users of the mobile Internet through Opera Mini were recorded by statistical systems in the “Norway” region, because requests went through an adapting server in this country.
- It is impossible to find out the user if he came from another computer and did not enter the password on the site.
- It is impossible to determine where the user came from, if the firewall filters the referrer field, or he clicked on the link in the mail, ICQ, reprinted it from the advertisement, etc.
- It is impossible to determine whether the user made a transaction if he decided at a certain stage to continue it offline (call the store, for example). This problem more significantly affects the proper calculation of the quality of online stores and requires additional measures offline :(
Standard, popular systems most often use the second method. Most of all, we’ll talk about Google Analytics, and in the next section we’ll look at the main metrics available to its users.
Source: https://habr.com/ru/post/66404/
All Articles