Analyze it: how to get additional benefit from client logs

As is known, the golden rule of dealing with a client is “not to bother”: neither with advertising, nor with news, nor with caring questions about what he likes and does not like in your programs. The same applies to technical support: the fewer calls, letters and remote sessions you need to collect all the necessary data, the better - for a company that will save a little money, and for a client who can save a lot of time, not to mention nerves of both sides. In the end, this is what makes you wonder: is it possible to somehow extract additional information for reflection from the data that we already have, without disturbing customers once again with newsletters and polls?

In this article I will talk about one of the ways we use at Parallels.

So, for this recipe, we need:

• User data storage (probably every large company has: a server where logs and data about client computers are added);
• a parser of the aforementioned user data (also probably available: who will give up unnecessary statistics on operating systems, configurations, models and versions);
• a couple of programmers: one from the side of the product and one more from the side of the internal development department (chief for storage and parsers);
• analyst (nice to meet you).

Point one: the registry of messages.
Participants: product programmer, analyst.
')
In any software product there must be at least three types of messages displayed to the user: errors, warnings and informational messages. Let's call them APP_ERR, APP_WARN and APP_INFO.

Warning: messages with questions like “Do you really want to format the C drive? Yes / No "do not participate in this recipe!

In the code and logs, these messages can be organized either by using digital codes (APP_ERR_1 and later), or by using names (APP_ERR_KABOOM), or by using combinations of both. In principle, the way of organizing for our venture is unimportant. We upload all the messages that the user sees into the file and feed the analytics, which in collaboration with the product programmer adds comments to each of them: when, to whom and why this message can be shown and what mechanisms are involved in this. For example: APP_ERR_KABOOM — shown by a component like this in case it is about to explode. Possible reasons: 1) the disk space has run out, 2) the rights to write to the disk have run out, 3) something is not good with the disk itself. This registry is created in order not to pull the product programmer once again during the subsequent work with messages. Further messages can be sorted by component or any other kind of signs to you.

Point two: search activities.
Participants: both programmers, analyst.

Now it would be good to understand how to extract message codes from logs with minimal losses. It directly depends on the size of the user base, the volume of the logs from each user and the available capacity. If users are a hundred per week, each of them sends logs in kilobytes, and the server is good enough, you can do without additional tasks and watch the log files themselves. If there are one hundred thousand users, and each sends by megabyte, then servers, we suppose, you cannot save enough, and the company probably already has some ways to optimize the process. In our case, the last error code is put in a separate line of the xml-file containing information about the product configuration, still on the user's side, which greatly simplifies the matter.

Point three: seizure and comprehension of the results of parsing.
Participants: programmer of internal development department, analyst.

Suppose we learned how to extract from logs everything that we need, and we can now give data about messages from the server. How exactly to do this, again, depends on the approach of each particular company: someone prefers colorful dashboards with visualization for every taste, someone costs the simple stored procedure on the server and the subsequent sorting in Excel. The main filter is the number of different users in a given sample of data: by version, for example, or by time, or by operating system. And then the fun begins - the conclusions.

Point four: conclusions.
Participants: analyst.

Errors . Why, it is asked, in such a strange and complicated way to analyze error messages, if the user himself runs with an error in technical support? And here I’ll get out of my pocket my favorite example about timeouts. Everyone has timeouts, and the waiting time is often set from complex considerations that are not quite consistent with the user's reality - say, by sight. Hence the most common type of problems: from time to time, some difficult process sadly reports a timeout. But not always, but depending on the computer or network load. User reproduction is by no means one hundred percent, the repetition of the operation is often successful, and technical support is far away, and how do you explain what happens if the message is automatically closed, for example. Therefore, if the timeout does not pop up every time or at least every second time, we may not know about it for quite some time. At the same time, without blocking the work completely, these messages create an extremely negative impression of the program. The same applies to many other situations with non-constant reproduction.

Warnings . The effect here is, in fact, the same: the program's constantly popping up cues are annoying. At the same time, nothing radically bad is happening at this moment, but repeated warnings make you wonder if you can change the interface or script so that the user doesn’t want to do something potentially unpleasant once again. For example, if an interruption of a process threatens with problems, do you not disable the ability to interrupt it If a delayed reboot can lead to a program crash and possible data loss, do not remove the Cancel button from the corresponding form. And so on.

Informational messages . With them, quite simply. First, as with warnings, you can think about changing the script or the interface so that a person does not run across the message once again. And secondly, you can see what information is most often required by the user, in what context, and expand the relevant sections in the documentation or knowledge base.

Thus, at the price of a small automation, the company receives plenty of additional statistics, without disturbing the user. These data as it accumulates can be analyzed in a variety of ways - compare versions, get immediate confirmation (or not) of their solutions, track problems typical of new operating systems, and so on. But they will definitely not be redundant in their work.