Logs as part of the product. How GrayLog Impacted Quality

Experience of using GrayLog in our projects and how it affected the quality of products.

This article is not about GrayLog's capabilities, but about how it helped our company.

Previously, we wrote logs only to files, access to these files had a limited number of developers, read logs only if something happened and the search in the logs looked like a punishment.

When we started a year ago with a new CRM integration project, we decided to try GrayLog.

Logs as part of the product

When we switched to GrayLog and began to frequently look in the logs, we noticed that some messages lack information to understand the situation, and some, on the contrary, are redundant. We began to clean up unnecessary messages, and change the message texts to make them understandable.

Employees began to use GrayLog constantly and report where the logs are incomprehensible. We began to improve the messages in the logs and correct errors in the messages, as we do with the program code. Logs have become part of the product, and we began to monitor their quality.

A big change in the logs happened when we decided to make them in Russian. This allowed employees from technical support to better understand what they were talking about, and developers to write in their native language, instead of pseudo-English, which was used before.

Messages in Russian

Problem analysis

In GrayLog you can visually see the problem. Below is a surge in requests from Bitrix24, which created a heavy load on the server. It can be seen when it started and when it ended.

Splash of log messages

With any search GrayLog shows such a graph of the distribution of messages in time. If you need to save a schedule to follow it, you can add it to the summary.

We are adding charts to the summary for a while to follow the reappearance of the problem.

Sample summary with error graphs

Problem Notifications

GrayLog can notify about problems. You can send a notification in the following cases:

• if a specific value appears in the field;
• if the numeric field value is exceeded or too low;
• keep track of the number of messages.

Setting notification conditions

Notifications are sent to the mail or callback url is called. The second method allows you to send an SMS message to the messenger or automatically scale the application.

Using logs by company employees

In GrayLog you can create streams - named filters. For threads, permissions are set. This makes available only some logs for employees. For example, create a stream with a filter by server name, so that only logs from the test server get into it, and make the tester access to the stream.

Thread list

In practice, we make little use of the division of rights. Technical support staff have access to production logs. This reduces the burden on developers, because colleagues from the support team understand some of the problems by logging and understand themselves.

Conclusion

GrayLog not only facilitated the search in the logs, but changed the process of working with logs. Logs have become part of the product. Now we monitor the quality of the logs on a par with the quality of the product. Logs became available to all developers and colleagues from technical support. It became possible to detect problems before they were reported by users.