How we use Zabbix monitoring system for retail
You will not be surprised by monitoring systems like Zabbix. However, in retail they are not frequent guests, and if they are used, they are more and more to control servers. We went further and applied them to monitor cash software and equipment.
Everything is completely transparent here: retailers and service companies rarely use monitoring systems, because it is difficult to assess their economic efficiency. With the introduction of business processes, everything is simple - X money and X effort. But to calculate how much they saved the retailer in the future heavier.
')
Service contractors usually do not implement monitoring systems yet, so as not to reduce the significance of their work. This is logical: Zabbix allows you to identify the problem before the client saw it. On the one hand, it improves the quality of services provided. On the other hand, the client sometimes has the erroneous impression that his business processes are well-established and work without any third-party assistance. But this can be solved by providing timely reporting.
However, even those retailers who agree to implement a monitoring system usually all end with monitoring servers, office computers, bespereboynik, active network equipment. We also do this:
As part of the information received, automatic requests are made to the Service Desk. A number of other data helps us in investigating incidents. A classic example: a user complains that his computer is slow. Without a monitoring system, it’s hard to track down - either when the engineer connects, everything will be fine, or the employee has a subjective impression (his working PC is objectively slower than a clever gaming computer that is at home). Therefore, we study the retrospective - the graphics for the time when the person watched the problem.
But all of the above is commonplace, nothing new. It so happened that we went further and with the help of Zabbix we began to monitor the performance of the cash register software and the cash register equipment. We do this for large international retailers that are widely represented on the Russian market in both the food and non-food segments. Also, our regional monitoring system was acquired by some regional networkers, who can now independently control the performance of their business processes.
Speaking frankly, the monitoring system was implemented in Pilot spontaneously, without any project and in parts. If the decision about this came from above, perhaps we would have followed the path of other service contractors and would not bother. But we have initiated the introduction of steel line employees - engineers. Faced with one or another breakdown of cash equipment or software glitch, they were looking for how to prevent it in the future. And they came to the idea of a monitoring system.
With its help, we get three solutions to problems:
Of course, our monitoring system affects far from all cash equipment because it does not always make sense. Take a barcode scanner. They are either working or not. And in the second case, the store employees will inform us about the problem much faster than the monitoring system. Therefore, we focused on the control of POS-terminals and cash registers (CCP) .
KKT gives through the driver enough information that allows you to judge its performance. For example:
Case from life: many probably remember the case of a bug in the firmware of one of the manufacturers of CCP in December 2017? As soon as the first information about the problem appeared, we set up a trigger in Zabbix, which signals the firmware version of the bug, and received a list of CCV that needs urgent work.
As part of monitoring the cash program, we monitor various signs:
Most often, Zabbix signals us about network problems: device unavailability, too long response time. Further, there are difficulties with the utilization of resources: low-power PCs are usually used for cash registers. The third most frequent issue is the validity of data from external systems.
Quite often there are reports of incorrect local time. Cash PCs are usually not introduced into AD and the ntp service there has to be configured separately, which is sometimes forgotten. And the wrong time at the checkout is fraught with major problems for the store: for example, selling alcohol when it is prohibited, which can lead to a fine or loss of a license.
Another field of activity, where Zabbix, rather, by the way, unexpectedly, turned out to be useful - the fight against fraud. It happens that contractors in the regions or individual field engineers who are paid by the job, enter into a criminal conspiracy with the users of the customer and solve problems that actually did not exist. We can bring them to clean water by analyzing the readings of the monitoring system. So far this is done manually, when a suspicious surge of activity is registered in a particular location, but we are working to automatically verify applications against Zabbix readings in all cases where this is possible.
Now from our monitoring system receives from 15 to 25% of applications. This is quite a small amount, but by the end of this year we want to bring it up to 50% for customers who have signed service agreements with us.
Why monitoring systems are rare in retail
Everything is completely transparent here: retailers and service companies rarely use monitoring systems, because it is difficult to assess their economic efficiency. With the introduction of business processes, everything is simple - X money and X effort. But to calculate how much they saved the retailer in the future heavier.
')
Service contractors usually do not implement monitoring systems yet, so as not to reduce the significance of their work. This is logical: Zabbix allows you to identify the problem before the client saw it. On the one hand, it improves the quality of services provided. On the other hand, the client sometimes has the erroneous impression that his business processes are well-established and work without any third-party assistance. But this can be solved by providing timely reporting.
However, even those retailers who agree to implement a monitoring system usually all end with monitoring servers, office computers, bespereboynik, active network equipment. We also do this:
- from the servers we obtain data on the utilization of processors, the performance of the fans, hard drives, memory, temperatures of processors and motherboards;
- from bespereboynik - status, charge level, information about how much time they will work in the event of a power outage;
- from network equipment - traffic on ports, utilization of resources.
As part of the information received, automatic requests are made to the Service Desk. A number of other data helps us in investigating incidents. A classic example: a user complains that his computer is slow. Without a monitoring system, it’s hard to track down - either when the engineer connects, everything will be fine, or the employee has a subjective impression (his working PC is objectively slower than a clever gaming computer that is at home). Therefore, we study the retrospective - the graphics for the time when the person watched the problem.
But all of the above is commonplace, nothing new. It so happened that we went further and with the help of Zabbix we began to monitor the performance of the cash register software and the cash register equipment. We do this for large international retailers that are widely represented on the Russian market in both the food and non-food segments. Also, our regional monitoring system was acquired by some regional networkers, who can now independently control the performance of their business processes.
Why we began to do this
Speaking frankly, the monitoring system was implemented in Pilot spontaneously, without any project and in parts. If the decision about this came from above, perhaps we would have followed the path of other service contractors and would not bother. But we have initiated the introduction of steel line employees - engineers. Faced with one or another breakdown of cash equipment or software glitch, they were looking for how to prevent it in the future. And they came to the idea of a monitoring system.
With its help, we get three solutions to problems:
- preventively - fix the problem before it happened. For example, when monitoring a hard disk, we see that the space on it has decreased to a critical level. And we take measures in this connection;
- after the fact - we solve the problem after it happened. For example, the fan on the processor failed. The processor is still warm, but it works. Sooner or later, of course, it will fail, but as long as we have the opportunity to replace the fan. That is, the user has not yet noticed the incident, but he already is. From its point of view, we solve the problem proactively, but from the point of view of equipment, after the fact;
- analytically - we get a large amount of data in retrospect for the analysis of incidents.
Of course, our monitoring system affects far from all cash equipment because it does not always make sense. Take a barcode scanner. They are either working or not. And in the second case, the store employees will inform us about the problem much faster than the monitoring system. Therefore, we focused on the control of POS-terminals and cash registers (CCP) .
CCP health monitoring
KKT gives through the driver enough information that allows you to judge its performance. For example:
- Various inventory data - versions of hardware, firmware, drivers, serial numbers. In general, the composition of the equipment at the service is fixed in the annexes to the contracts and stored in the CMDB, however, the customer is free to move and replace the equipment as he pleases. Of course, he does not always remember that it would be nice to notify the service company. This is where the monitoring system comes to the rescue, which tracks the changes in the equipment configuration. We wrote an integration module that adjusts the CMDB according to the inventory data from Zabbix. In addition to tracking the actual configuration of equipment at service facilities, it, coupled with the auto-detection functionality of the monitoring system, thoroughly reduces the time for starting inventory of a new client, if such work is provided for in the contract.
Case from life: many probably remember the case of a bug in the firmware of one of the manufacturers of CCP in December 2017? As soon as the first information about the problem appeared, we set up a trigger in Zabbix, which signals the firmware version of the bug, and received a list of CCV that needs urgent work.
- The CCP status code is an excellent parameter that allows you to track almost any malfunction, ranging from incorrectly set time or overheating of the printer head to the presence of unsent fiscal data on the fiscal drive.
Control of cash register software
As part of monitoring the cash program, we monitor various signs:
- service operability - software is enabled or not, does it open any network ports or is waiting for a connection;
- records in the logs - usually the software writes to the logs about the problems that have arisen, forms a set of errors. As an indirect sign, if the logs change, then the software works, if there are no new entries in them, then you need to create an application;
- in fact, the entries themselves in the logs - if an error message is encountered, the trigger is triggered. After processing, the records are transferred to ELK: Logstash from us removes the logs through the Zabbix API;
- the results of the work of the integration software that downloads, converts and sends data (for example, transmits information to the USAIS, CRF, receives a product line). So, recently an incorrectly formed data package with a nomenclature disabled the software of self-payment terminals, paralyzing their work in one of our customer’s stores. Thanks to the monitoring system, we were able to locate the problem on time;
- software versions, drivers - sometimes there are situations when, for example, the versions of two programs are not compatible, but for the work of cash software it is necessary that they interact;
- databases - we monitor the serviceability of services, the availability of network ports, the number of databases, their versions and the number of disabled databases;
- external services (for example, USAIS, with which we interact through IP networks in automatic mode).
Problems that most often go to the monitoring system
Most often, Zabbix signals us about network problems: device unavailability, too long response time. Further, there are difficulties with the utilization of resources: low-power PCs are usually used for cash registers. The third most frequent issue is the validity of data from external systems.
Quite often there are reports of incorrect local time. Cash PCs are usually not introduced into AD and the ntp service there has to be configured separately, which is sometimes forgotten. And the wrong time at the checkout is fraught with major problems for the store: for example, selling alcohol when it is prohibited, which can lead to a fine or loss of a license.
Fighting fraud and equipment downtime
Another field of activity, where Zabbix, rather, by the way, unexpectedly, turned out to be useful - the fight against fraud. It happens that contractors in the regions or individual field engineers who are paid by the job, enter into a criminal conspiracy with the users of the customer and solve problems that actually did not exist. We can bring them to clean water by analyzing the readings of the monitoring system. So far this is done manually, when a suspicious surge of activity is registered in a particular location, but we are working to automatically verify applications against Zabbix readings in all cases where this is possible.
Now from our monitoring system receives from 15 to 25% of applications. This is quite a small amount, but by the end of this year we want to bring it up to 50% for customers who have signed service agreements with us.
Source: https://habr.com/ru/post/442044/
All Articles