Methods for monitoring websites and services

Business on the Internet is now very much tied not only to the quality of the site itself, but also to its availability and performance 24/7/365. And although hosting providers claim sufficiently large uptime of sites and servers, this is not always enough to be sure of the reliability of the site. One of the assistants in this is monitoring both the accessibility and the operability of the main subsystems of the site. Let's see how this can be done now.

Regular check

One of the most common methods is to regularly check the performance of the site. You can do it yourself - in the morning, open the website of your favorite company or an online store (having it in the start page of the browser), be happy that it works. And close. Not very convenient and not at all effective. Accessibility problems usually begin unpredictably. It is better to detect them as soon as they arise. Also, the users themselves often help them — they start calling and writing that the site is not working (if the phone and mail are indicated somewhere outside the site). The most frequent signal of a non-working site - customers stop coming. There are already sounding the alarm and starting to find out what happened there. tearing hair, tearing bells, sprinkled with ash and salt. Although all this could have been avoided.

Accessibility monitoring

More educated and advanced site owners or managers who are responsible for it are aware of the need to constantly monitor the accessibility of the site — whether it is working or not. The most common mistake here is the wrong choice of monitoring frequency. Checking once a day of the site’s working capacity is, of course, useful, but loss of daily revenue is too high a cost of a mistake. The 10-minute interval is the most optimal for simple monitoring: most users will try to return to the site within 1-2 hours, and during this time, you can both detect problems and effectively eliminate them without much harm to the company’s business.

On the other hand, more frequent checks do not guarantee that the problem will be fixed more quickly than within an hour. After all, most of the time is spent not so much on finding the problem as on finding out its causes and correcting them.
')

Problem monitoring

When the level of accessibility of a site is already becoming critical for business - especially the presence of "floating" problems associated with the coincidence of a number of circumstances - then the usual monitoring becomes low. Here it is necessary to track several parameters of the site with a frequency of at least once a minute and from several geographic points (in order to maximally cover the minute interval with checks and establish possible problems related to the geography of users). Among the possible verification criteria are:

• Problems with the DNS server (when the site address cannot be determined at certain intervals, although the site itself is physically accessible).
• Problems with a long response time (when updating the cache, for example, or when performing "heavy" tasks on the server side).
• Problems with the planned execution of tasks (as a result of which the site will not be available only at certain points in time).
• Problems with long latency of static files (for example, due to network infrastructure or problems with physical media).
• Problems connecting to the database.
• etc.

In this case, it is necessary to monitor the site every minute from several points (or with the help of several independent services or check points). Monitoring may be short-lived (until problems are detected and corrected) or periodic (in order to prevent problems). It is also possible to regularly self-check the site for possible accessibility problems, but external verification is always better than internal for many reasons. Additionally, external services now provide detailed information about problems, up to error logs on the client side (with appropriate configuration, and error logs from the server side).

This method is especially good when you need to catch some kind of "floating" error. When you enable detailed logs of an error that occurs, even if there are no details of the error on the server side, it can be effectively tracked and eliminated. Several test points allow you to achieve a test frequency up to 10 seconds - and this is more than enough to find all that is needed.

Health monitoring

This topic is especially relevant for online stores, for which the ability to order or send a request is key. Also, this may include any complex functionality that can be affected by changes on the website (for example, an Internet bank account). In this case, it is necessary to set up chains of checks or set up complex conditions for conducting checks. Without the help of a qualified specialist, this is hardly possible, but now there are products and services that allow you to automate the whole or almost the entire process of such functional monitoring of the site.

It is very simple to estimate how important monitoring for a site is: it is enough to calculate the daily income generated by the site and correlate it with the cost of this monitoring. Understand how much each hour of site downtime during working hours costs, and decide on the effectiveness of monitoring and its details (this will be a simple availability check, health monitoring or comprehensive monitoring of all possible problems).

PS The topic does not affect the topics of the internal cluster monitoring device or the web server system - Nagios is better suited for this purpose, as well as issues of the speed and fault tolerance of the sites themselves - Load Load or your own systems, for example, have nothing to do with JMeter.

PPS Monitoring of the health of cloud services is also a topic for a separate topic.