Launch of the Monitoring service

The performance of a modern web service is affected by a variety of factors. With the constant growth in the amount of data and the increase in the number of users, the infrastructure of the service becomes more complex, and the problem of tracking its condition becomes acute. The available monitoring tools (for example, Nagios or Icinga) are in many cases insufficient.

Along with the growth of infrastructure, it is required to ensure the scalability of the monitoring system: the number of inspections is constantly increasing, reports are becoming more voluminous ... Tracking the stability of the web service in such conditions takes more time and money.
')
Understanding perfectly the complexity of the situation described, we have developed a service monitoring service for our clients. Its detailed description is presented below.

What is monitoring?

Monitoring is a service that allows you to check various services for availability from several points on the Internet and send notifications in case of fulfillment or non-fulfillment of a given condition. We have tried to make the monitoring service as flexible as possible, so that those who do not have the skills to administer web services can use it.

Available check types

The following types of checks are currently available:

• PING - checking the availability of communication with the server, helps to measure the delay and availability from the verification point to the server;
• TCP - check the status of the TCP port;
• UDP - check the status of the UDP port;
• SSH - check connection without authorization, as well as check the banner SSH;
• SMTP, POP3, IMAP - checking the operability of the mail service using the selected protocol;
• DNS - checking the correctness of the domain name conversion to the IP address on the specified resolver;
• HTTP GET - checking the content and response code, as well as searching for a substring in the returned data;
• HTTP HEAD - a lightweight GET request that allows you to check the presence of a document on the server by the response code without having to download the entire body of the document;
• HTTP basic authorization - verification of the basic authorization mechanism;
• HTTP digest - check the mechanism of digest authentication;
• FTP — check the availability of the FTP server;
• PostgreSQL - check availability of the PostgreSQL database server;
• MySQL - check availability of MySQL database server;
• SSL - check the validity of SSL certificates for domain names.

Points of presence

At the moment, check points are located in the following cities: Moscow, St. Petersburg, Yekaterinburg, Kiev, Amsterdam, Frankfurt, London, New York, Palo Alto. If a firewall is installed on your server, then when creating a check you can get the IP addresses of the points of presence to include them in the firewall rules.

Creating a check

Currently, the monitoring service is in test mode. Each user can create only three checks. Log into the control panel under your account. In the main menu, select the item "Monitoring". The following page will open:



Click on the "Create" button. In the drop-down menu, select the type of scan:



Then enter the scan parameters:



All parameters of checks are divided into mandatory (common for all types of checks) and individual (characteristic of certain types of checks; they are displayed in the "Other parameters" section.

Required parameters include:

• verification period - the frequency with which the verification will be carried out;
• check timeout - a period of time after which the check will be considered unsuccessful;
• sequence of checks - the order in which the check will be carried out from selected points of presence (sequentially, in parallel or in random order);
• host / IP - domain name or IP address of the server being checked;
• protocol - the priority of the protocol that will be used for verification (IPv4 or IPv6).

For each type of check, individual parameters can also be set:

• For SSH checks:
  
  • port for connection;
  • banner;
  
  
• For TCP checks:
  
  • send string;
  • response line;
  
  
• For UDP checks:
  
  • port for connection;
  • send string;
  • response line;
  
  
• For SMTP checks:
  
  • user;
  • password;
  • safe mode (on / off);
  • banner;
  • domain name;
  
  
• For POP3 and IMAP checks:
  
  • user;
  • password;
  • safe mode (on / off);
  • start_tls;
  • banner;
  • domain name;
  
  
• For HTTP GET checks:
  
  • port for connection;
  • user agent;
  • referrer;
  
  
• For HTTP HEAD checks:
  
  • port for connection;
  • substring;
  • user agent;
  • referrer;
  • For HTTP checks Basic authorization and HTTP Digest authorization;
  • port for connection;
  • user;
  • password;
  
  
• For FTP checks:
  
  • port for connection;
  • mode (active or passive);
  • anonymous (enable / disable);
  • user;
  • password;
  
  
• For PostreSQL and MySQL checks:
  
  • port for connection;
  • user;
  • password;
  
  
• For SSL checks:
  
  • port for connection;
  • period to check.
  
  

Having filled in the main and individual fields, indicate the points of presence from which the verification will be carried out.
After that, click on the "Create" button. Next, you need to specify the conditions under which you will be sent the satisfaction of the test results. When the creation of the scan is completed, the following window will appear:



Then click on the “Create conditions for created check” button. The form for adding a condition will open:



The following conditions can be specified:

• successful or unsuccessful completion of the verification procedure;
• Connection error;
• response time (it can be equal to or not equal to, greater than or equal to, less than or equal to that specified by the user).

Individual conditions may also be established for each type of inspection. Notifications of test results sent under specified conditions contain additional information that can be used to identify problem areas during inspections.

View verification information

The list of added checks is displayed on the tab "Checks":



To the right of each check are the control buttons:
- add a check to your favorites;
- testing verification;
- viewing and editing the scan parameters;
- delete check.

If you click on any check in the list, a card will open with detailed information:



It includes the following tabs:

• “Information” (name, host, type, creation date, SLA, information about the latest checks);
• “Graphs” (availability graphs and delay time; on the basis of a graph comparison, it is possible to determine the cause of the delays: in the network malfunction or in the service operation);
• "History" (a list of recent checks with an indication of time, access point, response time and result);
• "Statistics" (information about the sent notifications);
• “Setup” (form for viewing and editing the scan parameters);
• "Conditions" (information about the conditions established for the selected test).

Checks added to favorites are also shown in the list on the Information tab:



The Information tab also displays information on the five most recently executed conditions, the last five completed checks, and the last five actions you performed.

Detailed information on the results of all checks can be found on the “History” tab.
The full history of all your actions is shown on the "Actions" tab.

Newsletter Notifications

Notifications are sent:

• by e-mail;
• xmpp instant messaging;
• private messages in twitter (DM) - in this case, the client will need to subscribe to our account, from which the distribution is carried out.

After launching the service into commercial operation, it is planned to send SMS notifications.

Future plans

As mentioned above, the monitoring service is now operating in test mode. In the near future we are planning to expand the existing functionality. So, the “public statistics” function is currently being tested - a page accessible to all with data on the availability of selected checks. Our clients will be able to attach it to a separate domain in order to provide accessibility statistics to their users.

In the near future, it will be possible to periodically send out summary statistics by e-mail. There will be new types of checks.

We invite all of our readers to test the monitoring service. We would be very grateful for any comments, suggestions and suggestions.

For those who can not comment on posts on Habré, we invite to our blog .