How to check website availability online? Service overview HostTracker. Part 2
The HostTracker company began with a simple website based on a no less simple script that provided free services to everyone. But for several years of work, the service, developing as a startup, has acquired a solid toolkit for solving various tasks of people involved in the Network in one way or another. In the series of publications we decided to describe in detail the functions of our service, to share the experience of developing such projects, to mention the interesting tasks that we had to face. Note that often the development of new functions occurs with an eye on our personal needs, so our history can still be a good example of how to “sell what has been done for ourselves.”
In the previous part of the review, the service history was described, as well as the most basic function - regular checking of sites and servers and alerts about their performance. Starting the story of more advanced features, you need to tell where ideas come from and how priorities are set. One of the secrets often mentioned by businessmen working in the service sector, and to which the next few lines will be devoted, is: “all people are unique”. All client requests are individual, and you need to learn how to live with it. In the field of IT, this is felt particularly acutely: due to the large number of outstanding personalities and the varied experience they possess. However, it is not rare to come up with a way that manages to solve the problem of the client with the available means. But often you have to finish something with a file - and this is the main feature of such an activity. A host of HostTrecker functions have been developed or adapted to the needs of specific people and for their convenience. We ourselves are among these people - possessing a fairly large infrastructure, we also need everything to work properly. Therefore, we ourselves are the most fastidious customer.
Describing regular checks, it is advisable to immediately point out some useful attached functions. These include monitoring domain expiration and SSL certificate. Such, like, little things, deliver great inconvenience when manifested in an unexpected time. These functions appeared in HostTreker when, as always, one of our personal domains suddenly "ended". But similar problems arise even for large companies, since the extension of domains / certificates always depends on the human factor, the reliability of which will not be extended here. Some examples we have collected and described here . Therefore, the notification that this domain needs to be extended, which will fall not only into the spam folder where your hoster has already been registered due to the continuous flow of marketing letters, but also into the folder about your site crashes (not to mention the possibility of SMS alerts), of course extremely helpful.
')
Another interesting feature is the verification of DNSBL blacklisted domains. These lists are independent and are formed each according to their own algorithm and are created mainly for filtering suspicious subjects. Each administrator can set up their own web server, for example, do not receive emails from servers listed in a specific list. It helps to fight spam, the spread of malware, DDoS attacks and other problems. But all these lists have their own algorithms, and there is no guarantee that your site will not be there as a result of random circumstances. Moreover, statistics show that this happens regularly even with the most innocent sites. For example, no one will give a guarantee that the local King of Spam will not be registered on your next IP address, as a result of which the entire range will be entered as unreliable. What can it splash out of? Letters from you will cease to come to customers, the site will become worse to appear in search engines ... And so on and on. And you will learn about it when the changes become critical, and sometimes, alas, irreversible. Therefore, the function of control and notification of hit in the most popular black lists is also very popular.
This is a good example of how using a function can exceed all the expectations of developers. Earlier we mentioned several similar cases. The fact is that a lot of things now have a web interface. And for an even larger set, you can write it without too much difficulty. Therefore, besides the main idea - checking whether the page was loaded entirely, by simple parsing, - new spaces opened up. Adjusting to different clients, this function has become as universal as possible: it can search for many words at once, or just one of the list. Or vice versa - to respond to the appearance of certain phrases. It may also give an error in the line that contains the keyword. For example, many make a status page: “Server 1 OK” and so on. If it suddenly becomes “Error”, then “Server 1 Error” will come in the message - all diagnostics have already been done, you can immediately start the elimination.
In addition to alerts about the fall of sites, HostTreker performs another important function - it collects statistics for each checked site. At a time when various buttons on sites were still popular - the HostTrecker button also often appeared next to attendance indicators, weather forecast and other, of course, very important things. Now it is possible to insert a widget, although the button is also still available for retro fans. In addition, statistics is often an indicator for management: does the site and its administration department work well? Therefore, another scheduling feature is another useful feature. Even the most stable sites and servers need to be updated or reloaded from time to time. And it does not want to lose the notorious "nines" in terms of uptime. Planning technical work allows two goals (together or separately): do not send error notifications and do not record errors during a certain time interval in the statistics. Of course, it still smacks of cheating, but the planned downtime, especially during non-working time for the target audience, can be quite forgiven, from our point of view. However, checks during technical work are still going on, and they are regularly written to the log, therefore they can be useful for webmasters and administrators: the log allows you to determine how long the update or reboot took, what errors were issued (that is, the entire process: the server is unavailable for some time, then returns an application error, then another error, etc.), which “brakes” were observed and so on.
Technical work can be planned as a one-time or on a regular basis. For example, if a backup is made every night, or every Thursday a new release. The only condition is that the schedule of technical works should be made no later than 12 hours before the next site / server suspension. This is done to ensure that unexpected surprises do not accrue to technical work, and the statistics provided by HostTreker remain reliable.
We are often asked a question (including in the comments on Habré) - and why are you doing this? The same can be done independently. We answer: yes, you can. Especially if you need one thing. But the fact is that, roughly speaking, the whole business is built on human laziness. By making life easier for people, we free up the time of our customers for something more important, because the tasks we solve are full of routine. Well, do not forget that not everyone is able to personally collect a car for himself or grow bread. In general, we sincerely respect people who are able to do themselves well in our sphere, but practice shows that not everyone is ready to spend their time on this.
In addition, our service is used by: non-IT people; IT specialists who have to report something to the authorities; bosses themselves; lovers of reliability - and I will do my own, and use someone else's; and many other categories of people.
But we are all the same very happy to read all your comments, because on Habré it is not uncommon to catch something useful for yourself even between the lines.
Is it all about the feature set?
In the previous part of the review, the service history was described, as well as the most basic function - regular checking of sites and servers and alerts about their performance. Starting the story of more advanced features, you need to tell where ideas come from and how priorities are set. One of the secrets often mentioned by businessmen working in the service sector, and to which the next few lines will be devoted, is: “all people are unique”. All client requests are individual, and you need to learn how to live with it. In the field of IT, this is felt particularly acutely: due to the large number of outstanding personalities and the varied experience they possess. However, it is not rare to come up with a way that manages to solve the problem of the client with the available means. But often you have to finish something with a file - and this is the main feature of such an activity. A host of HostTrecker functions have been developed or adapted to the needs of specific people and for their convenience. We ourselves are among these people - possessing a fairly large infrastructure, we also need everything to work properly. Therefore, we ourselves are the most fastidious customer.
Trifle, but nice
Describing regular checks, it is advisable to immediately point out some useful attached functions. These include monitoring domain expiration and SSL certificate. Such, like, little things, deliver great inconvenience when manifested in an unexpected time. These functions appeared in HostTreker when, as always, one of our personal domains suddenly "ended". But similar problems arise even for large companies, since the extension of domains / certificates always depends on the human factor, the reliability of which will not be extended here. Some examples we have collected and described here . Therefore, the notification that this domain needs to be extended, which will fall not only into the spam folder where your hoster has already been registered due to the continuous flow of marketing letters, but also into the folder about your site crashes (not to mention the possibility of SMS alerts), of course extremely helpful.
')
Another interesting feature is the verification of DNSBL blacklisted domains. These lists are independent and are formed each according to their own algorithm and are created mainly for filtering suspicious subjects. Each administrator can set up their own web server, for example, do not receive emails from servers listed in a specific list. It helps to fight spam, the spread of malware, DDoS attacks and other problems. But all these lists have their own algorithms, and there is no guarantee that your site will not be there as a result of random circumstances. Moreover, statistics show that this happens regularly even with the most innocent sites. For example, no one will give a guarantee that the local King of Spam will not be registered on your next IP address, as a result of which the entire range will be entered as unreliable. What can it splash out of? Letters from you will cease to come to customers, the site will become worse to appear in search engines ... And so on and on. And you will learn about it when the changes become critical, and sometimes, alas, irreversible. Therefore, the function of control and notification of hit in the most popular black lists is also very popular.
Content Verification
This is a good example of how using a function can exceed all the expectations of developers. Earlier we mentioned several similar cases. The fact is that a lot of things now have a web interface. And for an even larger set, you can write it without too much difficulty. Therefore, besides the main idea - checking whether the page was loaded entirely, by simple parsing, - new spaces opened up. Adjusting to different clients, this function has become as universal as possible: it can search for many words at once, or just one of the list. Or vice versa - to respond to the appearance of certain phrases. It may also give an error in the line that contains the keyword. For example, many make a status page: “Server 1 OK” and so on. If it suddenly becomes “Error”, then “Server 1 Error” will come in the message - all diagnostics have already been done, you can immediately start the elimination.
What if the server MUST lie down?
In addition to alerts about the fall of sites, HostTreker performs another important function - it collects statistics for each checked site. At a time when various buttons on sites were still popular - the HostTrecker button also often appeared next to attendance indicators, weather forecast and other, of course, very important things. Now it is possible to insert a widget, although the button is also still available for retro fans. In addition, statistics is often an indicator for management: does the site and its administration department work well? Therefore, another scheduling feature is another useful feature. Even the most stable sites and servers need to be updated or reloaded from time to time. And it does not want to lose the notorious "nines" in terms of uptime. Planning technical work allows two goals (together or separately): do not send error notifications and do not record errors during a certain time interval in the statistics. Of course, it still smacks of cheating, but the planned downtime, especially during non-working time for the target audience, can be quite forgiven, from our point of view. However, checks during technical work are still going on, and they are regularly written to the log, therefore they can be useful for webmasters and administrators: the log allows you to determine how long the update or reboot took, what errors were issued (that is, the entire process: the server is unavailable for some time, then returns an application error, then another error, etc.), which “brakes” were observed and so on.
Technical work can be planned as a one-time or on a regular basis. For example, if a backup is made every night, or every Thursday a new release. The only condition is that the schedule of technical works should be made no later than 12 hours before the next site / server suspension. This is done to ensure that unexpected surprises do not accrue to technical work, and the statistics provided by HostTreker remain reliable.
Instead of an epilogue
We are often asked a question (including in the comments on Habré) - and why are you doing this? The same can be done independently. We answer: yes, you can. Especially if you need one thing. But the fact is that, roughly speaking, the whole business is built on human laziness. By making life easier for people, we free up the time of our customers for something more important, because the tasks we solve are full of routine. Well, do not forget that not everyone is able to personally collect a car for himself or grow bread. In general, we sincerely respect people who are able to do themselves well in our sphere, but practice shows that not everyone is ready to spend their time on this.
In addition, our service is used by: non-IT people; IT specialists who have to report something to the authorities; bosses themselves; lovers of reliability - and I will do my own, and use someone else's; and many other categories of people.
But we are all the same very happy to read all your comments, because on Habré it is not uncommon to catch something useful for yourself even between the lines.
Source: https://habr.com/ru/post/307784/
All Articles