Habraeffekt eyes of the server or why it is not necessary to distribute an apache a statics
In article I will tell impressions from habraeffekta observed on the personal server.
It will not be anything new for those who have long been engaged in the web, and especially for administrators who have decided to protect against DDOS.
Just a small note about how I observed habraeffekt.
')
I have already had a dedicated server for various web projects for quite some time now, as well as a file storage facility accessible from everywhere. At the beginning of the 2000s it was just an ordinary desktop machine, tasks gradually grew, the machine was graded from improvised or retired components. The last time the upgrade was in 2006, and to this day the iron in it has not changed.
Now for hardware this is: 2xPentium 3 866mhz / 512 MB / 5x9Gb uw-scsi raid5 (soft)
Honestly, since there has never been any serious load on the server, the question of software tuning has never popped up. It hosts several home sites, a couple of galleries, several forums and a couple of test self-written cms.
And so, the day before yesterday I posted my first two posts on Habré. As I was not familiar with habraeffekt yet, in posts pictures which lay direct links to a hosting were used.
At first, the posts were posted on blogs, so there was not much traffic. But then they, having gained a certain number of points, both got home. And then I received messages from monitoring that the host was overloaded, and almost immediately - that the memory was out of date.
On the chart, it looked like this:
The first graph is loadavg, the normal load for 100% of both processors is 2, and usually it is kept within 0.5-1. The second is a memory, the third is a swap.
It can be seen that at approximately 23:30 the physical memory was over, the swap began to end. At about 23:38, the swap ended the same, and the load went up strongly, because the server has already stopped responding and tried to digest what is.
At that moment, it was necessary to restart apache, because It was he who was launched in 150 copies and devoured all the memory. 300 megabytes of physical memory and about 120 swaps were released. For a short time it helped, but after a short period of time everything was repeated.
I analyzed the statistics on hits, we got two such nice graphics:
The first chart is hits per minute.
This server is completely unloaded, so 0-10 hits are what I usually watch. Here we see that starting from 23:20 the number of hits jumped to 600-800 per minute, and this was the end of Apache’s resources. Separately, I counted hits from the main page of Habr, two dvipega standing in the headlines, they are shown in blue graph.
The second schedule is traffic. At peaks, it shows up to 30 megabytes per minute. And regularly, 3-5 megabytes per minute from the main page.
Apache settings were default:
At the moments of load loadg more than 20-30 the machine becomes very thoughtful.
Top draws the screen every 3-4 minutes, the console commands are triggered with a delay of a minute.
Here is a screenshot taken from the top at about 1:50, during the last peak.
And the logs looked like this:
pastebin.com link
The apache begins to multiply, the memory ends, the oom-killer looks at the fattest memory (mysql) and nails it. After that, he tries to nail other Apaches, but they multiply quickly enough.
Oom-killer worked 37 times during the night, five times according to mysql (during those moments when it was launched), the rest is for Apache.
The first post contained 762 kb of images, the second 361 kb. This is under the cut, from the outside - in each one picture of 27 KB.
During the time from 10:00 pm to 4:00 am, 126,939 hits were recorded in the log, 4.14 GB of traffic was sent.
The entire load is only static return, no CGI / PHP from the site was loaded, only pictures were given. During this time, there were no more than a dozen requests to other parts of the site and other sites, so they did not affect the statistics.
The Apache log itself has grown by about 60 megabytes during this time.
The conclusion from the captain is obvious: apache does not give static well under high load.
Implementation of reverse proxy, for example nginx is necessary.
NB I ask real hosters not to be measured by the pips, it is clear that you are specifically setting up software for the download, and give this traffic every minute. My article is not about that.
Thanks for attention. Ready to listen to any interesting tips.
It will not be anything new for those who have long been engaged in the web, and especially for administrators who have decided to protect against DDOS.
Just a small note about how I observed habraeffekt.
')
I have already had a dedicated server for various web projects for quite some time now, as well as a file storage facility accessible from everywhere. At the beginning of the 2000s it was just an ordinary desktop machine, tasks gradually grew, the machine was graded from improvised or retired components. The last time the upgrade was in 2006, and to this day the iron in it has not changed.
Now for hardware this is: 2xPentium 3 866mhz / 512 MB / 5x9Gb uw-scsi raid5 (soft)
Honestly, since there has never been any serious load on the server, the question of software tuning has never popped up. It hosts several home sites, a couple of galleries, several forums and a couple of test self-written cms.
And so, the day before yesterday I posted my first two posts on Habré. As I was not familiar with habraeffekt yet, in posts pictures which lay direct links to a hosting were used.
At first, the posts were posted on blogs, so there was not much traffic. But then they, having gained a certain number of points, both got home. And then I received messages from monitoring that the host was overloaded, and almost immediately - that the memory was out of date.
On the chart, it looked like this:
The first graph is loadavg, the normal load for 100% of both processors is 2, and usually it is kept within 0.5-1. The second is a memory, the third is a swap.
It can be seen that at approximately 23:30 the physical memory was over, the swap began to end. At about 23:38, the swap ended the same, and the load went up strongly, because the server has already stopped responding and tried to digest what is.
At that moment, it was necessary to restart apache, because It was he who was launched in 150 copies and devoured all the memory. 300 megabytes of physical memory and about 120 swaps were released. For a short time it helped, but after a short period of time everything was repeated.
I analyzed the statistics on hits, we got two such nice graphics:
The first chart is hits per minute.
This server is completely unloaded, so 0-10 hits are what I usually watch. Here we see that starting from 23:20 the number of hits jumped to 600-800 per minute, and this was the end of Apache’s resources. Separately, I counted hits from the main page of Habr, two dvipega standing in the headlines, they are shown in blue graph.
The second schedule is traffic. At peaks, it shows up to 30 megabytes per minute. And regularly, 3-5 megabytes per minute from the main page.
Apache settings were default:
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 10000At the moments of load loadg more than 20-30 the machine becomes very thoughtful.
Top draws the screen every 3-4 minutes, the console commands are triggered with a delay of a minute.
Here is a screenshot taken from the top at about 1:50, during the last peak.
And the logs looked like this:
pastebin.com linkThe apache begins to multiply, the memory ends, the oom-killer looks at the fattest memory (mysql) and nails it. After that, he tries to nail other Apaches, but they multiply quickly enough.
Oom-killer worked 37 times during the night, five times according to mysql (during those moments when it was launched), the rest is for Apache.
The first post contained 762 kb of images, the second 361 kb. This is under the cut, from the outside - in each one picture of 27 KB.
During the time from 10:00 pm to 4:00 am, 126,939 hits were recorded in the log, 4.14 GB of traffic was sent.
The entire load is only static return, no CGI / PHP from the site was loaded, only pictures were given. During this time, there were no more than a dozen requests to other parts of the site and other sites, so they did not affect the statistics.
The Apache log itself has grown by about 60 megabytes during this time.
The conclusion from the captain is obvious: apache does not give static well under high load.
Implementation of reverse proxy, for example nginx is necessary.
NB I ask real hosters not to be measured by the pips, it is clear that you are specifically setting up software for the download, and give this traffic every minute. My article is not about that.
Thanks for attention. Ready to listen to any interesting tips.
Source: https://habr.com/ru/post/90677/
All Articles