Millions of visitors to WordPress against the server

My server, which will be the hero of the subsequent narration, is the usual middle-class server rented from FirstDedic with a DualCore Xeon E3110 3.00Ghz processor. RAM was installed 4 GB hard drive 500 GB. On the server, nginx 1.01 was installed as a frontend, and apache 2 as a backend, running scripts in CGI mode.

The story happened to a site that was posted on my server, in fact, not a site, but someone else’s personal blog . Earlier on the blog, traffic peaks of up to 10,000 per day were observed, but the server coped with such a load with a bang absolutely without optimization on standard configuration files.

And so, one fine “Women's Day”, in the morning, an SMS comes from the site monitoring service that the server is unavailable. Naturally, I immediately wake up from such news, and try to ping the server. Ping was present, but very sluggish. An SSH connection cannot be established because all server resources are given to an unknown process, or processes.

Having connected via KVM, I sent the server to a reboot, and immediately after the download I connected via SSH. In the processes, I saw a terrible picture: about 1000 php processes were launched on behalf of the blog's author, in addition, Load averages is more than a hundred. A very scary indicator that shows how much you have to wait for the process of your turn for a portion of resources.
')
Naturally, I only had time to see this by running the top command. Within a minute, the server stopped responding, and I had to restart it again, and immediately turn off apache after the restart. Now I am guaranteed to get a server that does not consume all resources. I started the analysis, I derived the number of open connections with the netstat team and was horrified. There were more than 10,000 connections established with nginx. This means that in the last minute there were 10 thousand attempts to access the client's site - a good load.

Trying to rummage through the WordPress settings, naturally with the consent of the client, I found that the WP Super Cache plugin was activated for caching, which I turned off because it was he who gave the most load to the file system. Turning off the plug-in, the site began to perform a lot of queries to the database - no wonder. Therefore, the first thing I did was turn on the query caching system in MySQL, since the load was given only by one page, to which there were many transitions. After turning on query caching, the database sighed more freely, but not as much as we would like, even though Wordpress itself was giving the main load.

Turning off all possible plug-ins and rewriting the theme with the least amount of requests, the load did not decrease. I had to go to extreme measures - I turned on forced caching of proxied requests in nginx. For this, I wrote the following line in the http section

proxy_cache_path /path/to/cache levels=1:2 keys_zone=wpblog:10m max_size=10m; 

In the necessary section of the server we register:

 proxy_cache_valid 200 3m; proxy_cache wpblog; proxy_pass http://127.0.0.1:8080; 

As soon as I did, the server load dropped dramatically. However, we received a lot of inconvenience associated with administration and commenting. Despite the inconvenience, the problem was solved. However, consciousness prompted that manually including such caching would not always be time and opportunity, and leaving as is is not an option for the author of the blog. Thus, it was necessary to prohibit caching for authorized users, and users who left their comments. The result is approximately the following server section:

 proxy_cache_valid 200 3m; location / { if ($http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) { set $do_not_cache 1; } proxy_no_cache $do_not_cache; proxy_cache_bypass $do_not_cache; proxy_cache wpblog; proxy_pass http://127.0.0.1:8080; } location ~* wp\-.*\.php|wp\-admin { proxy_pass http://127.0.0.1:8080; } location ~* ^.+\.(jpg|jpeg|gif|png|svg|js|css|mp3|ogg|mpe?g|avi|zip|gz|bz2?|rar)$ { root /path/to/static; access_log off; expires max; add_header Last-Modified: $date_gmt; } location ~* \/[^\/]+\/(feed|\.xml)\/? { proxy_pass http://127.0.0.1:8080; } 

After this, the inconvenience in the work is fully compensated by uninterrupted operation.

As a result of all of the above, the server was unavailable for 2 hours, during which time the traffic flow significantly decreased, but after this incident there were other similar holiday traffic spikes that the site successfully withstood without giving a noticeable load to the server. Ever since I have been trying to put this configuration on all the WordPress hosted sites.

Google Analytics on the client's site, after the server finally rose, showed 6000 visitors online. This figure quickly fell, because the relevance of the request for which the site was in the top of all search engines was lost every minute. By the end of the day, the number of visitors became seven-digit, but the owner of the resource still looks at me like a wolf, because the figure could be several times higher, as well as his income.

Using this config, I can say with confidence that the server transfers several similar sites. As my project after adding to Google News began to attract a lot of traffic, as was the case with Yandex blogs.