Comprehensive website optimization on WordPress
Dear residents of Habr!
Your attention is the story of how we optimized your site. The site is powered by Wordpress (most readers should wince at this phrase, knowing how WordPress is doing at speed). But still we did it, and the site began to fly. I’ll say right away that I can hardly be considered a server optimization professional, but what I’ve been able to achieve makes me very happy. Also, invaluable experience was gained, which I want to share with Habr's readers.
We began with a study of the causes of inhibition of the site. Personally, I like the LoadImpact service the most . Not so long ago, he had new features and a new interface, and the service itself began to be called "LoadImpact 2.0". All this is not said for advertising sake - the service is really good. You can also test the site using similar services - Pingdom Full Page Test and Google PageSpeed .
With the help of Load Impact Page Analyzer, we found a great value for the Time to first byte parameter (4 seconds), which told us that the PHP scripts that form the page run too long. It was necessary to look for the most brake components of the site.
')
Adding to the template footer.php line
we saw that 82 SQL queries are required to generate the main page, which is a lot. It became clear that you need to reduce the number of WordPress modules, and with them the number of SQL queries. After smashing up the installed WordPress plugins, of which there were about 40 pieces, we realized that most of them are really needed.
We decided to make profiling of all the PHP code in the template. We did not bother with Xdebug + Performance Testing (to do this, we had to install a bunch of third-party software), but we did a simple self-profiling of the form:
We immediately discovered the following major problems:
The biggest brake on our site was the geo-targeted advertising module. Its essence is that the site shows two types of banners - one is displayed for visitors from St. Petersburg, and the other for all others. Having gone deep into the module, I discovered this, sorry, shit code:
At this point, I even felt a little sorry for the processor, which all this time was doing empty work ...
The initial ranges of IP addresses of St. Petersburg were set in advance in the $ ip_part array. As you can see from this piece of code, for each function call, for some reason, a dynamic array of $ ip was created and filled with string (!) Data with IP addresses, stupidly after 4 nested loops. In each iteration of the loop, the visitor's IP address was compared line by line (!) With the resulting IP address string. Fix it was simple:
I do not deny that this code can be rewritten more optimally, however, even after this edit, the site has earned 2 seconds faster!
The second plugin called Embedded Stock Data, which on our page showed quotes Apple. It was found that this bastard at every page load went to another site (!) Through CURL. Naturally, this greatly increased the page load time. Solved the problem in a natural way - on the server once a minute, a script was called on the crown, which downloaded the required quote and saved it in a special file. Next we include this file. This gave us about 500 ms.
Side blocks "Review of applications" and "Most Interesting" also performed significant time. Looking inside the template (sidebar.php), I saw the classic mistake of novice web programmers called “ORDER BY RAND ()”. More information about the essence of this problem can, for example, read here .
For those interested, there was this code:
In it, we replaced the highlighted line with
The essence of the solution - we pre-select the 20 most recent entries, then choose from these 20 entries a random 4 pieces, and then select from the base posts with these numbers.
On this we saved another half a second.
Our iron server initially hosted a large number of sites, but the most important for traffic and value was AppleInsider.ru . We wanted to protect this site from the neighborhood with other sites, and it was decided to transfer it to the cloud of Clodo . The main advantage of Clodo can be called the "elasticity" of tariffing - how many resources you use, you pay so much. Hand on heart, I would also like to mention the technical support of the company, which even behaved correctly in a non-VIP version, answered all our tricky questions and helped in setting up.
“Under the guise of” with the transfer, some changes were made, for example, we removed the completely heavy Apache web server, and instead we made a bunch of nginx + php-fpm, which eats less memory. Nginx worked in the usual way - it gave statics, and everything else was redirected to the index.php script, in which further WordPress logic goes.
In order not to take away the resources of the hardware allocated for the nginx + php-fpm web server, we brought the MySQL database server to a separate server, connecting it to the web server via a local network.
We also set up two separate “cloud” servers - for our podcast AppleInsider.ru and Apple Stub and for online text broadcasts. We will tell about them in more detail next time.
The transition to Clodo has not significantly reduced the time of page generation, but this has secured the site itself.
At this stage, the page was formed in about 1 second. Already not bad, but it was not enough for us, because there were still a few moments that could be optimized.
For example, use a plugin to cache WordPress pages called WP Super Cache . It works like a regular cache - the user enters the page, it is generated and placed in the / wp-content / cache / supercache directory. The next visitor will be given a static, pre-formed HTML file. It would seem that everything is super - the site is flying ...
However, as it was soon discovered, the use of caching broke geotargeted advertising. Check for the entry of the visitor's address in the address range of St. Petersburg was performed at the PHP level, i.e. already behind the cache. Accordingly, the result of this check "forever" remained in the cache. But we still had to distinguish the display of banners.
We solved this task as follows: Javascipt instructions were placed in the template of the page in the right places, indicating that you need to load content from a specific URL into this div block. Thus, the user freely receives the cached page with Javascript instructions, and after the first letters appear, advertising and other structural blocks of the site, stored at separate special URL addresses, begin to load. Visually, this happens very quickly. So we managed to achieve two goals - firstly, due to caching the site was significantly accelerated, and secondly, geo-targeted advertising was working correctly.
The last step we made in optimization was to transfer static content (images, CSS styles, Javascript) to CloudStorage. Read more about it here . In a nutshell - this is the Content Delivery Network from Clodo, and in Russian - a very fast server, specially sharpened for the distribution of statics.
The integration with WordPress consists in correctly configuring two plugins - the CDN Sync Tool (for Clodo you need to use the special assembly of the PHP Cloud Files API library ) and the WP Super Cache.
The first plugin, CDN Sync Tool , is committed to uploading WordPress content to the repository, performing some actions along with this content. With the WP Minify plugin, it can compress CSS and JS files. He can also compress downloaded images on the fly. In this case, the GD library is used; for us, the GD Compression Level = 2 compression level turned out to be the most acceptable - to put it in terms of other programs, this is 90% quality. The most important thing, of course, is that with the standard upload of pictures by the authors, they began to be automatically uploaded to the repository.
For statics, we created a separate subdomain static.appleinsider.ru, and the initial content download was done using CyberDuck - this is a fairly convenient program for working with FTP, SFTP, WebDAV, Cloud Files, Google Docs and Amazon S3.
Here is a screenshot of the Clodo CDN Sync Tool plugin (clickable):
The second plugin, which we have already mentioned, WP Super Cache, is tricky - it takes the generated page and replaces all URLs for static files with URLs in the repository. For example, http://www.appleinsider.ru/Angry-Birds.jpg will be http://static.appleinsider.ru/Angry-Birds.jpg
Now, if there is no page in the cache, the page loads in 1.25 seconds, if there is 250 ms in the cache, and this taking into account all 4 factors - DNS Lookup + Connection + Time to first + Download. I think this is a great result!
In conclusion, I will give dry facts - statistics on the use of resources and financial costs.
For the week of work November 23, 2011 - November 30, 2011:
Total for the week of the site - 1771.16 rubles. Quite a lot, but stability and satisfied users are worth it :-)
Really looking forward to your comments.
Your attention is the story of how we optimized your site. The site is powered by Wordpress (most readers should wince at this phrase, knowing how WordPress is doing at speed). But still we did it, and the site began to fly. I’ll say right away that I can hardly be considered a server optimization professional, but what I’ve been able to achieve makes me very happy. Also, invaluable experience was gained, which I want to share with Habr's readers.
Go
We began with a study of the causes of inhibition of the site. Personally, I like the LoadImpact service the most . Not so long ago, he had new features and a new interface, and the service itself began to be called "LoadImpact 2.0". All this is not said for advertising sake - the service is really good. You can also test the site using similar services - Pingdom Full Page Test and Google PageSpeed .
With the help of Load Impact Page Analyzer, we found a great value for the Time to first byte parameter (4 seconds), which told us that the PHP scripts that form the page run too long. It was necessary to look for the most brake components of the site.
')
Adding to the template footer.php line
<!-- <?php echo get_num_queries(); ?> queries -->
we saw that 82 SQL queries are required to generate the main page, which is a lot. It became clear that you need to reduce the number of WordPress modules, and with them the number of SQL queries. After smashing up the installed WordPress plugins, of which there were about 40 pieces, we realized that most of them are really needed.
We decided to make profiling of all the PHP code in the template. We did not bother with Xdebug + Performance Testing (to do this, we had to install a bunch of third-party software), but we did a simple self-profiling of the form:
<!-- HEADER01 - <?php print microtime(true) ?> -->
We immediately discovered the following major problems:
- geo-targeting advertising module
- Apple stock price module
- side blocks "Review of applications" and "most interesting"
Time
The biggest brake on our site was the geo-targeted advertising module. Its essence is that the site shows two types of banners - one is displayed for visitors from St. Petersburg, and the other for all others. Having gone deep into the module, I discovered this, sorry, shit code:
if ($ ip_part)
{
for ($ y1 = $ ip_part [0] ['start']; $ y1 <= $ ip_part [0] ['end']; $ y1 ++)
for ($ y2 = $ ip_part [1] ['start']; $ y2 <= $ ip_part [1] ['end']; $ y2 ++)
for ($ y3 = $ ip_part [2] ['start']; $ y3 <= $ ip_part [2] ['end']; $ y3 ++)
for ($ y4 = $ ip_part [3] ['start']; $ y4 <= $ ip_part [3] ['end']; $ y4 ++)
{
$ ip [] = $ y1. ".". $ y2. ".". $ y3. ".". $ y4;
if ($ ip_addr == $ y1. ".". $ y2. ".". $ y3. "." $ y4) return true;
}
}
At this point, I even felt a little sorry for the processor, which all this time was doing empty work ...
The initial ranges of IP addresses of St. Petersburg were set in advance in the $ ip_part array. As you can see from this piece of code, for each function call, for some reason, a dynamic array of $ ip was created and filled with string (!) Data with IP addresses, stupidly after 4 nested loops. In each iteration of the loop, the visitor's IP address was compared line by line (!) With the resulting IP address string. Fix it was simple:
$ ip_tmp = explode (".", $ ip_addr);
if ($ ip_part)
if (($ ip_part [0] ['start'] <= $ ip_tmp [0]) and ($ ip_tmp [0] <= $ ip_part [0] ['end']))
if (($ ip_part [1] ['start'] <= $ ip_tmp [1]) and ($ ip_tmp [1] <= $ ip_part [1] ['end']))
if (($ ip_part [2] ['start'] <= $ ip_tmp [2]) and ($ ip_tmp [2] <= $ ip_part [2] ['end']))
if (($ ip_part [3] ['start'] <= $ ip_tmp [3]) and ($ ip_tmp [3] <= $ ip_part [3] ['end']))
return true;
I do not deny that this code can be rewritten more optimally, however, even after this edit, the site has earned 2 seconds faster!
Two
The second plugin called Embedded Stock Data, which on our page showed quotes Apple. It was found that this bastard at every page load went to another site (!) Through CURL. Naturally, this greatly increased the page load time. Solved the problem in a natural way - on the server once a minute, a script was called on the crown, which downloaded the required quote and saved it in a special file. Next we include this file. This gave us about 500 ms.Three
Side blocks "Review of applications" and "Most Interesting" also performed significant time. Looking inside the template (sidebar.php), I saw the classic mistake of novice web programmers called “ORDER BY RAND ()”. More information about the essence of this problem can, for example, read here .
For those interested, there was this code:
$ args = array ( 'caller_get_posts' => '1', 'post__not_in' => $ sticky, 'cat' => '75', 'fields' => 'ids', 'post_per_page' => '20', 'orderby' => 'id', ); $ sk_count = new WP_Query ($ args); $ args = array ( 'caller_get_posts' => '1', 'post__not_in' => $ sticky, 'showposts' => '4', 'cat' => '75', <b> 'orderby' => 'rand' </ b> ); $ sk_posts = new WP_Query ($ args);
In it, we replaced the highlighted line with
'post__in' => array_rand (array_flip ($ sk_count-> posts), 4)
The essence of the solution - we pre-select the 20 most recent entries, then choose from these 20 entries a random 4 pieces, and then select from the base posts with these numbers.
On this we saved another half a second.
Moving to Clodo
Our iron server initially hosted a large number of sites, but the most important for traffic and value was AppleInsider.ru . We wanted to protect this site from the neighborhood with other sites, and it was decided to transfer it to the cloud of Clodo . The main advantage of Clodo can be called the "elasticity" of tariffing - how many resources you use, you pay so much. Hand on heart, I would also like to mention the technical support of the company, which even behaved correctly in a non-VIP version, answered all our tricky questions and helped in setting up.
“Under the guise of” with the transfer, some changes were made, for example, we removed the completely heavy Apache web server, and instead we made a bunch of nginx + php-fpm, which eats less memory. Nginx worked in the usual way - it gave statics, and everything else was redirected to the index.php script, in which further WordPress logic goes.
In order not to take away the resources of the hardware allocated for the nginx + php-fpm web server, we brought the MySQL database server to a separate server, connecting it to the web server via a local network.
We also set up two separate “cloud” servers - for our podcast AppleInsider.ru and Apple Stub and for online text broadcasts. We will tell about them in more detail next time.
The transition to Clodo has not significantly reduced the time of page generation, but this has secured the site itself.
WP Super Cache and Asynchronous Advertising
At this stage, the page was formed in about 1 second. Already not bad, but it was not enough for us, because there were still a few moments that could be optimized.
For example, use a plugin to cache WordPress pages called WP Super Cache . It works like a regular cache - the user enters the page, it is generated and placed in the / wp-content / cache / supercache directory. The next visitor will be given a static, pre-formed HTML file. It would seem that everything is super - the site is flying ...
However, as it was soon discovered, the use of caching broke geotargeted advertising. Check for the entry of the visitor's address in the address range of St. Petersburg was performed at the PHP level, i.e. already behind the cache. Accordingly, the result of this check "forever" remained in the cache. But we still had to distinguish the display of banners.
We solved this task as follows: Javascipt instructions were placed in the template of the page in the right places, indicating that you need to load content from a specific URL into this div block. Thus, the user freely receives the cached page with Javascript instructions, and after the first letters appear, advertising and other structural blocks of the site, stored at separate special URL addresses, begin to load. Visually, this happens very quickly. So we managed to achieve two goals - firstly, due to caching the site was significantly accelerated, and secondly, geo-targeted advertising was working correctly.
ClodoStorage
The last step we made in optimization was to transfer static content (images, CSS styles, Javascript) to CloudStorage. Read more about it here . In a nutshell - this is the Content Delivery Network from Clodo, and in Russian - a very fast server, specially sharpened for the distribution of statics.
The integration with WordPress consists in correctly configuring two plugins - the CDN Sync Tool (for Clodo you need to use the special assembly of the PHP Cloud Files API library ) and the WP Super Cache.
The first plugin, CDN Sync Tool , is committed to uploading WordPress content to the repository, performing some actions along with this content. With the WP Minify plugin, it can compress CSS and JS files. He can also compress downloaded images on the fly. In this case, the GD library is used; for us, the GD Compression Level = 2 compression level turned out to be the most acceptable - to put it in terms of other programs, this is 90% quality. The most important thing, of course, is that with the standard upload of pictures by the authors, they began to be automatically uploaded to the repository.
For statics, we created a separate subdomain static.appleinsider.ru, and the initial content download was done using CyberDuck - this is a fairly convenient program for working with FTP, SFTP, WebDAV, Cloud Files, Google Docs and Amazon S3.
Here is a screenshot of the Clodo CDN Sync Tool plugin (clickable):
The second plugin, which we have already mentioned, WP Super Cache, is tricky - it takes the generated page and replaces all URLs for static files with URLs in the repository. For example, http://www.appleinsider.ru/Angry-Birds.jpg will be http://static.appleinsider.ru/Angry-Birds.jpg
Results
Now, if there is no page in the cache, the page loads in 1.25 seconds, if there is 250 ms in the cache, and this taking into account all 4 factors - DNS Lookup + Connection + Time to first + Download. I think this is a great result!
In conclusion, I will give dry facts - statistics on the use of resources and financial costs.
For the week of work November 23, 2011 - November 30, 2011:
Web
| Scale Server - RAM resources | 45708.00 GB * min | 457.080 rub. |
| Scale Server - processor resources | 1027116.50 CPU sec | 427.300 rub. |
| Use of disk resources | 5760.00 GB * hour | 57.600 rub. |
| Incoming traffic (full gigabytes) | 34 GB | 6.800 rub. |
| Outgoing traffic (full gigabytes) | 129 GB | 129.870 rub. |
| Disk usage for backups | 1910.00 GB * hour | 19.100 rub. |
| SMS notifications | 1 PC. | 5.000 rub. |
| Total debited from the account | 1102.75 rub. |
DB
| Scale Server - RAM resources | 22924.00 GB * min | 229.240 rub. |
| Scale Server - processor resources | 67490.34 CPU sec | 28.080 rub. |
| Use of disk resources | 3840.00 GB * hour | 38.400 rubles |
| Incoming traffic (full gigabytes) | 1 GB | 0.200 rub. |
| Outgoing traffic (full gigabytes) | 12 GB | 12.000 rub. |
| Disk usage for backups | 1910.00 GB * hour | 19.100 rub. |
| Total debited from the account | 327.02 rub. |
Cloudstorage
| Cloud Storage - Disk Usage | 1139.00 GB * hour | 11.390 rub. |
| Cloud Storage - incoming traffic (full gigabytes) | - | not charged |
| Cloud Storage - outgoing traffic (full gigabytes) | 330 GB | 330.000 rub. |
| Total debited from the account | 341.39 rub. |
Total for the week of the site - 1771.16 rubles. Quite a lot, but stability and satisfied users are worth it :-)
Really looking forward to your comments.
Source: https://habr.com/ru/post/132817/
All Articles