Custom PHP Project Optimization

In this publication I want to talk about how non-traditional methods we were able to reduce the load on the servers and speed up the processing time of the page several times.



Traditional methods, I think, are known to all:

• Optimization of SQL queries;
• Find and fix bottlenecks;
• Transition to Memcache for frequently used data;
• Installing APC, XCache and the like;
• Client optimization: CSS sprites, etc.

')
In our project, all this was done, but the problem of page processing speed remained. The average page processing speed was around 500ms. At one point, the idea came to analyze what resources there are and what they can spend.

After the analysis, the following main resources were identified that need to be monitored:

1. CPU time;
2. RAM;
3. Timeout of other resources (MySQL, memcache);
4. Disk latency.

In our project, we almost do not use writing to disk, so we immediately excluded the 4th item.

Next began the search for bottlenecks.

Stage 1

The first thing that was tried was profiling MySQL queries, searching for slow queries. The approach did not bring much success: several requests were optimized, but the average page processing time did not change much.

Stage 2

Next was an attempt to profile code with the help of XHProf. This gave acceleration in narrow places and could reduce the load by about 10-15%. But this did not solve the main problem either. (If you're interested, I can write a separate article on how to optimize using XHProf. You just let me know in the comments.)

Stage 3

In the third stage, the thought came to see how much memory is spent on processing the request. It turned out that this is the problem - a simple request may require you to load up to 20mb code in the RAM. And the reason for this was incomprehensible, since there is a simple page load - without queries to the database, or downloading large files.

It was decided to write an analyzer and find out how much memory each php file requires when it is turned on.

The analyzer is very simple: the project already had an autoloader for the files, which based on the class name itself loaded the required file (autoload). It simply added 2 lines: how much memory was there before the file was loaded, how much it became after.

Code example:

Profiler::startLoadFile($fileName); Include $fileName; Profiler::endLoadFile($fileName); 

The first line saves how much memory was in the beginning, the last line - subtracts the difference and adds to the list of downloads. The profiler also stores the file download queue (backtrack).

At the end of the execution of all the code, we added the output of the panel with the collected information.

 echo Profiler:showPanel(); 

The analysis showed very interesting reasons why excessive memory can be used. After we analyzed all the pages and removed the download of extra files, the page began to require less than 8mb of memory. Updating the page has become faster, the load on the servers has decreased and the same machines have the ability to process more clients.

Next comes the list of things that we changed. It is worth noting that the functionality itself has not changed. Only the structure of the code has changed.

All examples are made specially simplified, since there is no possibility to provide the original project code.

Example 1: Loading, but not using large parent classes

Example:

 class SysPage extends Page{ static function helloWorld(){ echo "Hello World"; } } 

The file itself is very small, but at the same time, it pulls at least one other file, which can be quite large.

There are several solutions:

1. Remove inheritance where not needed;
2. Make sure all parent classes are minimal in size.

Example 2: Using long classes, although only a small part of the class is needed

Very similar to the previous item. Example:

 class Page{ public static function isActive(){} //     1000-2000  } 

Naturally, PHP does not know that you need only 1 function. As soon as a call is made to this class, the entire file is loaded.

Decision:
If there are similar classes, then it is worth allocating the functionality that is used everywhere in a separate class. In the current class, you can refer to the new class for compatibility.

Example 3: Loading a large class for constants only

Example:

 $CONFIG['PAGES'] = array( 'news' => Page::PAGE_TYPE1, 'about' => Page::PAGE_TYPE2, ); 

Or:

 if(get('page')==Page::PAGE_TYPE1){} 

Those. The example is similar to the previous one, only now it is constant. The solution is the same as in the past case.

Example 4: Auto-creation in the constructor of classes that may not be used

Example:

 class Action{ public $handler; public function __construct(){ $this->handler = new Handler(); } public function handle1(){} public function handle2(){} public function handle3(){} public function handle4(){} public function handle5(){} public function handle6(){} public function handle7(){} public function handle8(){} public function handle9(){} public function handle10(){ $info = $this->handler->getInfo(); } } 

If Handler is used frequently, then there is no problem. A completely different question if it is used only in 1 of the functions out of 20.

Decision:
We remove from the constructor, go to the lazy loading through the __get magic method or, for example:

 public function handle10(){ $info = $this->handler()->getInfo(); } public function handler(){ if($this->handler===null) $this->handler = new Handler(); return $this->handler; } 

Example 5: Uploading unnecessary language files / configs

Example: You have a large file with settings for all pages of all menu items. There may be many arrays in the file.

If part of the data is rarely used and part is often used, then this file is a candidate for separation.

Those. It is worth loading these settings only when they are needed.

An example of the use of memory with us:

A file of 16 kb, just an array of data - requires from 100 kb and above.

Example 6: Using serialize / unserialize

Some of the settings that change frequently are stored in a file in serialize format. We found that it loads both the processor and the RAM, and it turned out that on PHP version 5.3.x, the unserialize function has a very strong memory leak.

After optimization, we got rid of these functions as much as possible where possible. We decided to store the data in files as an array saved via var_export and load it back with include / require. Thus, we were able to use the APC mode.

Unfortunately, for the data stored in memcache, this approach does not work.

Results

All these examples are easily found and very easily corrected without much changing the structure of the project. We took the rule: “any files that require more than 100kb should be checked to see if their downloads can be optimized. We also looked at the situation when several files were loaded from the same branch. In this situation, we looked at whether it was possible not to load the entire file branch at all. The key thought was: “everything we download should make sense. If you can not download in any way - it is better not to download.

Conclusion

After we removed everything above, one request to the server began to require about 2 times less RAM. This gave us a decrease in processor load of about 2 times without reconfiguring servers and reducing the average download speed of one page several times.

Future plans

If it is interesting, there is an idea in the plans to describe ways of profiling code and finding bottlenecks using XHprof.