Accelerate PHP (with ReactPHP)
In this post, I would like to share an unusual, for the PHP world, the way of building an application, if you like, architecture. This approach allows PHP tools to increase the number of requests processed at times. I will also share my experience in this direction. Of course, this approach is not free, in terms of code requirements, but let's do everything in order.
Yes, in the end we will get a performance increase of 30 times compared to regular PHP and 6 times compared to PHP + OPcache. But from the beginning, I would like to talk about existing, popular solutions to improve the performance of PHP applications.
')
Most modern istacats use APC / OPcache and this is considered standard and maximum for PHP. This approach has the least number of disadvantages, since This is the native (native) solution offered to us by the PHP command. All is good, but the speed is not enough.
HHVM is really good, for popular Linux distros there are already repositories and it remains only to install and configure that, in general, this is not a tricky thing. But this is a development from the facebook team and at the moment, HHVM severely restricts the choice of extensions, and if you suddenly have your patches for PHP extensions, it completely puts a “cross” on a painless transition from PHP to HHVM. You can also forget about PHP 5.5. It is worth noting the excellent work of the facebook team to increase the compatibility of HHVM with the main tools and frameworks, but this figure is still around 97%.
Of the options that come to my mind are still raw HippyVM and the framework PhalconPHP. A lot of reviews have been written about Phalcon and I think it makes no sense to repeat them. HippyVM is under development, by the way, this is an alternative to HHVM from the facebook team itself, written in python, which in my opinion makes this project even more vague.
Other options offer in the comments.
The classic PHP installation includes the installation of one of the Nginx, Apache or Lighttpd web servers that process incoming HTTP requests and redirect the dynamic to PHP. There are several options for connecting PHP to a web server:
All PHP acceleration solutions are generally aimed at speeding up a slow PHP interpreter at the time of redirecting a request from a web server to a script, which, as seen in the performance tests, gives its result. But there is a disadvantage in this solution, whatever one may say, but for every PHP request, the application has to declare classes, create instances, connect to databases, read the cache — initialize its environment. And no matter how we speed up the PHP interpreter, a lot of resources are spent on all initialization and this approach is clearly far from the desired, especially for high-loaded solutions. Why it happens? PHP was originally designed as a language of templates and a set of tools, and was not conceived as a standalone web server. In addition, in PHP there is no parallel execution or even asynchronous as in node.js, and all written extensions are blocking.
But PHP does not stand still. We have our own ecosystem with thousands of tools that can be easily installed thanks to Composer. PHP borrowed a lot of patterns from languages ​​like Java, JS and others, hello to the symfony team. There are tools that allow PHP to work asynchronously. On this topic there is already an article on Habré , for this I will not repeat in the description of this approach. I can only say that the asynchronous approach allows us to create not only a chat on the websocket, but also run a full HTTP server, which means that we will not have to initialize PHP for each request. Thus, as it is not difficult to guess, such an approach will negate the time spent on launching various frameworks and ultimately improve the response time.
This solution, as is clear from the title, is built on ReactPHP. React itself is more of a creation tool than a turnkey solution. Although it already has tools for processing incoming Http connections, as well as there are various tools, for example for working with websockets or async redis, but there are no patterns, routing, etc. that are usual for modern MVC frameworks. For this purpose, we will connect ReactPHP to an existing Symfony2 application.
ReactPHP is based on eventloop and to implement this architecture it offers the choice of installing one of the ext-libevent, ext-libev, ext-event. In case of failure, React works through stream_select and the asynchronous capabilities are minimized, since in fact, everything will be performed in turn without the ability to interrupt the process. Of course, you can omit this, because essentially asynchronous, this is a series of tasks within one process. But if the function will use non-blocking calls, for example to async-redis, then the eventloop based on stream_select will have to wait for the execution of this function, since There is no possibility to interrupt the php function for the duration of the non-blocking call. Of course, this can be bypassed by splitting the functional, but I hope the essence of the problem is clear.
I am a supporter of native solutions, and the installation of pecl extensions is not very much included there. In addition, the installation of pecl will be required on the entire server park and there will be problems on the hosting. But PHP has the ability to implement Corutin using PHP 5.5. Thanks to the wonderful article from nikic ( translation to Habré ), I decided to implement my eventloop implementation based on the described nikic task scheduler. Yes, it doesn’t sound easy, and if you’re unaccustomed to it, it really requires a thorough change in the presentation of building applications in PHP. But in my opinion the future of PHP is behind such solutions.
By the way, symfony was not chosen by chance. Implementing the stack of processing incoming requests Symfony, allows us to easily work without killing PHP after each request. In the meantime, I finished this post, proposals with a similar implementation are already coming on the Symfony channel. And the developers themselves do not hide the fact that such a solution has faded in their minds since the start of the launch of version 2.
But let's move from words to deeds. First we need your favorite Linux distribution with installed and configured nginx, php-cli 5.5.x, composer and your symfony application. If you do not have a Symfony application at hand, you can take a bare installation from the Symfony site, which will be an example. If you and the composer is not familiar, then in short can be found in my article to Satis .
Create a new folder, if the project already exists, then go to it:
Install the composer:
We put symfony2.4.4:
Add these lines to your composer.json:
We start updating packages:
Prepare symfony cache:
And we are launching a web server, while using only PHP and in one instance, test one, so to speak. The port can be selected to taste:
We check that everything works by opening localhost: 8080 in your favorite browser. A welcome page from symfony should open, though the pictures will not appear and css will not be loaded. In this way, we received a PHP web server that processes and accepts HTTP requests and is not dying. But we have only 1 process, there is no processing of statics and there is no balancer. As many have guessed, for this we need nginx.
We configure nginx for proxying dynamic requests to our PHP server, simultaneously acting as a balancer, and giving statics without PHP participation:
At the same time server_name (fastapp.com) must be entered into / etc / hosts:
Now so that we don’t have to suffer with the manual launch of the n-number of processes in our PHP application (represented by nginx conf. Configured for n = 8), go to the folder of our project and execute:
We correct the ./ppm.json file:
Sometimes after the changes it is required to update the cache, perhaps this is only in my case, since when writing the article made changes in the code:
Restart our PHP Process Manager:
We get in response:
First, we check the link localhost: 5501 in the browser, if everything is open then we try to open fastapp.com. Everything should open, with pictures and css.
Now you can burn using siege or ab tools to choose from:
Here are some test results of my (not helloworld) Symfony application, on a developer machine with AMD 8core, 8RAM and Fedora20.
We increase the number of parallel requests to 256.
Unfortunately, php-fpm has collapsed and refused to work with a limit of 32 processes against 256 parallel requests.
The idea of ​​running a symfony application through ReactPHP is not mine, borrowed from Marc from his article , for which he thanks a lot. By the way, he made his measurements and even compared with HHVM. Below is a graph from his article:
My contribution is to create an eventloop based on the work of nikic and finishing the process manager to, in general, performance, as well as the nuances of launching ReactPHP with a new eventloop. Perhaps with pecl event lib, it will all work faster, did not check. Unfortunately, my current projects do not meet the required quality of the code, so the operating time is gathering dust on the shelves of the “laboratory”, since This approach requires code without errors. That is, PHP, in fact, has no right to fall, and the omnivorous nature and dynamics of PHP does not contribute to this. This can be corrected by adding PHP PM to restart the fallen processes, as well as you can add tracking of changes in the code and also restart the processes. But not yet claimed. Also on this base, you can run websocket server. What was in the plans, but so it remained there.
I left such a server for the whole weekend under load, there were no memory leaks. There is one problem that so far there is neither the time nor need to look for: for some reason, after the load, 1-2 connections remain unclosed. At low loads, it is not possible to identify the cause, but for large loads it is necessary to spend time to figure out how to identify the cause. So far, added a timer, which every 10 seconds checks current connections for validity (resource, not resource) and kills dead ones.
It is also worth noting that the application, ideally, should take into account the new possibilities of asynchrony and interrupt (yield), and not be executed monolithically. It would also be good to use non-blocking functionality.
Yes, in the end we will get a performance increase of 30 times compared to regular PHP and 6 times compared to PHP + OPcache. But from the beginning, I would like to talk about existing, popular solutions to improve the performance of PHP applications.
')
OPcache
Most modern istacats use APC / OPcache and this is considered standard and maximum for PHP. This approach has the least number of disadvantages, since This is the native (native) solution offered to us by the PHP command. All is good, but the speed is not enough.
HHVM
HHVM is really good, for popular Linux distros there are already repositories and it remains only to install and configure that, in general, this is not a tricky thing. But this is a development from the facebook team and at the moment, HHVM severely restricts the choice of extensions, and if you suddenly have your patches for PHP extensions, it completely puts a “cross” on a painless transition from PHP to HHVM. You can also forget about PHP 5.5. It is worth noting the excellent work of the facebook team to increase the compatibility of HHVM with the main tools and frameworks, but this figure is still around 97%.
Of the options that come to my mind are still raw HippyVM and the framework PhalconPHP. A lot of reviews have been written about Phalcon and I think it makes no sense to repeat them. HippyVM is under development, by the way, this is an alternative to HHVM from the facebook team itself, written in python, which in my opinion makes this project even more vague.
Other options offer in the comments.
The classic PHP installation includes the installation of one of the Nginx, Apache or Lighttpd web servers that process incoming HTTP requests and redirect the dynamic to PHP. There are several options for connecting PHP to a web server:
- mod_php (apache)
- f (ast) cgi
- php-fpm
All PHP acceleration solutions are generally aimed at speeding up a slow PHP interpreter at the time of redirecting a request from a web server to a script, which, as seen in the performance tests, gives its result. But there is a disadvantage in this solution, whatever one may say, but for every PHP request, the application has to declare classes, create instances, connect to databases, read the cache — initialize its environment. And no matter how we speed up the PHP interpreter, a lot of resources are spent on all initialization and this approach is clearly far from the desired, especially for high-loaded solutions. Why it happens? PHP was originally designed as a language of templates and a set of tools, and was not conceived as a standalone web server. In addition, in PHP there is no parallel execution or even asynchronous as in node.js, and all written extensions are blocking.
But PHP does not stand still. We have our own ecosystem with thousands of tools that can be easily installed thanks to Composer. PHP borrowed a lot of patterns from languages ​​like Java, JS and others, hello to the symfony team. There are tools that allow PHP to work asynchronously. On this topic there is already an article on Habré , for this I will not repeat in the description of this approach. I can only say that the asynchronous approach allows us to create not only a chat on the websocket, but also run a full HTTP server, which means that we will not have to initialize PHP for each request. Thus, as it is not difficult to guess, such an approach will negate the time spent on launching various frameworks and ultimately improve the response time.
This solution, as is clear from the title, is built on ReactPHP. React itself is more of a creation tool than a turnkey solution. Although it already has tools for processing incoming Http connections, as well as there are various tools, for example for working with websockets or async redis, but there are no patterns, routing, etc. that are usual for modern MVC frameworks. For this purpose, we will connect ReactPHP to an existing Symfony2 application.
ReactPHP is based on eventloop and to implement this architecture it offers the choice of installing one of the ext-libevent, ext-libev, ext-event. In case of failure, React works through stream_select and the asynchronous capabilities are minimized, since in fact, everything will be performed in turn without the ability to interrupt the process. Of course, you can omit this, because essentially asynchronous, this is a series of tasks within one process. But if the function will use non-blocking calls, for example to async-redis, then the eventloop based on stream_select will have to wait for the execution of this function, since There is no possibility to interrupt the php function for the duration of the non-blocking call. Of course, this can be bypassed by splitting the functional, but I hope the essence of the problem is clear.
I am a supporter of native solutions, and the installation of pecl extensions is not very much included there. In addition, the installation of pecl will be required on the entire server park and there will be problems on the hosting. But PHP has the ability to implement Corutin using PHP 5.5. Thanks to the wonderful article from nikic ( translation to Habré ), I decided to implement my eventloop implementation based on the described nikic task scheduler. Yes, it doesn’t sound easy, and if you’re unaccustomed to it, it really requires a thorough change in the presentation of building applications in PHP. But in my opinion the future of PHP is behind such solutions.
By the way, symfony was not chosen by chance. Implementing the stack of processing incoming requests Symfony, allows us to easily work without killing PHP after each request. In the meantime, I finished this post, proposals with a similar implementation are already coming on the Symfony channel. And the developers themselves do not hide the fact that such a solution has faded in their minds since the start of the launch of version 2.
But let's move from words to deeds. First we need your favorite Linux distribution with installed and configured nginx, php-cli 5.5.x, composer and your symfony application. If you do not have a Symfony application at hand, you can take a bare installation from the Symfony site, which will be an example. If you and the composer is not familiar, then in short can be found in my article to Satis .
Create a new folder, if the project already exists, then go to it:
mkdir fastapp && cd fastapp Install the composer:
curl -sS https://getcomposer.org/installer | php We put symfony2.4.4:
php composer.phar create-project symfony/framework-standard-edition symfdir/ 2.4.4 && mv symfdir/* ./ && rm -fr symfdir Get
ls -l drwxrwxr-x 6 user user 4.0K Apr 30 11:25 app/ drwxrwxr-x 2 user user 4.0K Apr 30 11:25 bin/ drwxrwxr-x 3 user user 4.0K Mar 14 09:37 src/ drwxrwxr-x 13 user user 4.0K Apr 30 11:25 vendor/ drwxrwxr-x 3 user user 4.0K Apr 30 11:25 web/ -rw-rw-r-- 1 user user 2.0K Mar 14 09:37 composer.json -rw-rw-r-- 1 user user 56K Apr 30 11:25 composer.lock -rwxr-xr-x 1 user user 990K Apr 30 11:23 composer.phar* -rw-rw-r-- 1 user user 1.1K Mar 14 09:37 LICENSE -rw-rw-r-- 1 user user 5.7K Mar 14 09:37 README.md -rw-rw-r-- 1 user user 1.3K Mar 14 09:37 UPGRADE-2.2.md -rw-rw-r-- 1 user user 2.0K Mar 14 09:37 UPGRADE-2.3.md -rw-rw-r-- 1 user user 356 Mar 14 09:37 UPGRADE-2.4.md -rw-rw-r-- 1 user user 8.3K Mar 14 09:37 UPGRADE.md Add these lines to your composer.json:
{ "repositories": [ { "type": "vcs", "url": "http://github.com/Imunhatep/rephp" }, { "type": "vcs", "url": "http://github.com/Imunhatep/php-pm" } ], "minimum-stability": "dev", "prefer-stable": true, "require": { "imunhatep/php-pm": "@dev" } } To look like this
{ "name": "symfony/framework-standard-edition", "license": "MIT", "type": "project", "description": "The \"Symfony Standard Edition\" distribution", "autoload": { "psr-0": { "": "src/" } }, "repositories": [ { "type": "vcs", "url": "http://github.com/Imunhatep/rephp" }, { "type": "vcs", "url": "http://github.com/Imunhatep/php-pm" } ], "minimum-stability": "dev", "prefer-stable": true, "require": { "php": ">=5.5.3", "symfony/symfony": "~2.4", "doctrine/orm": "~2.2,>=2.2.3", "doctrine/doctrine-bundle": "~1.2", "twig/extensions": "~1.0", "symfony/assetic-bundle": "~2.3", "symfony/swiftmailer-bundle": "~2.3", "symfony/monolog-bundle": "~2.4", "sensio/distribution-bundle": "~2.3", "sensio/framework-extra-bundle": "~3.0", "sensio/generator-bundle": "~2.3", "incenteev/composer-parameter-handler": "~2.0", "imunhatep/php-pm": "@dev" }, "scripts": { "post-install-cmd": [ "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile" ], "post-update-cmd": [ "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets", "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile" ] }, "config": { "bin-dir": "bin" }, "extra": { "symfony-app-dir": "app", "symfony-web-dir": "web", "incenteev-parameters": { "file": "app/config/parameters.yml" }, "branch-alias": { "dev-master": "2.4-dev" } } } We start updating packages:
php composer.phar update Get
Loading composer repositories with package information Updating dependencies (including require-dev) - Installing stack/builder (v1.0.1) Loading from cache - Installing react/promise (v2.0.0) Loading from cache - Installing guzzle/parser (v3.9.0) Loading from cache - Installing evenement/evenement (v2.0.0) Loading from cache - Installing react/react (v0.4.1) Loading from cache - Installing imunhatep/rephp (dev-master 13adf26) Cloning 13adf2697681a5954978ac56fe2c8fdf6a21dc4a - Installing imunhatep/php-pm (dev-master 02f44ec) Cloning 02f44ecb41ca5b4c81d4bb6087da7a0ed4964656 react/react suggests installing ext-libevent (Allows for use of a more performant event-loop implementation.) react/react suggests installing ext-libev (Allows for use of a more performant event-loop implementation.) react/react suggests installing ext-event (Allows for use of a more performant event-loop implementation.) Writing lock file Generating autoload files Updating the "app/config/parameters.yml" file Clearing the cache for the dev environment with debug true Installing assets using the hard copy option Installing assets for Symfony\Bundle\FrameworkBundle into web/bundles/framework Installing assets for Acme\DemoBundle into web/bundles/acmedemo Installing assets for Sensio\Bundle\DistributionBundle into web/bundles/sensiodistribution Prepare symfony cache:
php app/console cache:warmup --env=dev And we are launching a web server, while using only PHP and in one instance, test one, so to speak. The port can be selected to taste:
php bin/ppm start --workers 1 --port 8080 We check that everything works by opening localhost: 8080 in your favorite browser. A welcome page from symfony should open, though the pictures will not appear and css will not be loaded. In this way, we received a PHP web server that processes and accepts HTTP requests and is not dying. But we have only 1 process, there is no processing of statics and there is no balancer. As many have guessed, for this we need nginx.
We configure nginx for proxying dynamic requests to our PHP server, simultaneously acting as a balancer, and giving statics without PHP participation:
upstream backend { server 127.0.0.1:5501; server 127.0.0.1:5502; server 127.0.0.1:5503; server 127.0.0.1:5504; server 127.0.0.1:5505; server 127.0.0.1:5506; server 127.0.0.1:5507; server 127.0.0.1:5508; } server { root /path/to/symfony/web/; server_name fastapp.com; location / { # try to serve file directly, fallback to rewrite try_files $uri @rewriteapp; } location @rewriteapp { if (!-f $request_filename) { proxy_pass http://backend; break; } } } At the same time server_name (fastapp.com) must be entered into / etc / hosts:
127.0.0.1 fastapp.com Now so that we don’t have to suffer with the manual launch of the n-number of processes in our PHP application (represented by nginx conf. Configured for n = 8), go to the folder of our project and execute:
cp vendor/imunhatep/rephp/ppm.json ./ We correct the ./ppm.json file:
{ "bootstrap": "\\PHPPM\\Bootstraps\\Symfony", "bridge": "HttpKernel", "appenv": "dev", "workers": 8, "port": 5501 } Sometimes after the changes it is required to update the cache, perhaps this is only in my case, since when writing the article made changes in the code:
app/console cache:warmup --env=dev Restart our PHP Process Manager:
php bin/ppm start We get in response:
8 slaves (5501, 5502, 5503, 5504, 5505, 5506, 5507, 5508) up and ready. First, we check the link localhost: 5501 in the browser, if everything is open then we try to open fastapp.com. Everything should open, with pictures and css.
Now you can burn using siege or ab tools to choose from:
siege -qb -t 30S -c 128 http://fastapp.com/ Here are some test results of my (not helloworld) Symfony application, on a developer machine with AMD 8core, 8RAM and Fedora20.
Php 5.5.10, via nginx + php-fpm:
siege -qb -t 30S -c 128 http://login.dev/signup Lifting the server siege... done. Transactions: 983 hits Availability: 100.00 % Elapsed time: 29.03 secs Data transferred: 4.57 MB Response time: 0.91 secs Transaction rate: 34.26 trans/sec Throughput: 0.16 MB/sec Concurrency: 124.23 Successful transactions: 983 Failed transactions: 0 Longest transaction: 1.81 Shortest transaction: 0.42 Php 5.5.10 with OPcache enabled, via nginx + php-fpm:
siege -qb -t 30S -c 128 http://login.dev/signup Lifting the server siege... done. Transactions: 5298 hits Availability: 100.00 % Elapsed time: 29.54 secs Data transferred: 24.15 MB Response time: 0.70 secs Transaction rate: 179.35 trans/sec Throughput: 0.82 MB/sec Concurrency: 126.43 Successful transactions: 5298 Failed transactions: 0 Longest transaction: 1.68 Shortest transaction: 0.07 Php 5.5.10 with OPcache enabled, via nginx + ReactPHP + Coroutine eventloop:
siege -qb -t 30S -c 128 http://fastlogin.dev/signup Lifting the server siege... done. Transactions: 30553 hits Availability: 100.00 % Elapsed time: 29.85 secs Data transferred: 157.63 MB Response time: 0.12 secs Transaction rate: 1023.55 trans/sec Throughput: 5.28 MB/sec Concurrency: 127.43 Successful transactions: 30553 Failed transactions: 0 Longest transaction: 0.76 Shortest transaction: 0.00 We increase the number of parallel requests to 256.
Php 5.5.10 with OPcache enabled, via nginx + php-fpm
siege -qb -t 30S -c 256 http://login.dev/signup siege aborted due to excessive socket failure; Transactions: 134 hits Availability: 10.48 % Elapsed time: 1.58 secs Data transferred: 0.78 MB Response time: 1.21 secs Transaction rate: 84.81 trans/sec Throughput: 0.49 MB/sec Concurrency: 102.93 Successful transactions: 134 Failed transactions: 1145 Longest transaction: 1.56 Shortest transaction: 0.00 Unfortunately, php-fpm has collapsed and refused to work with a limit of 32 processes against 256 parallel requests.
We try Php5.5.10 + ReactPHP + Coroutine eventloop
siege -qb -t 30S -c 256 http://fastlogin.dev/signup Lifting the server siege... done. Transactions: 29154 hits Availability: 100.00 % Elapsed time: 29.16 secs Data transferred: 150.40 MB Response time: 0.25 secs Transaction rate: 999.79 trans/sec Throughput: 5.16 MB/sec Concurrency: 252.70 Successful transactions: 29154 Failed transactions: 0 Longest transaction: 3.66 Shortest transaction: 0.00 Conclusion
The idea of ​​running a symfony application through ReactPHP is not mine, borrowed from Marc from his article , for which he thanks a lot. By the way, he made his measurements and even compared with HHVM. Below is a graph from his article:
My contribution is to create an eventloop based on the work of nikic and finishing the process manager to, in general, performance, as well as the nuances of launching ReactPHP with a new eventloop. Perhaps with pecl event lib, it will all work faster, did not check. Unfortunately, my current projects do not meet the required quality of the code, so the operating time is gathering dust on the shelves of the “laboratory”, since This approach requires code without errors. That is, PHP, in fact, has no right to fall, and the omnivorous nature and dynamics of PHP does not contribute to this. This can be corrected by adding PHP PM to restart the fallen processes, as well as you can add tracking of changes in the code and also restart the processes. But not yet claimed. Also on this base, you can run websocket server. What was in the plans, but so it remained there.
I left such a server for the whole weekend under load, there were no memory leaks. There is one problem that so far there is neither the time nor need to look for: for some reason, after the load, 1-2 connections remain unclosed. At low loads, it is not possible to identify the cause, but for large loads it is necessary to spend time to figure out how to identify the cause. So far, added a timer, which every 10 seconds checks current connections for validity (resource, not resource) and kills dead ones.
It is also worth noting that the application, ideally, should take into account the new possibilities of asynchrony and interrupt (yield), and not be executed monolithically. It would also be good to use non-blocking functionality.
Source: https://habr.com/ru/post/220393/
All Articles