HHVM: testing performance and compatibility with Joomla, Bitrix and Wordpress
About HHVM already wrote on Habré. In short: HHVM is a Facebook virtual machine that, due to the translation and JIT compilation of the code, allows you to speed up PHP several times. The developers also promise almost complete compatibility with PHP 5.4.
I decided to compare HHVM with the native interpreter on several tests, as well as try to run CMS on it.
')
Testing was carried out in a virtual machine with the configuration of a processor 2 core Core i5 @ 3.3 GHz, 8 GB of RAM.
OS: Ubuntu 12.04 x86_64.
PHP: 5.3.10-1ubuntu3.7
Nginx: 1.1.19
Apache: 2.2.22
HHVM: 2.1.0-dev
In the settings, the execution timeout and memory_limit were increased, APC was enabled. No additional tuning was made.
HHVM is installed from source according to the instructions .
Several scripts were prepared, each of which was launched 50 times: 10 times in PHP CLI, HHVM CLI, Apache + mod_php, nginx + php-fpm, and nginx + HHVM. The average execution time in seconds is presented in the table. The last two columns show the gain in execution speed when using HHVM.
1 - Apache could not execute asort.php and sort.php - dropped from out of memory
asort - sorting an associative array of strings with 10 million elements
bcmath_factorial - factorial calculation of the number 1000 by the bcmul function
curl_multi - download 100 files of 1 MB in 10 streams from the Internet
domdocument - analysis of large XML (122 thousand tags) - for example, took sitemap ag.ru
for - 3 nested for for 1000 iterations (total 1 billion)
geoip - 100 thousand IP searches based on GeoIP
getid3 - getting ID3 tags from MP3 using getID3 library () - 10 thousand times
mb_strings - use mb_strlen, mb_ereg_replace, mb_substr, mb_strpos, mb_strtolower 1000 times on the first volume of "War and Peace"
md5 - generation of md5 from numbers from 1 to 100 million
preg - search preg_match_all hex codes of colors, tags and URLs on an HTML page - 1000 times (for example, the main Habra is taken)
prime - getting all prime numbers from 2 to 1 million by bruising "in the forehead"
sort - sorting an array of rows of 10 million items with numeric indices
sqlite - 10 thousand SELECT queries to the sqlite3 database
strings - use strlen, str_replace, substr, strpos, strtolower 10 thousand times on the first volume of "War and Peace"
tcp_client - 10 thousand connections to localhost and reading 2 Mb by TCP ( CHARGEN )
tcpdf - generation of multipage PDF by TCPDF library - 10 times (taken example_061.php)
thumbnail - 10 thousand thumbnail creations using GD imagecopyresized ()
I also wanted to get rid of the test with Smarty , but HHVM was unable to perform it.
Of course, benchmarks can not fully show the situation with performance. But still some conclusions can be drawn.
The fact that Wordpress runs on HHVM was known in advance . Only a small adjustment in one file is needed.
A simple test: loading the main page 100 times in one thread.
nginx + HHVM: 4.498 s
Apache + mod_php: 9.997 c
Apache + mod_php + APC: 6.567 c
nginx + php-fpm + APC: 5.460 s
I draw your attention that this was a test of the page response time, and not load testing in several streams.
Bitrix and Joomla on HHVM refused to start, giving a fatal error. I suspect that Bitrix has no hope of launching at all, since it uses the deprecated functions and mbstring.func_overload.
Internet review has shown that it is Worpress that is easiest to launch under HHVM for now. With other CMS and frameworks, things are not so bright, for example, in a recent topic on Habré, Symfony2 did not take off .
Perhaps, for whom productivity is very important, you should immediately write a framework for HHVM?
1) HHVM gives a good performance boost on some tasks - it is possible to use it to start the processes of background processing of something. Remember that HHVM is not effective when using global variables.
2) Currently, the use of existing libraries, frameworks and CMS with HHVM is difficult. HHVM is sharpened for performance and may not support various beautiful designs. However, in the future the situation will improve, because and HHVM may start supporting a better language, and developers may be puzzled by adapting their code to HHVM.
3) On Wordpress, HHVM showed an impressive performance increase - 2 times compared to mod_php.
PS test scripts , raw data (Excel)
I decided to compare HHVM with the native interpreter on several tests, as well as try to run CMS on it.
')
Test conditions
Testing was carried out in a virtual machine with the configuration of a processor 2 core Core i5 @ 3.3 GHz, 8 GB of RAM.
OS: Ubuntu 12.04 x86_64.
PHP: 5.3.10-1ubuntu3.7
Nginx: 1.1.19
Apache: 2.2.22
HHVM: 2.1.0-dev
In the settings, the execution timeout and memory_limit were increased, APC was enabled. No additional tuning was made.
HHVM is installed from source according to the instructions .
Benchmarks
Several scripts were prepared, each of which was launched 50 times: 10 times in PHP CLI, HHVM CLI, Apache + mod_php, nginx + php-fpm, and nginx + HHVM. The average execution time in seconds is presented in the table. The last two columns show the gain in execution speed when using HHVM.
| CLI | HTTP | Increase speed compared | |||||
| Test | Php | HHVM | Apache + mod_php | Nginx + php_fpm | Nginx + HHVM | PHP CLI | PHP-FPM |
| asort | 42.4 | 17.9 | N / A 1 | 40.9 | 18.9 | 137.0% | 116.7% |
| bcmath_factorial | 3.4 | 3.0 | 3.4 | 3.4 | 2.8 | 13.9% | 21.5% |
| curl_multi | 8.6 | 8.4 | 9.7 | 9.0 | 9.7 | 2.3% | -7.0% |
| domdocument | 50.2 | 51.0 | 54.3 | 50.1 | 50.1 | -1.5% | 0.0% |
| for | 60.5 | 36.9 | 60,8 | 61.0 | 211.2 | 63.7% | -71.1% |
| geoip | 14.4 | 15.6 | 14.7 | 14.9 | 5.0 | -7.8% | 198.0% |
| getid3 | 46.5 | 26.9 | 46.2 | 45.6 | 8.5 | 72.4% | 435.5% |
| mb_strings | 22,8 | 16.8 | 7.0 | 22.2 | 17.0 | 35.8% | 31.0% |
| md5 | 36.8 | 35.7 | 37.0 | 37.2 | 27.2 | 3.2% | 36.7% |
| preg | 11.6 | 10.9 | 11.6 | 11.7 | 10.9 | 6.1% | 7.9% |
| prime | 4.2 | 5.6 | 4.2 | 4.3 | 2.5 | -24,1% | 71.8% |
| sort | 46.3 | 17,8 | N / A 1 | 44.1 | 17.5 | 160.1% | 151.8% |
| sqlite | 12.4 | 10.0 | 12.6 | 12.2 | 15.2 | 23.7% | -19.9% |
| strings | 38.6 | 37.5 | 37,8 | 38.2 | 36.0 | 3.0% | 6.3% |
| tcp_client | 15.4 | 13.2 | 15.5 | 17.3 | 13.6 | 16.9% | 27.7% |
| tcpdf | 3.3 | 2.4 | 3.4 | 4.4 | 1.4 | 38.7% | 215.1% |
| thumbnail | 100.8 | 94.8 | 93.7 | 99.9 | 94.2 | 6.3% | 6.1% |
1 - Apache could not execute asort.php and sort.php - dropped from out of memory
asort - sorting an associative array of strings with 10 million elements
bcmath_factorial - factorial calculation of the number 1000 by the bcmul function
curl_multi - download 100 files of 1 MB in 10 streams from the Internet
domdocument - analysis of large XML (122 thousand tags) - for example, took sitemap ag.ru
for - 3 nested for for 1000 iterations (total 1 billion)
geoip - 100 thousand IP searches based on GeoIP
getid3 - getting ID3 tags from MP3 using getID3 library () - 10 thousand times
mb_strings - use mb_strlen, mb_ereg_replace, mb_substr, mb_strpos, mb_strtolower 1000 times on the first volume of "War and Peace"
md5 - generation of md5 from numbers from 1 to 100 million
preg - search preg_match_all hex codes of colors, tags and URLs on an HTML page - 1000 times (for example, the main Habra is taken)
prime - getting all prime numbers from 2 to 1 million by bruising "in the forehead"
sort - sorting an array of rows of 10 million items with numeric indices
sqlite - 10 thousand SELECT queries to the sqlite3 database
strings - use strlen, str_replace, substr, strpos, strtolower 10 thousand times on the first volume of "War and Peace"
tcp_client - 10 thousand connections to localhost and reading 2 Mb by TCP ( CHARGEN )
tcpdf - generation of multipage PDF by TCPDF library - 10 times (taken example_061.php)
thumbnail - 10 thousand thumbnail creations using GD imagecopyresized ()
I also wanted to get rid of the test with Smarty , but HHVM was unable to perform it.
Interpretation of results
Of course, benchmarks can not fully show the situation with performance. But still some conclusions can be drawn.
- The execution time of each test run differed slightly from the average, which means that the result can be considered stable.
- There are two anomalous results: the mb_string test on Apache + mod_php is too fast and the for for nginx + HHVM is too slow. I rechecked the tests, but I do not know what was the matter.
- What is in the CLI version, that when used for web HHVM gives an advantage in half the tests.
- In some cases, using HHVM as a nginx backend (i.e., not as a CLI) gives an even greater increase - apparently, due to JIT activation. Notice how much faster getid3 and TCPDF libraries work.
CMS testing
Wordpress
The fact that Wordpress runs on HHVM was known in advance . Only a small adjustment in one file is needed.
A simple test: loading the main page 100 times in one thread.
nginx + HHVM: 4.498 s
Apache + mod_php: 9.997 c
Apache + mod_php + APC: 6.567 c
nginx + php-fpm + APC: 5.460 s
I draw your attention that this was a test of the page response time, and not load testing in several streams.
Bitrix and Joomla
Bitrix and Joomla on HHVM refused to start, giving a fatal error. I suspect that Bitrix has no hope of launching at all, since it uses the deprecated functions and mbstring.func_overload.
Internet review has shown that it is Worpress that is easiest to launch under HHVM for now. With other CMS and frameworks, things are not so bright, for example, in a recent topic on Habré, Symfony2 did not take off .
Perhaps, for whom productivity is very important, you should immediately write a framework for HHVM?
findings
1) HHVM gives a good performance boost on some tasks - it is possible to use it to start the processes of background processing of something. Remember that HHVM is not effective when using global variables.
2) Currently, the use of existing libraries, frameworks and CMS with HHVM is difficult. HHVM is sharpened for performance and may not support various beautiful designs. However, in the future the situation will improve, because and HHVM may start supporting a better language, and developers may be puzzled by adapting their code to HHVM.
3) On Wordpress, HHVM showed an impressive performance increase - 2 times compared to mod_php.
PS test scripts , raw data (Excel)
Source: https://habr.com/ru/post/190566/
All Articles