Transition to PHP 5.5 and unit tests
4 years have passed since the transition from PHP 4.4 to PHP 5.3 to Badoo, it’s time to update PHP, this time right away to PHP 5.5. In addition to new features, the new version of PHP once again brought us a significant increase in performance, so we had many reasons for the upgrade. In this article, we will talk about how we switched to PHP 5.5, what “rakes” we collected, and why we once again rewrote our system to run unit tests based on PHPUnit.
Figure 1. General Architecture
Last time, we switched from the fourth version of PHP to the fifth, and our version of PHP 5.3 contained patches to
')
The main problems that we have:
After switching to PHP 5.5, our unit tests began to take place much longer (several times), so we decided to once again refine our “launch-board” to solve this problem.
With the help of the xhprof extension, we quickly found out that our tests are “slowing down” due to the fact that the performance of the runkit extension has dropped significantly, so we began to look for the cause. As a result, it turned out that the problem seems to be in adding a “mystical” runtime cache for PHP 5.4, which needs to be reset every time after calling the “runkit _ * _ redefine” function.
Each runkit extension traverses all registered classes, methods, and functions and flushes this cache. We naively tried to disable it, but after that PHP began to fall, so we had to find another solution.
Before switching to PHP 5.5, we already had unit test launches as an add-on for phpunit, which divided one large suite of unit tests into several smaller ones: at that time we used test launches in 11 streams (Ilya Kudinov already talked about this on conferences, including at Badoo LoveQA: www.youtube.com/watch?v=gAisPsfbLkg ).
We spent a few simple benchmarks and found out that tests pass several times faster if we divide our suite not into 11 parts, as it was before, but 128 or more (with a fixed number of processor cores). In each suite, there were only about 10-15 files, so we called this concept "microgroups". We have about 150 such microwells, and each of them was suspiciously well suited to be a “task” for some kind of script (the task consists of a list of files for the corresponding suite, and it, in turn, runs phpunit with the appropriate parameters ).
It so happened that the author of this article has no relation to QA, but was one of the main developers of the “new scripting framework”, which, in fact, is a “cloud” for scripts and supports the concept of tasks (about our cloud, we also told at conferences and be sure to tell you more about it in Habré). And since we have tasks in the form of file lists for each phpunit suite, it means that they can also be “thrust into the cloud”, which we decided to do. The idea is very simple: since we have many small tasks, they can be run on several servers independently of each other, which should further speed up the passing of tests.
We run tests from several different sources:
All these types of test runs combine what you need to first “fetch” (fetch) a branch from some source, and then start running the tests on this branch.
This fact determined the architecture of our new "cloud" test (Fig. 1, at the beginning of the article):
First, one task is created for the master process, which:
Figure 2. Available directories in MySQL
After that, the master process analyzes the original phpunit suite to run the tests and divides it into the required number of parts (no more than 10 files per microsuite). The resulting tasks (thread-processes) are added as tasks to the cloud and begin execution on servers available for execution.
The first task, which gets to the new server, prepares the selected directory for the test run, taking the desired commit from the server on which the master process is running. In order for all other tasks that fell on the corresponding server not to do the same at the same time as the first task, file locks are used (Fig. 3).
At the same time, there can be several runs of tests for more complete utilization of the resources of our cluster: tests pass quickly, and the preparation of source texts takes up much of the time rather than directly executing code.
Fig. 3. Locks during directory preparation
Some tests can go much longer than the others, and we have statistics on the passing times for each test, so we use this information to run the “long” tests first. This strategy allows you to achieve a more even load on the servers during the execution of tests, as well as to reduce the overall time for their execution (Fig. 4).
Figure 4. Accounting test execution time
“In good weather, our entire suite of 28,000 unit tests passes in 1 minute, so tests that last longer become a bottleneck, and our system posts the authors of the corresponding tests to the board of shame, which is shown at each test run. In addition, if there are few tests left, a list of those who stayed is shown (Fig. 5).
Figure 5. Board of shame: a list of tests that are performed for more than a minute
In itself, the launch of unit tests was the first script that was transferred to the cloud. It helped eliminate a lot of bugs and flaws in the cloud, at the same time significantly speeding up the passage of unit tests.
After switching to PHP 5.5, we were able to use new language features, significantly reduced CPU consumption on our servers (on average by 25%), and also transferred our unit tests to the cloud. The latter allowed us to reduce the total time for passing the tests from 5-6 minutes (for PHP 5.5 - tens of minutes) to one minute, at the same time moving the load from the common dev-server to the cloud.
Yuri youROCK , developer of badoo
Figure 1. General Architecture
"Rakes" in the transition from PHP 5.3 to PHP 5.5
Last time, we switched from the fourth version of PHP to the fifth, and our version of PHP 5.3 contained patches to
$a = &new ClassName(); “old” PHP syntax, for example, $a = &new ClassName(); , and so that our codebase can work on PHP4 and PHP5 at the same time. This time we did not have such restrictions, so during the transition we simply found and replaced all outdated constructions with more relevant ones, and the rewriting of the code was completed with this.')
The main problems that we have:
- part of the deprecated-language feature has been removed;
- the mysql extension has become deprecated ;
- poor performance of the runkit extension, which we use when writing unit tests.
After switching to PHP 5.5, our unit tests began to take place much longer (several times), so we decided to once again refine our “launch-board” to solve this problem.
Runkit and PHP 5.4+
With the help of the xhprof extension, we quickly found out that our tests are “slowing down” due to the fact that the performance of the runkit extension has dropped significantly, so we began to look for the cause. As a result, it turned out that the problem seems to be in adding a “mystical” runtime cache for PHP 5.4, which needs to be reset every time after calling the “runkit _ * _ redefine” function.
Each runkit extension traverses all registered classes, methods, and functions and flushes this cache. We naively tried to disable it, but after that PHP began to fall, so we had to find another solution.
The concept of "microsites" (microsuite)
Before switching to PHP 5.5, we already had unit test launches as an add-on for phpunit, which divided one large suite of unit tests into several smaller ones: at that time we used test launches in 11 streams (Ilya Kudinov already talked about this on conferences, including at Badoo LoveQA: www.youtube.com/watch?v=gAisPsfbLkg ).
We spent a few simple benchmarks and found out that tests pass several times faster if we divide our suite not into 11 parts, as it was before, but 128 or more (with a fixed number of processor cores). In each suite, there were only about 10-15 files, so we called this concept "microgroups". We have about 150 such microwells, and each of them was suspiciously well suited to be a “task” for some kind of script (the task consists of a list of files for the corresponding suite, and it, in turn, runs phpunit with the appropriate parameters ).
Tests "in the cloud"
It so happened that the author of this article has no relation to QA, but was one of the main developers of the “new scripting framework”, which, in fact, is a “cloud” for scripts and supports the concept of tasks (about our cloud, we also told at conferences and be sure to tell you more about it in Habré). And since we have tasks in the form of file lists for each phpunit suite, it means that they can also be “thrust into the cloud”, which we decided to do. The idea is very simple: since we have many small tasks, they can be run on several servers independently of each other, which should further speed up the passing of tests.
Common architecture
We run tests from several different sources:
- automatic test runs with AIDA :
- on the branch (git branch) of the task;
- by "build" - the code that will go to production;
- on the master branch; - “Manual” test runs initiated by developers or testers from a dev server.
All these types of test runs combine what you need to first “fetch” (fetch) a branch from some source, and then start running the tests on this branch.
This fact determined the architecture of our new "cloud" test (Fig. 1, at the beginning of the article):
First, one task is created for the master process, which:
- selects the available directory in the database (Fig. 2);
- downloads the git branch from the right place (shared repository or dev server);
- (optional) performs git merge master;
- (optional) creates a new commit with all local changes.
Figure 2. Available directories in MySQL
After that, the master process analyzes the original phpunit suite to run the tests and divides it into the required number of parts (no more than 10 files per microsuite). The resulting tasks (thread-processes) are added as tasks to the cloud and begin execution on servers available for execution.
The first task, which gets to the new server, prepares the selected directory for the test run, taking the desired commit from the server on which the master process is running. In order for all other tasks that fell on the corresponding server not to do the same at the same time as the first task, file locks are used (Fig. 3).
At the same time, there can be several runs of tests for more complete utilization of the resources of our cluster: tests pass quickly, and the preparation of source texts takes up much of the time rather than directly executing code.
Fig. 3. Locks during directory preparation
Some tests can go much longer than the others, and we have statistics on the passing times for each test, so we use this information to run the “long” tests first. This strategy allows you to achieve a more even load on the servers during the execution of tests, as well as to reduce the overall time for their execution (Fig. 4).
Figure 4. Accounting test execution time
“In good weather, our entire suite of 28,000 unit tests passes in 1 minute, so tests that last longer become a bottleneck, and our system posts the authors of the corresponding tests to the board of shame, which is shown at each test run. In addition, if there are few tests left, a list of those who stayed is shown (Fig. 5).
Figure 5. Board of shame: a list of tests that are performed for more than a minute
In itself, the launch of unit tests was the first script that was transferred to the cloud. It helped eliminate a lot of bugs and flaws in the cloud, at the same time significantly speeding up the passage of unit tests.
results
After switching to PHP 5.5, we were able to use new language features, significantly reduced CPU consumption on our servers (on average by 25%), and also transferred our unit tests to the cloud. The latter allowed us to reduce the total time for passing the tests from 5-6 minutes (for PHP 5.5 - tens of minutes) to one minute, at the same time moving the load from the common dev-server to the cloud.
Yuri youROCK , developer of badoo
Source: https://habr.com/ru/post/220211/
All Articles