Parallelization of tests or one head is good, but two heads are better
At some point, if you try hard and long to keep the test coverage at least 80% of the code, the run of the full test suite will take longer than it takes to take a smoke break and read new articles from the habr. In turn, this leads to the fact that the full suite (suite) will be launched less and less. Hudson will begin to report broken builds, and then the effect of a broken window will work and a broken build will become the norm.
You can try to run a full run before each commit. But spending time on cinema in the form of cucumber features running across the screen, as well as exiting the stream, will reduce the effectiveness of developers at times.
In one of our projects, in which, according to the records of redmine, about 400 hours of work of our team were invested, the situation with tests before parallelization looked like this (a couple of days ago):
')
18 minutes !!!
During this time, the developer can make coffee, smoke a cigarette, go to the toilet, pinch a nice colleague for the ass and have time to watch the last 3 minutes of the “matrix” on the screen. If you require him to run a full run before each commit, then he will only do what to watch the “matrix” andpinch priests to drink coffee.
But the analysis of the processor load during the run shows that only one core is involved in the work, regardless of how many of them there are. As the proverb says, it's better to lose a day, and then fly five minutes later. Poryskav in Google, we found gem parallel_tests . Now we are not looking with such envy at the erlang group, which can safely parallelize their tests on a cluster of rented cloud machines in Selectel .
Heme parallel_tests compared with peers ( hydra , testjour ) is distinguished by the fact that it allows us to parallelize our integration tests on cucumber + capybara + selenium-webdriver, which, due to the abundant ayaxification of our application, cannot be performed without a real browser, in other words - running through gem capybara firefox. This actually runs several copies of firefox, to which the load is more or less evenly distributed. htop at the same time demonstrates the full load of all cores, no matter how many they are on the machine. And, which plays a significant role, it is very easy to adapt the application to use this gem.
First, it is necessary to ensure the existence of several test bases, one per core in a typical case. The heme itself sets the TEST_ENV_NUMBER environment variable at startup, so the database.yml can be entered in the test section.
Naturally, it is necessary to create bases before a run with your hands
Secondly, it is necessary to ensure synchronization of database schemas during additional migrations or re-posting of old ones:
Thirdly, it is necessary to start the run differently.
This is for running cucumber features. You can rspec and ordinary rail tests to drive.
Yes, gem, of course, must be registered in the Gemfile and put
As a result, these are the results of the last assembly of the Hudson on the eight-nuclear selectk kseoncheg
Yes, we have reduced the number of steps and scenarios, because besides using parallelization, the test code has undergone some refactoring. But 150 vs. 151 is not much less. But in time - 5 and a half minutes, against almost 18. This is still taking into account approximately 60-70 seconds of constant costs for a fresh pull from the repository, the remigration of databases and the launch of firefoxes.
The result is obvious, and as a result, the build in Hudson has stabilized in just one day - the last 5 commits have successfully completed a full set of tests, which I wish everyone.
UPD: If the step_definitions solution is not found in parallel mode, the solution was found by the Alder habrachelovek. Then you need to run the tests like this:
You can try to run a full run before each commit. But spending time on cinema in the form of cucumber features running across the screen, as well as exiting the stream, will reduce the effectiveness of developers at times.
In one of our projects, in which, according to the records of redmine, about 400 hours of work of our team were invested, the situation with tests before parallelization looked like this (a couple of days ago):
151 scenarios (151 passed)
3997 steps (3997 passed)
17m49.257s
')
18 minutes !!!
During this time, the developer can make coffee, smoke a cigarette, go to the toilet, pinch a nice colleague for the ass and have time to watch the last 3 minutes of the “matrix” on the screen. If you require him to run a full run before each commit, then he will only do what to watch the “matrix” and
But the analysis of the processor load during the run shows that only one core is involved in the work, regardless of how many of them there are. As the proverb says, it's better to lose a day, and then fly five minutes later. Poryskav in Google, we found gem parallel_tests . Now we are not looking with such envy at the erlang group, which can safely parallelize their tests on a cluster of rented cloud machines in Selectel .
Heme parallel_tests compared with peers ( hydra , testjour ) is distinguished by the fact that it allows us to parallelize our integration tests on cucumber + capybara + selenium-webdriver, which, due to the abundant ayaxification of our application, cannot be performed without a real browser, in other words - running through gem capybara firefox. This actually runs several copies of firefox, to which the load is more or less evenly distributed. htop at the same time demonstrates the full load of all cores, no matter how many they are on the machine. And, which plays a significant role, it is very easy to adapt the application to use this gem.
First, it is necessary to ensure the existence of several test bases, one per core in a typical case. The heme itself sets the TEST_ENV_NUMBER environment variable at startup, so the database.yml can be entered in the test section.
test:
database: xxx_test<%= ENV['TEST_ENV_NUMBER'] %>
Naturally, it is necessary to create bases before a run with your hands
rake parallel:create
Secondly, it is necessary to ensure synchronization of database schemas during additional migrations or re-posting of old ones:
rake parallel:prepare
Thirdly, it is necessary to start the run differently.
rake parallel:features
This is for running cucumber features. You can rspec and ordinary rail tests to drive.
Yes, gem, of course, must be registered in the Gemfile and put
bundle install
As a result, these are the results of the last assembly of the Hudson on the eight-nuclear selectk kseoncheg
Results:
17 scenarios (17 passed)
347 steps (347 passed)
31 scenarios (31 passed)
780 steps (780 passed)
8 scenarios (8 passed)
149 steps (149 passed)
26 scenarios (26 passed)
1363 steps (1363 passed)
26 scenarios (26 passed)
463 steps (463 passed)
17 scenarios (17 passed)
331 steps (331 passed)
6 scenarios (6 passed)
88 steps (88 passed)
19 scenarios (19 passed)
410 steps (410 passed)
Took 338.989782947 seconds
Finished: SUCCESS
Yes, we have reduced the number of steps and scenarios, because besides using parallelization, the test code has undergone some refactoring. But 150 vs. 151 is not much less. But in time - 5 and a half minutes, against almost 18. This is still taking into account approximately 60-70 seconds of constant costs for a fresh pull from the repository, the remigration of databases and the launch of firefoxes.
The result is obvious, and as a result, the build in Hudson has stabilized in just one day - the last 5 commits have successfully completed a full set of tests, which I wish everyone.
UPD: If the step_definitions solution is not found in parallel mode, the solution was found by the Alder habrachelovek. Then you need to run the tests like this:
parallel:features[2,'','--require features']
Source: https://habr.com/ru/post/123994/
All Articles