Unit testing. Chip tuning

It doesn't matter what approach is used when writing tests: TDD, BDD, or some other. Unit tests are the primary protective barrier that helps to avoid bugs. A well-described cases will help colleagues to understand what is happening in the project and not to break the wood in the code.

Let's get to the bottom:

There is a specific problem: 5k + unit tests pass in 12 minutes - this is twice the installation time of the packages and the build itself.

This is a lot.

If you estimate how much time with each assembly it takes on a day - it becomes sad!

Picking each test for problems will not change the situation much. Tests can not be thrown out, and time must be reduced.

There is a small and handy karma-sharding plugin that allows you to run multiple browsers in parallel, the distribution of test cases in them will fall on the developer’s shoulders.

The usual configuration for unit tests in an angular starter with a webpack and karma looks briefly like this:

The karma config karma.conf.js installs the files that will be processed:

files: [ { pattern: './config/spec-bundle.js', watched: false }, { pattern: './src/assets/**/*', watched: false, included: false, served: true, nocache: false } ] 

then it connects the preprocessor for the webpack.test.js webpack config

 preprocessors: { './config/spec-bundle.js': ['coverage', 'webpack', 'sourcemap'] } 

Note that the spec-bundle.js file appears twice.

Inside this file the following happens:

The first step is to install the necessary dependencies, without which our angular code will not run in the tests:

 Error.stackTraceLimit = Infinity; require('core-js/es6'); require('core-js/es7/reflect'); require('zone.js/dist/zone'); require('zone.js/dist/long-stack-trace-zone'); require('zone.js/dist/proxy'); // since zone.js 0.6.15 require('zone.js/dist/sync-test'); require('zone.js/dist/jasmine-patch'); // put here since zone.js 0.6.14 require('zone.js/dist/async-test'); require('zone.js/dist/fake-async-test'); require('rxjs/Rx'); var testing = require('@angular/core/testing'); var browser = require('@angular/platform-browser-dynamic/testing'); testing.TestBed.initTestEnvironment( browser.BrowserDynamicTestingModule, browser.platformBrowserDynamicTesting() ); 

The second part of the file is the context for the unit test files. When building a webpack, it will load all split files on this schedule in this context. These are the very same unit tests that karma launches:

 /** * Ok, this is kinda crazy. We can use the context method on * require that webpack created in order to tell webpack * what files we actually want to require or import. * Below, context will be a function/object with file names as keys. * Using that regex we are saying look in ../src then find * any file that ends with spec.ts and get its path. By passing in true * we say do this recursively */ var testContext = require.context('../src', true, /\.spec\.ts/); /** * Get all the files, for each file, call the context function * that will require the file and load it up here. Context will * loop and require those spec files here */ function requireAll(requireContext) { return requireContext.keys().map(requireContext); } /** * Requires and returns all modules that match */ var modules = requireAll(testContext); 

Since in the development mode we do not need to run all the tests at once, one module is enough, then problems with the test run time are not necessary for such a module. Everything happens fast enough.

Those 12 minutes is a production / test build. Consider this case.

To solve the problem, we will parallelize all tests for N browsers. Connecting the karma-sharding plugin does not require large manipulations:

First, adding karma to frameworks

 frameworks: [..., 'sharding'] 

Secondly, this is the addition of the config of the plugin itself.

 sharding: { specMatcher: /(spec|test)s?\.js/i, base: '/base', getSets: function(config, basePath, files) { // splitForBrowsers - some util function return splitForBrowsers(files.served) .map(oneBrowserSet => [someInitScript].concat(oneBrowserSet)); } } 

By default, the config can be not defined. But then it will work only for the basic configuration, when there are several files with tests:

 [a1.spec.js, a2.spec.js, … aN.spec.js] 

then the set for N browsers looks like this:

 [a1.spec.js], [a2.spec.js], … [aN.spec.js] 

and for N / 2, respectively:

 [a1.spec.js, a2.spec.js], [a3.spec.js, a4.spec.js], … [aN-1.spec.js, aN.spec.js] 

and then everything is simple, but not in our case with Angular and webpack. One test file consists of two parts:

 1 -   //some setup code 2 -  - //require.context('../src', true, /\.spec\.ts/); 

When testing the entire application, we need several such files with different sets of tests, however, the assembly of such files will take a lot of time, since it does not happen instantly. For example, the resulting code converted into webpack modules of such necessary dependencies approaches 100k lines.

But we will cut!

That is, we will divide each such independent file into two parts: the first is all the necessary dependencies and settings — setup.js, and the second is the test suite connected via the webpack context — testsN.js. Since setup.js is a common set of installations, the same for all sets of test cases, such a file will be one.

As a result, we should have the following set of files:

setup.js
tests1.js
tests2.js
...
testsN.js

Which we need to collect in the following sets:

 [setup.js, tests1.js], [setup.js, tests2.js], … [setup.js, testsN.js] 

Step 1

First of all, go through the whole code in search of all the necessary files with unit tests and distribute them into several files - testsN.js, depending on how many browsers you plan to use - some N. One such file, for example, the same tests1.js looks like that:

 require('/Users/guest/test-project/src/modules/accounts/accounts.spec.ts'); require('/Users/guest/test-project/src/modules/cards/cards.spec.ts'); require('/Users/guest/test-project/src/modules/users/users.spec.ts'); ... 

Distribution of cases by files of course can be implemented as you please. In our case, this is an approximately uniform distribution of the number of cases in the * .spec.ts file.

We collect all necessary files in any folder convenient for us - a certain tmp directory.
After that we set the following enrties to the webpack:

 entry: { entry = fs.readdirSync(path.join(tmp)).reduce((entries, fileName) => { entries[fileName] = path.join(tmp, fileName); return entries; }, {setup: path.join(tmp, 'setup.ts')}); }, 

It is important to setup to build as a common chunk, then we can load it before each set of tests.

 new CommonsChunkPlugin({ name: 'setup', minChunks: module => /setup.test/.test(module.resource) }) 

So after launching the webpack, we’ll get compiled setup.js and N files with test cases. This launch will be the first step of two when running tests.

Step # 2

This is the setting of karma and karma-sharding. As already presented, the settings are not many. The most interesting is the function that collects a set of test cases getSets:

 const {splitArray, isSpecFile} = require('karma-sharding/lib/utils'); … function getSets(config, basePath, files) { const setupScript = files.served.find(file => file.path.indexOf('setup') > -1); const specs = files.served .map(file => return config.base + file.path) .filter(filePath => isSpecFile(filePath, config.specMatcher) && !/(setup)/.test(filePath)); return splitArray(specs, config.browserCount).map(set => { return set.concat([config.base + setupScript.path.replace(basePath, '')]); }); } 

The specMatcher config - here we find the compiled setup.js and all the testsN.js files in some tmp directory.

And that's all - the config for karma is ready.
Further only start webpack for assembly of tests and start karma!

Well, and of course the numbers:

5k + unit tests
To: with one browser - 12 minutes
After: on 10 browsers - 3 minutes

Four times, Karl !!!