Misconceptions about automated testing

Hello, my name is Dmitry Karlovsky and this is a continuation of the traditional heading "Why do we not like to write tests?". The short answer is: because the bonuses received from them do not outweigh the effort. If so, then we are doing something wrong. Let's see what could have gone wrong ..

This article grew out of the "Misconceptions" section of Longrid, "The Concept of Automatic Testing," through the addition of new errors and arguments.

Unit tests faster component

Yes, moki are usually executed faster than real code. However, they hide some kinds of errors, which is why they have to write more tests. If the framework does not know how to be lazy and does a lot of unnecessary work to raise the component tree (for example, web-components with nails nailed to the DOM or TestBed in Angular that creates everything in the world during initialization), then the tests slow down significantly, but not so fatally. If the framework does not render until it is asked for it and does not create components until they are needed (such as $ mol_view ), component tests pass no slower than modular ones.

With component tests it is difficult to locate the error

Yes, if they are executed in a random order, then an error in logic can drop a bunch of tests from which it may not be clear where to start digging. This, unfortunately, is a common anti-pattern - to find all files with a given extension and execute them in a random order, saying that the tests do not depend on each other. And this is true for unit tests.

However, it makes sense to perform component tests in order from less dependent components to more dependent ones. Then the first fallen test will show the source of the problem. The rest of the tests can usually not be performed anymore, which is great saving time for passing the tests. Again, in the MAM architecture, all the code (which is production, that test) is serialized in a single order. This ensures that dependency tests are performed before the dependent tests, which means that they can safely rely on the dependency to work correctly. If you use other tools, consider how you can use them to build tests in the correct order.

Templates do not need to test

It is necessary to test the logic. A rare template engine ( mustache , view.tree ) prohibits embedding logic into templates, which means they should also be tested. Often, unit tests are not suitable for this ( enzyme as a rare exception), so you still have to resort to component ones.

Tests must follow the Given / When / Then pattern.

Yes, sometimes in the test scenario, you can highlight these steps, but do not suck them out of your finger when they are not. Often, the script has a simpler (for example, only Then block) or complex (Given / Check / When / Then) structure. A few examples:

Pure functions often have only the Then block:

console.assert( Math.pow( 2 , 3 ) === 8 ) // Then 

No less often, the action (When) consists precisely in preparing the state (Given):

 component.setState({ name : 'Jin' }) // Given/When console.assert( component.greeting === 'Hello, Jin!' ) // Then 

And it happens that verification is not needed, because the very fact of successful code execution is sufficient:

 ensurePerson({ name : 'Jin' , age : 33 }) 

Similar code is completely meaningless:

 const component = new MyComponent // Given expect( component ).toBeTruthy() // Then 

Just like a test that never fell, it doesn't test anything. So an assort who never threw an exception does not check anything.

In the right test there should be only one assert.

It is not uncommon for us to check that we have correctly prepared the state with a mid-point calibration:

 wizard.nextStep().nextStep() // Given console.assert( wizard.passport.isVisible === false ) // Check wizard.toggleRegistration() // When console.assert( wizard.passport.isVisible === true ) // Then 

It is impossible to break this test into the following two, since the second implicitly relies on the state created by the first:

 wizard.nextStep().nextStep() // When console.assert( wizard.passport.isVisible === false ) // Then 

 wizard.nextStep().nextStep() // Given wizard.toggleRegistration() // When console.assert( wizard.passport.isVisible === true ) // Then 

Imagine that the requirements have changed and now by default we show the registration form:

 wizard.nextStep().nextStep() // When console.assert( wizard.passport.isVisible === true ) // Then 

Now, if toggleRegistration implemented in such a way that, for example, it uses its state to speed up work, then it will pass the second test, still returning true and it turns out that the first application of toggleRegistration will not change anything in the form:

 isPassportVisible = false toggleRegistration() { this.passport.isVisible = this.isPassportVisible = !this.isPassportVisible } 

In the variant with the additional verification of the default state, we would have caught the dropped test in this case. Moreover, one should not be afraid to write longer scripts if the next step is based on the state of the previous one.

 wizard.nextStep().nextStep() // When console.assert( wizard.passport.isVisible === false ) // Then wizard.toggleRegistration() // When console.assert( wizard.passport.isVisible === true ) // Then wizard.toggleRegistration() // When console.assert( wizard.passport.isVisible === false ) // Then 

Usually, the argument against this approach is the difficulty of understanding which of the asserts has fallen. But wait, no one forces you to use such a testing tool that does not provide comprehensive information about the location of the test crash. A good tool (for example, $ mol_test ) will even helpfully stop the debugger in this place, allowing you to immediately begin to investigate the problem.

To summarize, we can recommend writing tests not using the "Given / When / Then" pattern, but as a small adventure, starting from absolute emptiness and through a certain number of actions, going through a number of states, which we check.