How to tame autotests
Dodo said:
- The correctness of the form is irrelevant! And then he placed everyone in a circle without any order. Nobody gave the command - everyone ran when they wanted.
L. Carroll, Alice's Adventures in Wonderland
Developing automation testing, you can find a lot of places to make efforts. Spraying efforts and pursuing false goals, we will not only waste time and resources, but also harm the development.
')
If you know at what level of development there is a project testing automation now and where to invest in such a situation, you can not only achieve greater returns, but also improve the development as a whole. The basic principles of investing resources can be tried to formulate in the form of a short manifesto.
Perhaps the most common and widely accepted manifesto in the field of test automation was created in 2003 by Gerard Mesaroche and his colleagues at ClearStream Consulting.
In their application, they declare 12 requirements that each individual test must meet. For example, one of these requirements is brevity. The test should be as simple as possible, but not simpler.
Subsequently, the described ideas were developed in the book “xUnit testing templates. Refactoring test code xUnit Test Patterns " . This is a valuable guide for developers, where Mészáros examines in detail the many problems associated with unit tests and offers programming patterns to solve them.
However, understanding how to develop individual tests is not enough for the successful development of automation. Resources are always limited. Therefore, a key success factor will also be an understanding of what qualities need to be developed in the first place.
The basic qualities of tests are trust and support . They need to be ensured first.
Trust is the complete confidence of all those working with automated tests in the results of their launch. If green tests do not guarantee that the test has worked - they will not be trusted by manual testers. If the red test does not indicate a problem, programmers will not believe. If the tests are randomly red, then green, they will all be ignored equally.
Supportability - the ability of developers to work with the application code, without putting an unacceptable amount of effort into supporting existing tests in working condition.
Imbalance in the amount of work to support the tests and the amount of changes in the code will lead to a development collapse. Imagine that making changes to the code in one case requires changing one test class, and in the other case hundreds of test classes, or just for each modified line of code you need to correct one hundred lines in the tests. Then the cost of development will soar to the skies, and developers will strive to make the minimum changes to the code. As a result, the product code will go into a freeze and its development will stop.
A fallen test that did not reveal the essence of the identified problem, “expected true was false”, will require in-depth examination of the test code to solve it. Such efforts will be constantly wasted, taking time away from really important tasks.
Ensuring the credibility and sustainability of tests is a technically difficult and time-consuming task, but its solution has a small visible effect. Many take the presence of these qualities for granted. However, if tests are unsupported or not trusted, any other automation initiatives are doomed to fail.
Having achieved success in compliance with sanitary standards, you can proceed to maximizing the benefits that tests can bring to the development. To do this, you need to maximize, but without compromising stability, increase the speed of test run and ensure their extensibility - the ability to create new tests.
Speed is the total time required to run the tests, which determines their applicability at a qualitative level.
Detection of errors at the earliest possible stage is a principle that allows you to significantly increase development productivity, since it is always easier to fix problems at the moment they occur. And fast tests allow to provide it.
Quick tests draw the attention of the developer to an error while he is still in the game . If tests are performed for a long time, the developer will know about the error after switching to other tasks. This is too late, as it will take extra effort to get back to the problem. Waste of time. Annoying. Demotivates.
Extensibility - the ability of developers to create new tests quickly and efficiently.
If the creation of each test turns into the invention of bicycles in the spirit of “how to change the behavior of the method”, “how to create a test user”, “how to make a screenshot of the page”, etc., one can hardly expect that the code coverage of the tests will increase. Rather, it will crawl.
Coverage of all new code with tests is the main way to get the most out of writing tests, since it is the new code that is most susceptible to errors and refinements. However, it will be very difficult to embody this principle without solving the problem of extensibility of tests.
Ensuring the speed of test execution and its extensibility is a way to get tangible benefits from automation and make test writing consciously attractive to developers.
From the point of view of manual testing, at this stage the work is reduced indirectly. The problem found by autotests will be fixed by the developer immediately, and will not require the participation of manual testing. This is good, but this is not all that tests can give.
Working with autotests and simultaneously maintaining manual testing, I want to combine these processes naturally, so that each of them receives the maximum benefit from the other. And for this you need to integrate the processes of automation and manual testing and ensure that the product is covered with tests.
Coverage - automatic verification, at a minimum, of all manual cases throughout the product, or by its individual modules.
Implementing a full test coverage in a natural way requires writing new tests. If the problem of the extensibility of tests is not solved, such a process will prove to be unnecessarily expensive, and the tests obtained as a result will be difficult to maintain. Expanding the set of tests will invariably lead to an increase in time for their execution. Here speed is important, slow tests significantly lose in price, even if there are a lot of them.
Integration with manual testing - replacement of a part of manual testing work with automated tests.
Ideally, you should strive to ensure that “what has not changed” is never manually tested again. Of course, this is most difficult to achieve.
The idea of ​​integrating tests into the manual testing process also does not make sense if the extensibility of tests is not ensured, since the automation of manual scripts will not be possible to the extent necessary. The speed of the tests is just as important. Running the automatic tests too slow will make it easier for the tester to check everything by hand rather than wait for the automation to run.
Providing coverage and integration with manual testing is what gives the concept of automated testing a holistic and complete look.
In the long run, automation of testing is necessary, and resources for its maintenance are inevitable. However, if you achieve a result at a qualitative level, you can at least start reducing the costs of manual testing. This will itself lead to a shortened development cycle.
Since we are forced to write tests, it is necessary to ensure that these efforts bring maximum benefit to all, and the work moves faster.
Dmitry Mamonov
Development department,
Merge's division into master,
Department of work with git,
Leading operator bash console 2 discharge
- The correctness of the form is irrelevant! And then he placed everyone in a circle without any order. Nobody gave the command - everyone ran when they wanted.
L. Carroll, Alice's Adventures in Wonderland
Developing automation testing, you can find a lot of places to make efforts. Spraying efforts and pursuing false goals, we will not only waste time and resources, but also harm the development.
')
If you know at what level of development there is a project testing automation now and where to invest in such a situation, you can not only achieve greater returns, but also improve the development as a whole. The basic principles of investing resources can be tried to formulate in the form of a short manifesto.
Test automation manifest
Trust in tests and their support is the most important thing .
The speed and extensibility of tests is important only if there is trust and support .
Covering the entire product with tests and integrating automation with manual testing makes sense only when confidence, sustainability, speed and extensibility of tests are achieved.
Criticism
Perhaps the most common and widely accepted manifesto in the field of test automation was created in 2003 by Gerard Mesaroche and his colleagues at ClearStream Consulting.
In their application, they declare 12 requirements that each individual test must meet. For example, one of these requirements is brevity. The test should be as simple as possible, but not simpler.
Subsequently, the described ideas were developed in the book “xUnit testing templates. Refactoring test code xUnit Test Patterns " . This is a valuable guide for developers, where Mészáros examines in detail the many problems associated with unit tests and offers programming patterns to solve them.
However, understanding how to develop individual tests is not enough for the successful development of automation. Resources are always limited. Therefore, a key success factor will also be an understanding of what qualities need to be developed in the first place.
Compliance with sanitary standards
The basic qualities of tests are trust and support . They need to be ensured first.
Trust is the complete confidence of all those working with automated tests in the results of their launch. If green tests do not guarantee that the test has worked - they will not be trusted by manual testers. If the red test does not indicate a problem, programmers will not believe. If the tests are randomly red, then green, they will all be ignored equally.
Supportability - the ability of developers to work with the application code, without putting an unacceptable amount of effort into supporting existing tests in working condition.
Imbalance in the amount of work to support the tests and the amount of changes in the code will lead to a development collapse. Imagine that making changes to the code in one case requires changing one test class, and in the other case hundreds of test classes, or just for each modified line of code you need to correct one hundred lines in the tests. Then the cost of development will soar to the skies, and developers will strive to make the minimum changes to the code. As a result, the product code will go into a freeze and its development will stop.
A fallen test that did not reveal the essence of the identified problem, “expected true was false”, will require in-depth examination of the test code to solve it. Such efforts will be constantly wasted, taking time away from really important tasks.
Ensuring the credibility and sustainability of tests is a technically difficult and time-consuming task, but its solution has a small visible effect. Many take the presence of these qualities for granted. However, if tests are unsupported or not trusted, any other automation initiatives are doomed to fail.
Maximize Developer Benefits
Having achieved success in compliance with sanitary standards, you can proceed to maximizing the benefits that tests can bring to the development. To do this, you need to maximize, but without compromising stability, increase the speed of test run and ensure their extensibility - the ability to create new tests.
Speed is the total time required to run the tests, which determines their applicability at a qualitative level.
Detection of errors at the earliest possible stage is a principle that allows you to significantly increase development productivity, since it is always easier to fix problems at the moment they occur. And fast tests allow to provide it.
Quick tests draw the attention of the developer to an error while he is still in the game . If tests are performed for a long time, the developer will know about the error after switching to other tasks. This is too late, as it will take extra effort to get back to the problem. Waste of time. Annoying. Demotivates.
Extensibility - the ability of developers to create new tests quickly and efficiently.
If the creation of each test turns into the invention of bicycles in the spirit of “how to change the behavior of the method”, “how to create a test user”, “how to make a screenshot of the page”, etc., one can hardly expect that the code coverage of the tests will increase. Rather, it will crawl.
Coverage of all new code with tests is the main way to get the most out of writing tests, since it is the new code that is most susceptible to errors and refinements. However, it will be very difficult to embody this principle without solving the problem of extensibility of tests.
Ensuring the speed of test execution and its extensibility is a way to get tangible benefits from automation and make test writing consciously attractive to developers.
From the point of view of manual testing, at this stage the work is reduced indirectly. The problem found by autotests will be fixed by the developer immediately, and will not require the participation of manual testing. This is good, but this is not all that tests can give.
Reduced development and manual testing cycle
Working with autotests and simultaneously maintaining manual testing, I want to combine these processes naturally, so that each of them receives the maximum benefit from the other. And for this you need to integrate the processes of automation and manual testing and ensure that the product is covered with tests.
Coverage - automatic verification, at a minimum, of all manual cases throughout the product, or by its individual modules.
Implementing a full test coverage in a natural way requires writing new tests. If the problem of the extensibility of tests is not solved, such a process will prove to be unnecessarily expensive, and the tests obtained as a result will be difficult to maintain. Expanding the set of tests will invariably lead to an increase in time for their execution. Here speed is important, slow tests significantly lose in price, even if there are a lot of them.
Integration with manual testing - replacement of a part of manual testing work with automated tests.
Ideally, you should strive to ensure that “what has not changed” is never manually tested again. Of course, this is most difficult to achieve.
The idea of ​​integrating tests into the manual testing process also does not make sense if the extensibility of tests is not ensured, since the automation of manual scripts will not be possible to the extent necessary. The speed of the tests is just as important. Running the automatic tests too slow will make it easier for the tester to check everything by hand rather than wait for the automation to run.
Providing coverage and integration with manual testing is what gives the concept of automated testing a holistic and complete look.
In the long run, automation of testing is necessary, and resources for its maintenance are inevitable. However, if you achieve a result at a qualitative level, you can at least start reducing the costs of manual testing. This will itself lead to a shortened development cycle.
Since we are forced to write tests, it is necessary to ensure that these efforts bring maximum benefit to all, and the work moves faster.
Dmitry Mamonov
Development department,
Merge's division into master,
Department of work with git,
Leading operator bash console 2 discharge
Source: https://habr.com/ru/post/322218/
All Articles