Do not be fooled by your customers.
This is my first post on Habré, so do not judge strictly. I do a lot of work not only in development, but also in setting processes, including testing. And always somewhat skeptical about manual testing, more precisely, the part of it that is responsible for “ensuring the operability of the existing functionality” (in the common people, regression testing). What is wrong with this testing and why many companies use it then? Who is interested in the answer to these questions, can spend another couple of minutes for further reading.
The problem is simple enough, but its essence does not lie on the surface. Imagine yourself in the place of the customer. You come with a set of requirements for the executor, in this case we will consider the development team as an executor. You have agreed on all the details with the team and they are ready to start working on your product. You, as an intelligent and experienced customer, correctly set priorities and give the team a requirement for a demand in the right order for the business. It seems that while everything sounds good and the situation does not resemble a problematic one.
But here passes one or two months and the unpleasant detail becomes clear - more and more time is spent for testing. It is quite logical - after all, the finished functionality in the product becomes more and more and you need to constantly monitor that it still works. This effect is called the “regression spiral of death” (the term is peeped in the speech of Max Dorofeev “Monkey vs. Robots” ). This spiral develops over time and becomes wider and wider. And if earlier testers had time to "run through" the product in a few hours, then soon it starts to take a few days.
Everything is very simple - by giving the customer “finished” work, we don’t give him a way for free and it is easy to monitor its performance in the future. Exaggerating, it may sound like this: “We finished working on X, Y and Z. But after a couple of weeks there are chances that they will stop working normally ...” But how is that? After all, the customer paid for the completion of the work. How should he be now? He is offered a very simple solution - pay us constantly an additional fee for the fact that we will monitor the performance during the life of the product.
')
It turns out that the cost of one specific functionality will continue to grow throughout the project . And the faster the team works, the faster the cost of maintaining the regression will grow. This is not fair! In life, we would never agree to this. Imagine, you are ordering repairs, and you are offered to pay extra money to ensure that the living room does not fall apart while working on the kitchen ...
Why, for all the flaws of such a model, it will not cease to be applied? There are several reasons:
At the same time, in many modern grocery companies, such costs are skeptical and do not support the idea of ​​manual regression testing. In such companies, there is no monkey position of a tester, which is so familiar to the outsourcing market.
In order to work honestly, we need to revise the “readiness criteria” of the development team and expand them with the availability of automated acceptance tests. Before working on a specific functionality, a good team (note that I did not use the word Agile) asks the customer questions about how the functionality should work and how the customer will check readiness. This is the process of formation of acceptance criteria . They represent a mini-contract between the customer and the team to implement this functionality.
Further, a good team provides these acceptance criteria with concrete examples, data and “bolts” to the working product. Thus, the possibility is added with the help of acceptance tests at any time to check whether this or that functionality in the product works after any changes. Anyone can run these automated acceptance tests, usually they are added to the Continuous Integration server and run on each change or in manual mode.
And now no one spends time on manual regression testing. This time is spent on research testing, on helping the development team to complete its work on time and with a high level of quality, on ensuring the operability of the channel for communicating requirements from the customer to the development team. At the same time, the customer has on hand a powerful tool to control the quality and performance of his product, while realizing that he did not pay for it in vain.
Someone may notice that it will take time to write and support this kind of automated tests. The cost of writing is of course included in the evaluation of the implementation of work on the development. But the cost of support will depend directly on the level of the team. Well-written tests will be based on the right tools and frameworks that abstract test logic from the details of your product and make it possible to change something only with major changes in the product.
Think about whether to continue to deceive your customer. Or is it better to work so that you can be proud of the results of your work, while removing the maximum of boring manual work and leaving only an interesting creative component?
UPD: By coincidence, on the same day, two interesting articles about the team writing of acceptance tests and the evolution of the need for such tests to run in the cloud caught my eye. Both articles are in English.
The root of the problem
The problem is simple enough, but its essence does not lie on the surface. Imagine yourself in the place of the customer. You come with a set of requirements for the executor, in this case we will consider the development team as an executor. You have agreed on all the details with the team and they are ready to start working on your product. You, as an intelligent and experienced customer, correctly set priorities and give the team a requirement for a demand in the right order for the business. It seems that while everything sounds good and the situation does not resemble a problematic one.
But here passes one or two months and the unpleasant detail becomes clear - more and more time is spent for testing. It is quite logical - after all, the finished functionality in the product becomes more and more and you need to constantly monitor that it still works. This effect is called the “regression spiral of death” (the term is peeped in the speech of Max Dorofeev “Monkey vs. Robots” ). This spiral develops over time and becomes wider and wider. And if earlier testers had time to "run through" the product in a few hours, then soon it starts to take a few days.
Well, where is the deception?
Everything is very simple - by giving the customer “finished” work, we don’t give him a way for free and it is easy to monitor its performance in the future. Exaggerating, it may sound like this: “We finished working on X, Y and Z. But after a couple of weeks there are chances that they will stop working normally ...” But how is that? After all, the customer paid for the completion of the work. How should he be now? He is offered a very simple solution - pay us constantly an additional fee for the fact that we will monitor the performance during the life of the product.
')
It turns out that the cost of one specific functionality will continue to grow throughout the project . And the faster the team works, the faster the cost of maintaining the regression will grow. This is not fair! In life, we would never agree to this. Imagine, you are ordering repairs, and you are offered to pay extra money to ensure that the living room does not fall apart while working on the kitchen ...
Who needs all this?
Why, for all the flaws of such a model, it will not cease to be applied? There are several reasons:
- This model is beneficial to outsourcing companies. Indeed, in this case, the customer “falls into slavery” and constantly pays even for what has been paid for a long time and should work. And as soon as the regression spiral enters a new large round, you can hire more testers, and the test lead, test manager, rushed to them ...
- Many customers are told that “this is a normal approach in IT” and that’s how it all works. Many of them do not even know about deception and therefore do not try to fight it.
- So historically, the company and it makes all the projects "on the thumb." No one is trying to analyze options for improving the development process, and even more so to bring them to life.
At the same time, in many modern grocery companies, such costs are skeptical and do not support the idea of ​​manual regression testing. In such companies, there is no monkey position of a tester, which is so familiar to the outsourcing market.
Solution to the problem
In order to work honestly, we need to revise the “readiness criteria” of the development team and expand them with the availability of automated acceptance tests. Before working on a specific functionality, a good team (note that I did not use the word Agile) asks the customer questions about how the functionality should work and how the customer will check readiness. This is the process of formation of acceptance criteria . They represent a mini-contract between the customer and the team to implement this functionality.
Further, a good team provides these acceptance criteria with concrete examples, data and “bolts” to the working product. Thus, the possibility is added with the help of acceptance tests at any time to check whether this or that functionality in the product works after any changes. Anyone can run these automated acceptance tests, usually they are added to the Continuous Integration server and run on each change or in manual mode.
And now no one spends time on manual regression testing. This time is spent on research testing, on helping the development team to complete its work on time and with a high level of quality, on ensuring the operability of the channel for communicating requirements from the customer to the development team. At the same time, the customer has on hand a powerful tool to control the quality and performance of his product, while realizing that he did not pay for it in vain.
Someone may notice that it will take time to write and support this kind of automated tests. The cost of writing is of course included in the evaluation of the implementation of work on the development. But the cost of support will depend directly on the level of the team. Well-written tests will be based on the right tools and frameworks that abstract test logic from the details of your product and make it possible to change something only with major changes in the product.
Conclusion
Think about whether to continue to deceive your customer. Or is it better to work so that you can be proud of the results of your work, while removing the maximum of boring manual work and leaving only an interesting creative component?
UPD: By coincidence, on the same day, two interesting articles about the team writing of acceptance tests and the evolution of the need for such tests to run in the cloud caught my eye. Both articles are in English.
Source: https://habr.com/ru/post/168391/
All Articles