How to prove the importance of tests for each project participant
Imagine you have fully implemented a new feature in two days. The code is written, it works and everything is cool. Your manager says that you can immediately release. "But what about the tests?" Exclaim some meticulous colleague. "Why?" - You will answer in one voice with the manager. Why do we need to write tests? How to explain their need to others? Why involve testers, analysts and other participants? In this post I will tell you how to explain the benefits of tests to any project participant, and also why it is worth automating tests. And we will support all this with serious research.
Do you need automation? Let's start the argument from the reverse.
The programmer is one of the most expensive project participants, and his work greatly influences the budget. And in general, writing tests is not a lordly affair. You can simply hire students for a penny to give them both writing these tests and testing programs. Let them poke into displays, press keys - everything will be fine.
')
Autotest development is a very tedious job. It can be entrusted to anyone. I spent 5 years learning to program in the university, and another two years in graduate school. I want to design a cool architecture, apply cool patterns and launch rockets to Mars. And tests are not a task of my level.
Why bother writing tests if my code works? Each line of it went through my inner genius. How can it even create problems in the application? This is ruled out, I see through every single baytik.
Today I wrote a feature on one team, tomorrow I’ll go to another team to make a new feature. What will be further here will be dealt with without me, even if there is any mistake.
Now go through all the points and disprove each. At the same time we will cite some more interesting facts from studies of well-known companies like Microsoft ( On the Effectiveness of Unit Test Automation at Microsoft by Laurie Williams, Gunnar Kudrjavets, Nachiappan Nagappan), and also a less well-known company from South America.
Referring to Steve McConnell's “Perfect Code”. If we estimate how the labor costs change over time, then we can see that gradually the proportion of testing performed by developers decreases from 25% to 8-10%. More resources are starting to steal other activities.
There is another study where Microsoft studied about 30 internal teams of various sizes - 28-35 developers and 14-20 testers each.
This study shows roughly the same numbers. With the introduction of automated testing practices, development time increased by an average of 26%. But development is not the whole project activity, there is also design, integration, implementation, and so on. Therefore, a quarter growth in just one of many processes is a small loss to the budget, especially in light of the fall in manual testing costs.
Perhaps writing tests is somewhat boring - the test code is trivial and rarely carries interesting tasks. But your client definitely does not need an eternal fire, which you will put out, finding defects during implementation or beta testing.
The less you believe in the reference of your code, the better it will be in the end both for the team and for the product. And first of all to you - learn from your mistakes and without unpleasant consequences.
If you write the code, and then abandon its support, then after a month or two, you can easily fly past the release. As part of the overall testing of the product, a lot of defects will be found - in the end, your feature and several other features of your colleagues will not reach production.
Go on the offensive. Let's look at the tests through the eyes of all participants in the development process: the client, the product owner, the tester and the developer.
What happens when they do not write tests? For example, you lose a satellite that is insured for 2.6 billion rubles.
On November 28, Roskosmos launched the satellite Meteor-M No. 2-1 using the Soyuz-2.1b rocket. He did not fly - slightly deviated from the course and fell into the ocean. Software developers, who were responsible for adjusting the route, forgot to specify the actual data. It was planned that the satellite would be launched into orbit from the Baikonur cosmodrome, and the launch took place several tens of thousands of kilometers from it - from the Vostochny cosmodrome. One small test to verify the input data - and 2.6 billion rubles would have been saved.
On the Internet, you can find many stories about how one small error in software can lead to billions of dollars in expenses. Most are connected with medicine and space, there are many examples in the banking industry. For example, in 2012, the same update was rolled into all the servers of one large Western bank. After that, the servers with the old code began to generate additional reports, because the necessary locks disappeared on the pre-emptive front. As a result, wasted generated reports for 440 million dollars.
On the other hand, what happens if the product code is completely closed by tests? With him you can safely do what no one did to you. And this is very cool.
An important characteristic of a product with a future is the possibility of its expansion and long-term support. In English, this is called the beautiful word Maintainability .
This means that a product that has been developing for several years (or even several decades) constantly provides some minimum level of quality, and the number of defects in it does not exceed a certain maximum. We should not do worse than user experience. We should not significantly degrade the quality of our features. This can be guaranteed through tests, manual and automated.
In another study, Microsoft, you can peep another interesting figure. As part of a team of 32 developers, the number of defects that reached the user decreased by 20.9% - after the second version of the product, development teams began to apply test automation and write unit tests. At the same time, the second version acquired additional features, increased in code volume by about 20%, and another 20% was rewritten. Yes, and testers themselves began to notice that the quality of their software has become higher - it has become more interesting to work.
IBM also conducted similar studies and eventually found out that the cost of removing the bug detected after the release is 4-5 times higher than when detected during design or implementation. Self-tests, if they are run regularly, help in time to signal that there are any problems. They help cut these extra cost in the budget. As a result, the cost of software support will be lower - and the quality of user experience will increase.
Here the question is a little different: why do we need test automation? Microsoft research comes to the rescue again. According to him, after automating testing, testers have become much harder to find bugs. But with the same time costs, they now find much more complex bugs that were more likely to leak into production. At the same time, there are much less trivial errors, since they are removed even before testers.
Here is a bug report for one internal Microsoft project:
The first version is before the introduction of automatic testing, the second is after. Errors are divided into four categories. The higher the category, the more difficult the errors. In total, in the second version, the number of bugs decreased by 21%. The number of bugs from simple categories (3 and 4) decreased significantly. The share of more complex bugs remained the same or slightly increased due to the fact that the total number of bugs decreased.
Now testers can write more complex test cases that are as close as possible to the user experience. The programmer will most likely eliminate the simplest errors himself if he systematically supplements his code with some kind of test code.
Why do testers get fewer bugs? There is a pyramid known to many that divides various types of testing into layers. This pyramid says that the most primitive unit tests that affect only a small part of the software should be the most in the project. A complex end-to-end or manual tests should be the minimum number, but at the same time they should look for the most complex bugs - these are the so-called flashing, floating or gray tests. If this pyramid is turned over, a so-called testing funnel appears, into which various bugs are poured.
Source: https://twitter.com/noahsussman
At each level of this funnel, the bugs are filtered out. The more complex and interesting the bug, the lower it falls in this funnel. A large number of unit tests eliminates most of the bugs before the manual testing phase. Therefore, the time of the manual testing itself is reduced, and only bugs related to some specific functionality can penetrate to the user. 90-95% will never face them in their lives.
For the developer, I singled out as many as five advantages that tests give him:
When writing a test, the developer begins to better understand the requirements through communication with non-technical experts. Suppose you need to implement some input field. When you write tests to your code, you start asking questions: what if I substitute an empty value here? What if I enter 150 characters instead of the 10 allowed? Clarifying these points, you make your code more stable and eliminate the most simple errors.
Tests allow us to guarantee the stability of our application when refactoring, especially when bold refactoring. Suppose you have some old module - a huge Legacy code, which many are afraid to touch, because it is not clear what is happening there. If this thing is closed with tests, they will qualitatively verify that the main cases work for us. It will not be so scary to go in and refactor it all under the hood, change it to some new implementations. We will simply confirm that the changes that appeared in our application did not break anything.
Tests serve as excellent documentation for your code. It is much easier to understand someone else's code when there is some consumer of this code. And tests are the very first consumer. Do not go to Jira, do not study lengthy comments in Confluence - you can simply open a test class, a set of some test methods for your code, and understand what this code does just by reading the names of test methods and understanding what data is being transmitted there.
If we use iterative practice of writing tests, such as TDD, or we plan in advance how we will cover some specific entity in our program with tests, this will make its interface better. We begin to think over what data we will push into this entity, what data it will return to us. This will allow you to immediately write a high-quality interface instead of constantly adding new contexts and parameters.
It is clear that tests reduce the number of errors in the code. Less mistakes - more free time. Here is what the developers themselves say about writing tests (in this case modular) in a Microsoft study:
Writing tests should be supported by the management. If you decide that you are going to write tests, while the others do not write, the initiative will most likely stall in a couple of months. Management must support writing tests and understand why they appear at all in a project.
Of course, development takes longer with tests - an average of 26%. Depending on the complexity of the programming language, the features and experience of the programmer, this proportion can be reduced to 8% or jump to 40-50% of the time. If time to market is as critical as possible for you - put off writing tests, release MVP and return to tests later.
In my opinion, not only developers, but also testers should write tests within the team. All team members should have access to the code, and all of them should be able to influence code changes. Testers can offer more exceptional situations and a variety of edge-cases than average developers. Accordingly, test quality and coverage will improve.
I hope all the above arguments will help you explain within the teams, and at the same time to yourself, why we need tests, why we should write them every day and how they make the code better.
Do you need automation? Let's start the argument from the reverse.
Why don't I need to write tests?
1. It is expensive
The programmer is one of the most expensive project participants, and his work greatly influences the budget. And in general, writing tests is not a lordly affair. You can simply hire students for a penny to give them both writing these tests and testing programs. Let them poke into displays, press keys - everything will be fine.
')
2. It is tedious
Autotest development is a very tedious job. It can be entrusted to anyone. I spent 5 years learning to program in the university, and another two years in graduate school. I want to design a cool architecture, apply cool patterns and launch rockets to Mars. And tests are not a task of my level.
3. It is useless
Why bother writing tests if my code works? Each line of it went through my inner genius. How can it even create problems in the application? This is ruled out, I see through every single baytik.
4. This is “not my problem”
Today I wrote a feature on one team, tomorrow I’ll go to another team to make a new feature. What will be further here will be dealt with without me, even if there is any mistake.
Why is this whole list wrong?
Now go through all the points and disprove each. At the same time we will cite some more interesting facts from studies of well-known companies like Microsoft ( On the Effectiveness of Unit Test Automation at Microsoft by Laurie Williams, Gunnar Kudrjavets, Nachiappan Nagappan), and also a less well-known company from South America.
1. On a project scale, it is inexpensive.
Referring to Steve McConnell's “Perfect Code”. If we estimate how the labor costs change over time, then we can see that gradually the proportion of testing performed by developers decreases from 25% to 8-10%. More resources are starting to steal other activities.
There is another study where Microsoft studied about 30 internal teams of various sizes - 28-35 developers and 14-20 testers each.
This study shows roughly the same numbers. With the introduction of automated testing practices, development time increased by an average of 26%. But development is not the whole project activity, there is also design, integration, implementation, and so on. Therefore, a quarter growth in just one of many processes is a small loss to the budget, especially in light of the fall in manual testing costs.
2. This is much more fun than production problems.
Perhaps writing tests is somewhat boring - the test code is trivial and rarely carries interesting tasks. But your client definitely does not need an eternal fire, which you will put out, finding defects during implementation or beta testing.
3. It is useful for everyone.
The less you believe in the reference of your code, the better it will be in the end both for the team and for the product. And first of all to you - learn from your mistakes and without unpleasant consequences.
4. This will be your problem.
If you write the code, and then abandon its support, then after a month or two, you can easily fly past the release. As part of the overall testing of the product, a lot of defects will be found - in the end, your feature and several other features of your colleagues will not reach production.
Testing in terms of team roles
Go on the offensive. Let's look at the tests through the eyes of all participants in the development process: the client, the product owner, the tester and the developer.
Customer
What happens when they do not write tests? For example, you lose a satellite that is insured for 2.6 billion rubles.
On November 28, Roskosmos launched the satellite Meteor-M No. 2-1 using the Soyuz-2.1b rocket. He did not fly - slightly deviated from the course and fell into the ocean. Software developers, who were responsible for adjusting the route, forgot to specify the actual data. It was planned that the satellite would be launched into orbit from the Baikonur cosmodrome, and the launch took place several tens of thousands of kilometers from it - from the Vostochny cosmodrome. One small test to verify the input data - and 2.6 billion rubles would have been saved.
On the Internet, you can find many stories about how one small error in software can lead to billions of dollars in expenses. Most are connected with medicine and space, there are many examples in the banking industry. For example, in 2012, the same update was rolled into all the servers of one large Western bank. After that, the servers with the old code began to generate additional reports, because the necessary locks disappeared on the pre-emptive front. As a result, wasted generated reports for 440 million dollars.
On the other hand, what happens if the product code is completely closed by tests? With him you can safely do what no one did to you. And this is very cool.
Product Owner
An important characteristic of a product with a future is the possibility of its expansion and long-term support. In English, this is called the beautiful word Maintainability .
This means that a product that has been developing for several years (or even several decades) constantly provides some minimum level of quality, and the number of defects in it does not exceed a certain maximum. We should not do worse than user experience. We should not significantly degrade the quality of our features. This can be guaranteed through tests, manual and automated.
In another study, Microsoft, you can peep another interesting figure. As part of a team of 32 developers, the number of defects that reached the user decreased by 20.9% - after the second version of the product, development teams began to apply test automation and write unit tests. At the same time, the second version acquired additional features, increased in code volume by about 20%, and another 20% was rewritten. Yes, and testers themselves began to notice that the quality of their software has become higher - it has become more interesting to work.
IBM also conducted similar studies and eventually found out that the cost of removing the bug detected after the release is 4-5 times higher than when detected during design or implementation. Self-tests, if they are run regularly, help in time to signal that there are any problems. They help cut these extra cost in the budget. As a result, the cost of software support will be lower - and the quality of user experience will increase.
Tester
Here the question is a little different: why do we need test automation? Microsoft research comes to the rescue again. According to him, after automating testing, testers have become much harder to find bugs. But with the same time costs, they now find much more complex bugs that were more likely to leak into production. At the same time, there are much less trivial errors, since they are removed even before testers.
Here is a bug report for one internal Microsoft project:
The first version is before the introduction of automatic testing, the second is after. Errors are divided into four categories. The higher the category, the more difficult the errors. In total, in the second version, the number of bugs decreased by 21%. The number of bugs from simple categories (3 and 4) decreased significantly. The share of more complex bugs remained the same or slightly increased due to the fact that the total number of bugs decreased.
Now testers can write more complex test cases that are as close as possible to the user experience. The programmer will most likely eliminate the simplest errors himself if he systematically supplements his code with some kind of test code.
Why do testers get fewer bugs? There is a pyramid known to many that divides various types of testing into layers. This pyramid says that the most primitive unit tests that affect only a small part of the software should be the most in the project. A complex end-to-end or manual tests should be the minimum number, but at the same time they should look for the most complex bugs - these are the so-called flashing, floating or gray tests. If this pyramid is turned over, a so-called testing funnel appears, into which various bugs are poured.
Source: https://twitter.com/noahsussman
At each level of this funnel, the bugs are filtered out. The more complex and interesting the bug, the lower it falls in this funnel. A large number of unit tests eliminates most of the bugs before the manual testing phase. Therefore, the time of the manual testing itself is reduced, and only bugs related to some specific functionality can penetrate to the user. 90-95% will never face them in their lives.
Developer
For the developer, I singled out as many as five advantages that tests give him:
- Understanding the requirements
When writing a test, the developer begins to better understand the requirements through communication with non-technical experts. Suppose you need to implement some input field. When you write tests to your code, you start asking questions: what if I substitute an empty value here? What if I enter 150 characters instead of the 10 allowed? Clarifying these points, you make your code more stable and eliminate the most simple errors.
- Stability when refactoring
Tests allow us to guarantee the stability of our application when refactoring, especially when bold refactoring. Suppose you have some old module - a huge Legacy code, which many are afraid to touch, because it is not clear what is happening there. If this thing is closed with tests, they will qualitatively verify that the main cases work for us. It will not be so scary to go in and refactor it all under the hood, change it to some new implementations. We will simply confirm that the changes that appeared in our application did not break anything.
- Formation of documentation
Tests serve as excellent documentation for your code. It is much easier to understand someone else's code when there is some consumer of this code. And tests are the very first consumer. Do not go to Jira, do not study lengthy comments in Confluence - you can simply open a test class, a set of some test methods for your code, and understand what this code does just by reading the names of test methods and understanding what data is being transmitted there.
- API enhancement
If we use iterative practice of writing tests, such as TDD, or we plan in advance how we will cover some specific entity in our program with tests, this will make its interface better. We begin to think over what data we will push into this entity, what data it will return to us. This will allow you to immediately write a high-quality interface instead of constantly adding new contexts and parameters.
- Error reduction
It is clear that tests reduce the number of errors in the code. Less mistakes - more free time. Here is what the developers themselves say about writing tests (in this case modular) in a Microsoft study:
Let's sum up
Writing tests should be supported by the management. If you decide that you are going to write tests, while the others do not write, the initiative will most likely stall in a couple of months. Management must support writing tests and understand why they appear at all in a project.
Of course, development takes longer with tests - an average of 26%. Depending on the complexity of the programming language, the features and experience of the programmer, this proportion can be reduced to 8% or jump to 40-50% of the time. If time to market is as critical as possible for you - put off writing tests, release MVP and return to tests later.
In my opinion, not only developers, but also testers should write tests within the team. All team members should have access to the code, and all of them should be able to influence code changes. Testers can offer more exceptional situations and a variety of edge-cases than average developers. Accordingly, test quality and coverage will improve.
I hope all the above arguments will help you explain within the teams, and at the same time to yourself, why we need tests, why we should write them every day and how they make the code better.
Source: https://habr.com/ru/post/354096/
All Articles