What a project manager should know about testing
I have been testing for 8 years. Manual and automated, in the role of a tester and test manager, as an employee of a company and as a representative of an outsourcer. And on almost all projects I encounter the same problem: project managers do not understand why they need testing.
If you ask an average RM a simple question: “Why is there testing on this project?”, Then the answer most often is “You are a test manager, and you should answer this question.”
But when you come to the hairdresser you do not tell the master “you yourself know what I need”? And in the grocery store you do not ask the seller to throw in your basket what you need? You can consult, you can learn “how can you?”, Ask for options, but the decision is always yours. What is the difference between testing? Maybe because too few project managers understand why they need it?
')
In this article I will try to play the role of a seller who shows the client: “But what happens at all?” Many things will be described, perhaps in too much detail, too simply ... Do not be angry, I just really want to be understood :)
Oops ... If we proceed from this approach, then in all companies with testing everything is in order. Bugs are, fix, checked ... And what, testing is always useful? Always reveals its full potential?
Something tells me that no. Then what is the difference between those rare cases when “testing is great” and the overwhelming majority?
I will try to give another illustration. You never went for a massage in your life. You go to the salon (and what, everyone goes, why don't you go?) And order a half-hour session. You have a fixed budget and there are requirements: “Massage me.” A lady of indistinct appearance vaguely fiddling with your quite distinct shoulders, no buzz, you leave the salon and think “oh, massage is a complete garbage”. But after all your requirement “massage me” was fulfilled! Maybe the matter is incorrect requirements? Your back hurt and you wanted the pain to go away? Or not enough stretch marks? Or would you like to relax? The more accurate the requirements, the greater the chance for the contractor to satisfy them. But if you cannot formulate what you want, then the performer will not be able to do what is needed, and you will not be able to objectively evaluate the result.
Oops ... A simple example from practice. The testing team finds defects in a timely manner, the development team does not have the time to correct them, the product goes unchanged to the market, customers say “sucks” ... What, was testing bad?
Or let's do the opposite. Testers carefully examine the product, using it both in the tail and in the mane, they do not find a single error, it is released to the market, users are satisfied, caps fly into the air, flowers are sent to the address of the developer ... Testers are great or not?
In general, there are many examples. But the conclusion is almost always the same: quality testing has no effect. Just as your haircut does not affect your success. "Hey, hairdresser, cut me so that I pass the exam!". No logic, right? In testing the same. Testing provides information, in a certain budget, in a certain format, at a certain speed ... But testing does not affect the quality in any way, even if it really wants to.
What then can provide testing?
All our results, all our work is described by a formula of four variables: test coverage, information provided, speed and budget.
Let's first look at these points, and then move on to the formulas themselves.
These are the% of possible user scenarios, conditions, settings, input parameters, etc., that were tested by the testing team. The higher the percentage, the more bugs found, the less missed. The more bugs can be fixed if desired and possible.
At the same time, the test coverage and the number of tests, as well as the resources spent on testing, are linked very indirectly. Qualified testers have a whole arsenal of techniques and tools that allow you to "shove un-suited", aka optimize the test suite, aka ensure maximum coverage with the minimum number of tests.
Coverage can and should be measured: traceability matrix, code coverage,% of missing defects and lots of other metrics to help you.
The meme has been living for years about the tester, with panic in his eyes saying “nothing works!”. And if you ask for an explanation in more detail, he replies: “well, absolutely nothing!”. He may have provided good test coverage, but, alas, he did not provide adequate information. What kind of information might a project need from testing?
If the testing team does not provide intelligible information about the product and the project (and this is its main function!), Then it is very difficult to make decisions.
OK, the coverage is good, the information is clear ... Only late. Or long. Or long, and therefore late.
All of you are clearly faced with a situation: before the release of 3 days, there is a critical bug. The developer gets it in the head, starts to fix it, and it turns out that this bug has been for half a week, and actually it could have been found much earlier. And that this bug 2 months ago would have been fixed in half an hour, and now half the food is tied to it, and it will take a week to mess around.
Few people undertake to analyze TOC projects, and few people notice that testing sometimes extends releases several times. Here, replace the testing with timely, and get the release twice as fast.
This is usually imperceptible, and it seems that bugs are the source of all the troubles ... But sometimes finding them earlier = reducing the cost of the entire development cycle!
Or another aspect of testing speed: the time required to test the assembly. For example, we are preparing for release. Get RC (release candidate). Testing takes a week to make sure it works, the RM agrees, allocates a week, on the fifth day there is a critical bug ... 10 minutes to fix, and again 5 days, and again on the fifth there is a critical bug ... Faced? Sounds familiar?
Perhaps this is the easiest item of all. Naturally, it is relative. 1000 rubles - is it a lot or a little? For a loaf of bread, perhaps, a lot, for a trip to Bali, perhaps, a little. Therefore, the budget is the money that you are willing to pay, but not for “testing”, but for concrete and understandable results of this testing.
And then that's what. To bring the project maximum benefit, you need to know exactly what it needs right now. To know what you need, you need to analyze its performance, what exactly is outside of the norm: time, quality or budget. And after that create your unique formula with test requirements. At the same time, let's be honest: “shorten the time” or “increase coverage” - these are all the same “massage me.”
Need accurate performance. What for? To evaluate the changes. To see if they really affect the performance of the entire project as a whole (let's be honest, you don’t need coverage by ourselves!). To control the process.
Notice, I never said "unit tests", "automation", "test design", etc. All actions are made to the process only by the results of certain goals! Need to speed up assembly testing? Implement automation. Need to increase the speed of finding defects? Unit Tests. Quality of reporting? Moderation of defects. In testing hundreds of approaches, tools, solutions that can be combined based on your goals. But the tools themselves are secondary.
Therefore, HOW the tester / test manager / test outsourcer will achieve your goals - this is his headache. But no one better than attentive to his project, RM will not say that this project is necessary!
If you ask an average RM a simple question: “Why is there testing on this project?”, Then the answer most often is “You are a test manager, and you should answer this question.”
But when you come to the hairdresser you do not tell the master “you yourself know what I need”? And in the grocery store you do not ask the seller to throw in your basket what you need? You can consult, you can learn “how can you?”, Ask for options, but the decision is always yours. What is the difference between testing? Maybe because too few project managers understand why they need it?
')
In this article I will try to play the role of a seller who shows the client: “But what happens at all?” Many things will be described, perhaps in too much detail, too simply ... Do not be angry, I just really want to be understood :)
Why even formulate any expectations from testing?
Kamon! Testing is all there is pressing the buttons, well, if automated. And so it is clear what goals! It is necessary to find bugs, and the more, the better. What else could be the goals of testing?
Oops ... If we proceed from this approach, then in all companies with testing everything is in order. Bugs are, fix, checked ... And what, testing is always useful? Always reveals its full potential?
Something tells me that no. Then what is the difference between those rare cases when “testing is great” and the overwhelming majority?
I will try to give another illustration. You never went for a massage in your life. You go to the salon (and what, everyone goes, why don't you go?) And order a half-hour session. You have a fixed budget and there are requirements: “Massage me.” A lady of indistinct appearance vaguely fiddling with your quite distinct shoulders, no buzz, you leave the salon and think “oh, massage is a complete garbage”. But after all your requirement “massage me” was fulfilled! Maybe the matter is incorrect requirements? Your back hurt and you wanted the pain to go away? Or not enough stretch marks? Or would you like to relax? The more accurate the requirements, the greater the chance for the contractor to satisfy them. But if you cannot formulate what you want, then the performer will not be able to do what is needed, and you will not be able to objectively evaluate the result.
What can be the expectations from testing?
Testing is needed in order to improve the quality of the product. Well, it's obvious!
Oops ... A simple example from practice. The testing team finds defects in a timely manner, the development team does not have the time to correct them, the product goes unchanged to the market, customers say “sucks” ... What, was testing bad?
Or let's do the opposite. Testers carefully examine the product, using it both in the tail and in the mane, they do not find a single error, it is released to the market, users are satisfied, caps fly into the air, flowers are sent to the address of the developer ... Testers are great or not?
In general, there are many examples. But the conclusion is almost always the same: quality testing has no effect. Just as your haircut does not affect your success. "Hey, hairdresser, cut me so that I pass the exam!". No logic, right? In testing the same. Testing provides information, in a certain budget, in a certain format, at a certain speed ... But testing does not affect the quality in any way, even if it really wants to.
What then can provide testing?
All our results, all our work is described by a formula of four variables: test coverage, information provided, speed and budget.
Let's first look at these points, and then move on to the formulas themselves.
1. Test coating
These are the% of possible user scenarios, conditions, settings, input parameters, etc., that were tested by the testing team. The higher the percentage, the more bugs found, the less missed. The more bugs can be fixed if desired and possible.
At the same time, the test coverage and the number of tests, as well as the resources spent on testing, are linked very indirectly. Qualified testers have a whole arsenal of techniques and tools that allow you to "shove un-suited", aka optimize the test suite, aka ensure maximum coverage with the minimum number of tests.
Coverage can and should be measured: traceability matrix, code coverage,% of missing defects and lots of other metrics to help you.
2. Information provided
The meme has been living for years about the tester, with panic in his eyes saying “nothing works!”. And if you ask for an explanation in more detail, he replies: “well, absolutely nothing!”. He may have provided good test coverage, but, alas, he did not provide adequate information. What kind of information might a project need from testing?
- Qualitatively described bugs in the bug tracker
- Statistical product information for forecasting releases and making decisions on processes
- Quality metrics for making decisions about releases
- Evaluation of the test itself: what kind of coverage, what is verified, how much testing is enough, etc.
If the testing team does not provide intelligible information about the product and the project (and this is its main function!), Then it is very difficult to make decisions.
3. Testing speed
OK, the coverage is good, the information is clear ... Only late. Or long. Or long, and therefore late.
All of you are clearly faced with a situation: before the release of 3 days, there is a critical bug. The developer gets it in the head, starts to fix it, and it turns out that this bug has been for half a week, and actually it could have been found much earlier. And that this bug 2 months ago would have been fixed in half an hour, and now half the food is tied to it, and it will take a week to mess around.
Few people undertake to analyze TOC projects, and few people notice that testing sometimes extends releases several times. Here, replace the testing with timely, and get the release twice as fast.
This is usually imperceptible, and it seems that bugs are the source of all the troubles ... But sometimes finding them earlier = reducing the cost of the entire development cycle!
Or another aspect of testing speed: the time required to test the assembly. For example, we are preparing for release. Get RC (release candidate). Testing takes a week to make sure it works, the RM agrees, allocates a week, on the fifth day there is a critical bug ... 10 minutes to fix, and again 5 days, and again on the fifth there is a critical bug ... Faced? Sounds familiar?
4. Budget
Perhaps this is the easiest item of all. Naturally, it is relative. 1000 rubles - is it a lot or a little? For a loaf of bread, perhaps, a lot, for a trip to Bali, perhaps, a little. Therefore, the budget is the money that you are willing to pay, but not for “testing”, but for concrete and understandable results of this testing.
So what's next?
And then that's what. To bring the project maximum benefit, you need to know exactly what it needs right now. To know what you need, you need to analyze its performance, what exactly is outside of the norm: time, quality or budget. And after that create your unique formula with test requirements. At the same time, let's be honest: “shorten the time” or “increase coverage” - these are all the same “massage me.”
Need accurate performance. What for? To evaluate the changes. To see if they really affect the performance of the entire project as a whole (let's be honest, you don’t need coverage by ourselves!). To control the process.
Notice, I never said "unit tests", "automation", "test design", etc. All actions are made to the process only by the results of certain goals! Need to speed up assembly testing? Implement automation. Need to increase the speed of finding defects? Unit Tests. Quality of reporting? Moderation of defects. In testing hundreds of approaches, tools, solutions that can be combined based on your goals. But the tools themselves are secondary.
Therefore, HOW the tester / test manager / test outsourcer will achieve your goals - this is his headache. But no one better than attentive to his project, RM will not say that this project is necessary!
Source: https://habr.com/ru/post/135761/
All Articles