Testing is not a search for errors!
Many people believe that software testing is a search for errors. Sometimes I say to testers: “do not try to find as many errors as possible, try to miss as little as possible!”, And they don’t understand me: what is the difference?
And the difference is huge! In this article I want to tell what it is, and what tools you need to use for this useful testing.
I am testing a product. My task is to get as many bugs as possible. It is logical! It is always nice to get bugs to a tester, this is a visible measurable result of work, And the more of them, the more I appreciate as a tester.
What areas will I test in this case? First of all, the most unstable. Often they are unstable because they are of lower priority, but it does not matter, the number of bugs is much more important.
')
What happens if I encounter a complex reproducible bug? ROI on his research is considered in the head very quickly. Why should I bother with him, if I can start 3 less critical, but simple ones in an institution?
What tests will I do first? Koneno, the most non-standard. Enter “War and Peace” into the login field, divide by zero, insert a photo in the .exe format into the profile.
I will tell you a secret - sometimes at interviews, testers in response to a request to "test the calculator" list interesting and efficient tests, but among the first thirty there are no test "check addition" and other basic operations.
This is exactly what the search for errors looks like - having nothing to do with testing.
I am testing a product. My task is to skip as few priority bugs as possible for the user. The less bugs are missed, the less customer dissatisfaction is expressed - the higher I estimate the effectiveness of my work.
What areas will I test in this case? Naturally, I will start with the highest priority for the user. Even if they work stably and successfully, I will still check the main user scenarios in order not to miss serious problems.
What happens if I run into difficulties? For example, with a complexly reproducible defect, or a lack of understanding of the user's business process, or a lack of requirements? If this is an important functional, then I will find out "what is wrong", "how correctly". As a result, it may take a lot of time to create a defect, and from the point of view of the bug / time, the result of testing efficiency will not be very high, but I will have deeper knowledge about the product, architecture, users.
What tests will I do first? Of course, the very best. Execution of the most basic scenario in the most basic conditions to make sure that the most important functionality works. And only after that I will move to less standard scenarios.
In the case of the search for errors, in the short term, the results are higher: the bugs start up more and immediately.
But in the long run, everything is not so rosy:
For testing to be effective and useful in the long run, you need to follow simple rules and use key testing tools:
Clicking on the buttons, you can get a lot of bugs - but you can not say that it was checked. The only solution is to document the tests. Detailed test cases, depressing testers and taking a lot of time, are needed very rarely. But checklists with a list of "what you need to check" are necessary.
What do they give:
The key to success in conducting tests - creating a map on which you will go. The goal is to cover the entire product. But please, do not need excuses about the terrible resource intensity - I covered projects with millions of lines of code in less than a month and a half. And in the process of writing such tests, unexpected questions were raised and critical errors surfaced that, despite the presence of unfortunate testers, have been hanging out in the product for years.
In order not to be blind kittens, it is necessary to evaluate the effectiveness of testing. Analyze the missed errors and the reasons for missing them. Covering the functionality and code tests. Level of user satisfaction through questionnaires and feedback collection. The quality of making mistakes by polling the developers.
There is ALWAYS something to improve, and the lack of a continuous process of improvement is an inevitable swamp.
Many believe that testing has some mythical purpose. And that they are always the same.
No matter how wrong!
In each project, company, team, goals of their own. Do all understand them the same way? Did you say them out loud?
To bring the maximum benefit, you need to understand well what this very benefit is. And do not be surprised if the opinion of RMs and developers will not match yours. We must not persuade them, and adapt to the current project goals!
It's a mystery to me how this is possible, but nonetheless it is a fact: testers often check the product without knowing anything about the user.
No guesswork and no thought about "on average about the industry"! Testers should IDEALLLY know their users. Often, this information is not provided to them by analysts. Think again! Without knowing the user, it is impossible to test the product normally.
To illustrate, here’s a bug I recently got in a bug tracker:
Visit the website of the tested product http: //****.ru in the Firefox browser
Enter username and password
Log in from the same computer in Opera browser
Requests to re-enter the login and password, does not automatically login.
Such bugs are not just useless, they disgrace testers and discredit the industry as a whole! To get defects correctly, it is necessary to understand the platform on which the test product is written. If we are talking about web testing, then you can at least specify the error code returned by the server in the bug report, see the details with a fireball, provide detailed information and save a lot of time on development!
Many developers do not like testers. And rightly so!
But good testers love and appreciate everything. But testers, not clickers and bagzavodiltsev!
Learn to find out what is wrong, what other members of the development team do not like. Be sure to investigate the missed errors and do everything in order to not miss them anymore. Do not pursue the establishment of bugs - your mantra should be "user happiness", "quality product" and "successful project", and not "to get as many bugs as possible" - VERY often these 2 goals are too far from each other.
And may the force be with you!
And the difference is huge! In this article I want to tell what it is, and what tools you need to use for this useful testing.
What is the search for errors?
I am testing a product. My task is to get as many bugs as possible. It is logical! It is always nice to get bugs to a tester, this is a visible measurable result of work, And the more of them, the more I appreciate as a tester.
What areas will I test in this case? First of all, the most unstable. Often they are unstable because they are of lower priority, but it does not matter, the number of bugs is much more important.
')
What happens if I encounter a complex reproducible bug? ROI on his research is considered in the head very quickly. Why should I bother with him, if I can start 3 less critical, but simple ones in an institution?
What tests will I do first? Koneno, the most non-standard. Enter “War and Peace” into the login field, divide by zero, insert a photo in the .exe format into the profile.
I will tell you a secret - sometimes at interviews, testers in response to a request to "test the calculator" list interesting and efficient tests, but among the first thirty there are no test "check addition" and other basic operations.
This is exactly what the search for errors looks like - having nothing to do with testing.
What is testing?
I am testing a product. My task is to skip as few priority bugs as possible for the user. The less bugs are missed, the less customer dissatisfaction is expressed - the higher I estimate the effectiveness of my work.
What areas will I test in this case? Naturally, I will start with the highest priority for the user. Even if they work stably and successfully, I will still check the main user scenarios in order not to miss serious problems.
What happens if I run into difficulties? For example, with a complexly reproducible defect, or a lack of understanding of the user's business process, or a lack of requirements? If this is an important functional, then I will find out "what is wrong", "how correctly". As a result, it may take a lot of time to create a defect, and from the point of view of the bug / time, the result of testing efficiency will not be very high, but I will have deeper knowledge about the product, architecture, users.
What tests will I do first? Of course, the very best. Execution of the most basic scenario in the most basic conditions to make sure that the most important functionality works. And only after that I will move to less standard scenarios.
Test and error search results
In the case of the search for errors, in the short term, the results are higher: the bugs start up more and immediately.
But in the long run, everything is not so rosy:
- due to the lack of deep knowledge about the product, gradually begins to grow% of missing defects
- the development team is busy fixing terrible-terrible-unthinkable bugs, obtained by clicking on the same button 144 times under IE in full moon
- Some horrible unpleasant and obvious user bugs fall into the release.
- the number of errors found in a long-term perspective is falling
How to go from finding errors to testing?
For testing to be effective and useful in the long run, you need to follow simple rules and use key testing tools:
1. Product analysis and test documentation
Clicking on the buttons, you can get a lot of bugs - but you can not say that it was checked. The only solution is to document the tests. Detailed test cases, depressing testers and taking a lot of time, are needed very rarely. But checklists with a list of "what you need to check" are necessary.
What do they give:
- You analyze the product, write out the main features, actions, their parameters. Thus, the risk of forgetting something is significantly reduced.
- The checklists are a great reminder, “you have to go deeper here.” There is some vague feature with insufficient description. How to test it? In testing without tests, the easiest way is to say “I will come back to this later,” and never to return. And with the tests - you will have a test in which it is not clear how and what to check, you will see such tests and will not forget the need to find out.
- Check lists can and should be agreed. With developers, analysts. The whole team is involved in the testing process, testers learn a lot about the product, collective intelligence improves the quality of testing. And besides a one-time improvement in the quality of an individual checklist, the quality of testing as a whole improves: testers begin to take more account of in testing, to develop, this knowledge over time pays off in the form of more effective testing.
The key to success in conducting tests - creating a map on which you will go. The goal is to cover the entire product. But please, do not need excuses about the terrible resource intensity - I covered projects with millions of lines of code in less than a month and a half. And in the process of writing such tests, unexpected questions were raised and critical errors surfaced that, despite the presence of unfortunate testers, have been hanging out in the product for years.
2. Evaluation of testing
In order not to be blind kittens, it is necessary to evaluate the effectiveness of testing. Analyze the missed errors and the reasons for missing them. Covering the functionality and code tests. Level of user satisfaction through questionnaires and feedback collection. The quality of making mistakes by polling the developers.
There is ALWAYS something to improve, and the lack of a continuous process of improvement is an inevitable swamp.
3. Discussion of testing objectives with the team
Many believe that testing has some mythical purpose. And that they are always the same.
No matter how wrong!
In each project, company, team, goals of their own. Do all understand them the same way? Did you say them out loud?
To bring the maximum benefit, you need to understand well what this very benefit is. And do not be surprised if the opinion of RMs and developers will not match yours. We must not persuade them, and adapt to the current project goals!
4. Understanding users and their business processes
It's a mystery to me how this is possible, but nonetheless it is a fact: testers often check the product without knowing anything about the user.
- How is this product used?
- Why is it needed at all, what problems does it solve?
- What is the average user qualification?
- In what conditions do users work? What environments, equipment?
No guesswork and no thought about "on average about the industry"! Testers should IDEALLLY know their users. Often, this information is not provided to them by analysts. Think again! Without knowing the user, it is impossible to test the product normally.
5. Technical Qualification and Architecture Understanding
To illustrate, here’s a bug I recently got in a bug tracker:
Visit the website of the tested product http: //****.ru in the Firefox browser
Enter username and password
Log in from the same computer in Opera browser
Requests to re-enter the login and password, does not automatically login.
Such bugs are not just useless, they disgrace testers and discredit the industry as a whole! To get defects correctly, it is necessary to understand the platform on which the test product is written. If we are talking about web testing, then you can at least specify the error code returned by the server in the bug report, see the details with a fireball, provide detailed information and save a lot of time on development!
findings
Many developers do not like testers. And rightly so!
But good testers love and appreciate everything. But testers, not clickers and bagzavodiltsev!
Learn to find out what is wrong, what other members of the development team do not like. Be sure to investigate the missed errors and do everything in order to not miss them anymore. Do not pursue the establishment of bugs - your mantra should be "user happiness", "quality product" and "successful project", and not "to get as many bugs as possible" - VERY often these 2 goals are too far from each other.
And may the force be with you!
Source: https://habr.com/ru/post/149903/
All Articles