Quality is the responsibility of the team. Our QA Experience
I work as a QA engineer at Miro . I'll tell you about our experiment on transferring to the developers a part of the tasks of testing and transforming the role of the tester into the role of QA (Quality assurance).
First, briefly about our development process. We have daily client releases and from 3 to 5 server releases per week. The development team has 60+ people who are divided into 10 functional scrum teams.
I work in the Integration team, whose task is the integration of our service into external products and the integration of external products into our service. For example, we have integrated Jira task tracker . Jira Cards - visual display of tasks with which you can conveniently work on the board, without going to Jira.
')
It all started with a banal problem. When one of the testers went to the hospital - the performance of the team was greatly reduced. The team continued to work on the tasks, but the code reached the testing stage and the task was paused. As a result, the new functionality did not get to production on time.
Leaving the tester on vacation - a separate story. He needs to find in advance one of the testers who is ready to take on his tasks in addition to his own, to agree, to plunge into the tasks. At the same time, two testers go on vacation - an unaffordable luxury.
We began to think about how to solve the problem, and realized that the real problem is that the narrow neck of the development process is testing. I will show on the example of several stories.
I have a developer. Everyone has their own tasks. The developer finished one of the tasks and gave it to me for testing. Since this task has a higher priority than my current ones, I switch to it. I find bugs, I bring everything to Jira and give it back to the developer for revision.
Returning to the tasks that had to be postponed. The developer switches from new tasks to the task I returned. Rules bugs and returns to me for testing. I find the bugs again and return the task again. This process can go on for a very long time.
As a result, the total time spent working on a task increases several times, after which Time to market increases, which is critical for us as a grocery company. There are several reasons for increasing the time spent on the task:
There are several developers in our scrum team. They regularly submit their tasks to me for testing. Formed a queue of tasks for which I take on the basis of priorities.
So we worked for about a year, but during this time the company has greatly increased: the number of projects and developers has increased, and hence the number of tasks in testing. At the same time, in our team, I was still the only tester and could not multiply increase my speed. As a result, more and more tasks accumulated in the testing queue, and the process of transferring tasks between the developer and the tester increased the waiting time.
Suddenly I got sick. Delivery of new features from our production team has completely stopped. What should the team do in this situation? You can ask for help from a tester from another team. But, with a high degree of probability, he will not be immersed in our specifics, which will adversely affect the quality and timing of both teams.
The conclusion from both stories is the same - the teams depend too much on the tester:
The most obvious thought is to increase the staff of testers. This will work, but only up to a certain point: the number of tasks will constantly grow, and it will be impossible to increase the number of testers indefinitely - at some point it will become expensive and inefficient. On the topic of “resource thinking” problems (can't you solve the problem? Hire another employee) , wrote well Fedor Ovchinnikov , CEO of Dodo Pizza.
It is much more efficient to maintain the speed and quality of development within the framework of current resources. Therefore, we decided to launch an experiment that will help teams create functionality at once, taking into account all the risks and borderline situations. They called it Transform tester to QA, because it is about the transformation of one of the roles in the team: from a monkey tester who identifies errors behind the developer, to a QA engineer who deliberately ensures quality at all stages of the development process.
Objectives of the experiment:
The first step was to change the thinking of the team. Everyone is used to the fact that the tester is responsible for the quality of the team.
Our hypothesis was that increasing the time to prepare and test tasks will help reduce the number of iterations in the work on one task. This, in turn, will allow you to display new functionality on production without losing speed and quality, and maybe faster with better quality.
Together with the team and with testers from other teams, we have developed a new scheme of work. For six months, how the team works on it, we have taken into account the difficulties and problems that have arisen along the way, and today it looks like this:
Product Owner presents a task to the team. The team analyzes the formulation to identify border situations from the technical and product sides. If there are questions that need to be further investigated - a separate task is put, for which time is allocated in the sprint.
The result of the analysis of the problem statement is a technical solution that consists of one or several developers. All relevant company employees, including QA, should be familiar with and agree with the technical solution. The technical solution describes the problem we are solving, the functional blocks of the product that will be affected, and the proposed plan for making changes to the code.
This solution allows us to identify a simpler, better solution based on the experience of relevant participants in the development process. Having a detailed technical description on hand - it is easier to see and avoid blocking problems, it is easier to carry out a code review.
Here is an example of some blocks of the technical solution:
In parallel, the developer or QA compiles a test scenario that takes into account all possible places of occurrence of problems. If the developer does this, he will necessarily give the script a QA check, which checks it for completeness and sufficiency. If the script is QA - the developer must confirm that he understands how to perform each of his items. The team also evaluates the script for correctness.
To compile and store scripts we use HipTest.
The developer starts writing the code based on the technical solution and taking into account all possible situations described in the test documentation. Passes code review and checks the code for test scenarios. Produces merzh in master.
At this stage, QA provides the necessary support to the developer. For example, it helps with reproducing complex cases, setting up a test environment, carrying out load testing, suggests possible nuances of outputting large features to production.
This is the final stage. Here, the team may be required to pre / post-release actions, for example, the inclusion of new functionality for beta users.
The new scheme of work required from developers a greater immersion in the testing process. Therefore, as QA, it was important for me to provide them with all the necessary tools and teach the basics of functional testing. Together with QA from other teams, we have compiled a list of the necessary documentation, created the missing testing instructions and refined the existing ones taking into account things that are not obvious to developers.
Then we gave developers access to all the testing tools and test environments and began to jointly conduct testing. At first, the developers did not want to take responsibility for the test results, because nobody else checked anything for them, it was unusual for them. Each developer had only a test script, the functionality developed by him and the Merge button. But gradually they got used to it.
Six months have passed since the start of the experiment. The graph displays statistics of the number of bugs by week in our team. The red color indicates the number of all bugs found by the team, green - the number of corrected.
It can be seen that the level of the red line remained at the same level, except for a surge at the beginning of the experiment. There are no more bugs, which means that we have kept the current level of quality.
At the same time, the average work time on the tasks decreased by only 2%: before the start of the experiment it was 12 hours and 40 minutes, after - 12 hours and 25 minutes. So we were able to maintain the current speed of work on the tasks.
As a result, our team now has no hard dependence on QA. If I get sick or go on vacation - the team will continue to work without loss in speed and quality.
Do we consider the experiment successful, despite the fact that the indicators of time and quality remained the same? Yes, we believe, because we have kept the speed and quality of work and freed up QA time for more conscious work on product quality and development processes. For example, for conducting research testing, increasing the coverage of functional tests and describing the principles and rules for developing test documentation in all teams.
In other teams we sowed the grain of the experiment. For example, in one of the QA teams, it is no longer testing what is described by the developer in pull-quest, because the developer can see it himself. In another team, QA analyzes the changes of the request and checks only complex and non-obvious cases.
This is a difficult and long experiment. It is not implemented by clicking the fingers, you need to carefully prepare for it and not wait for quick results.
Be prepared for negativity from the team. You can't just say that the developers are now checking the functionality. It is necessary to prepare them for this, develop approaches, explain the value of the experiment for the team and the product.
Loss of speed at the start is inevitable. Time to train developers, write missing documentation and reorganize processes takes time, which will have to be sacrificed in favor of the experiment.
There is no ready solution. Similar processes are being implemented, for example, in Atlassian , but this does not mean that you will also be able to implement them as is. The adaptation to the company's culture and the specifics of the teams is important.
First, briefly about our development process. We have daily client releases and from 3 to 5 server releases per week. The development team has 60+ people who are divided into 10 functional scrum teams.
I work in the Integration team, whose task is the integration of our service into external products and the integration of external products into our service. For example, we have integrated Jira task tracker . Jira Cards - visual display of tasks with which you can conveniently work on the board, without going to Jira.
')
How did the experiment begin
It all started with a banal problem. When one of the testers went to the hospital - the performance of the team was greatly reduced. The team continued to work on the tasks, but the code reached the testing stage and the task was paused. As a result, the new functionality did not get to production on time.
Leaving the tester on vacation - a separate story. He needs to find in advance one of the testers who is ready to take on his tasks in addition to his own, to agree, to plunge into the tasks. At the same time, two testers go on vacation - an unaffordable luxury.
We began to think about how to solve the problem, and realized that the real problem is that the narrow neck of the development process is testing. I will show on the example of several stories.
Story One: an endless task toss
I have a developer. Everyone has their own tasks. The developer finished one of the tasks and gave it to me for testing. Since this task has a higher priority than my current ones, I switch to it. I find bugs, I bring everything to Jira and give it back to the developer for revision.
Returning to the tasks that had to be postponed. The developer switches from new tasks to the task I returned. Rules bugs and returns to me for testing. I find the bugs again and return the task again. This process can go on for a very long time.
As a result, the total time spent working on a task increases several times, after which Time to market increases, which is critical for us as a grocery company. There are several reasons for increasing the time spent on the task:
- The task constantly shifts between development and testing.
- The task is idle while waiting for the tester or developer.
- The developer and tester have to regularly switch between tasks, which requires additional time and energy.
History Two: A Growing Queue of Tasks
There are several developers in our scrum team. They regularly submit their tasks to me for testing. Formed a queue of tasks for which I take on the basis of priorities.
So we worked for about a year, but during this time the company has greatly increased: the number of projects and developers has increased, and hence the number of tasks in testing. At the same time, in our team, I was still the only tester and could not multiply increase my speed. As a result, more and more tasks accumulated in the testing queue, and the process of transferring tasks between the developer and the tester increased the waiting time.
Suddenly I got sick. Delivery of new features from our production team has completely stopped. What should the team do in this situation? You can ask for help from a tester from another team. But, with a high degree of probability, he will not be immersed in our specifics, which will adversely affect the quality and timing of both teams.
The conclusion from both stories is the same - the teams depend too much on the tester:
- The performance of the whole team is limited by the performance of the tester.
- The number of developers cannot be increased without increasing the staff of testers.
- Increasing the speed of development does not make sense if all the tasks are queued for testing.
- While the developer’s work is checked by the tester, the developer’s sense of responsibility for quality is reduced.
- If there is no tester, then the process of withdrawing a new functional suffers.
Let's increase the staff of testers?
The most obvious thought is to increase the staff of testers. This will work, but only up to a certain point: the number of tasks will constantly grow, and it will be impossible to increase the number of testers indefinitely - at some point it will become expensive and inefficient. On the topic of “resource thinking” problems (can't you solve the problem? Hire another employee) , wrote well Fedor Ovchinnikov , CEO of Dodo Pizza.
It is much more efficient to maintain the speed and quality of development within the framework of current resources. Therefore, we decided to launch an experiment that will help teams create functionality at once, taking into account all the risks and borderline situations. They called it Transform tester to QA, because it is about the transformation of one of the roles in the team: from a monkey tester who identifies errors behind the developer, to a QA engineer who deliberately ensures quality at all stages of the development process.
Let's improve the development processes.
Objectives of the experiment:
- Remove the dependence of the team on the testers, but without loss of quality and time.
- Improve the quality assurance of QA-engineers in teams.
The first step was to change the thinking of the team. Everyone is used to the fact that the tester is responsible for the quality of the team.
Our hypothesis was that increasing the time to prepare and test tasks will help reduce the number of iterations in the work on one task. This, in turn, will allow you to display new functionality on production without losing speed and quality, and maybe faster with better quality.
Together with the team and with testers from other teams, we have developed a new scheme of work. For six months, how the team works on it, we have taken into account the difficulties and problems that have arisen along the way, and today it looks like this:
- Presentation of the production.
- Technical solution and test script.
- Development and verification.
- Release
Formulation of the problem
Product Owner presents a task to the team. The team analyzes the formulation to identify border situations from the technical and product sides. If there are questions that need to be further investigated - a separate task is put, for which time is allocated in the sprint.
Technical solution
The result of the analysis of the problem statement is a technical solution that consists of one or several developers. All relevant company employees, including QA, should be familiar with and agree with the technical solution. The technical solution describes the problem we are solving, the functional blocks of the product that will be affected, and the proposed plan for making changes to the code.
This solution allows us to identify a simpler, better solution based on the experience of relevant participants in the development process. Having a detailed technical description on hand - it is easier to see and avoid blocking problems, it is easier to carry out a code review.
Here is an example of some blocks of the technical solution:
Task Description
Are new entities or approaches being added to the system?
If yes, then they are described and it is explained why it is impossible to solve the problem within the framework of existing approaches.
Does the server interaction change within the cluster? Are new cluster roles being added?
Is the data model changing?
For the server we are talking about objects and models.
If the data model is complex, you can present it as a UML diagram, or as a text description.
Does the interaction between the client and the server change?
Description of the changes. If this is an API, will it be possible to give it to external users? Do not forget about error handling - i.e. specify the correct reason.
Test script
In parallel, the developer or QA compiles a test scenario that takes into account all possible places of occurrence of problems. If the developer does this, he will necessarily give the script a QA check, which checks it for completeness and sufficiency. If the script is QA - the developer must confirm that he understands how to perform each of his items. The team also evaluates the script for correctness.
To compile and store scripts we use HipTest.
Design and validation
The developer starts writing the code based on the technical solution and taking into account all possible situations described in the test documentation. Passes code review and checks the code for test scenarios. Produces merzh in master.
At this stage, QA provides the necessary support to the developer. For example, it helps with reproducing complex cases, setting up a test environment, carrying out load testing, suggests possible nuances of outputting large features to production.
Release of the finished functional
This is the final stage. Here, the team may be required to pre / post-release actions, for example, the inclusion of new functionality for beta users.
Documentation and tools
The new scheme of work required from developers a greater immersion in the testing process. Therefore, as QA, it was important for me to provide them with all the necessary tools and teach the basics of functional testing. Together with QA from other teams, we have compiled a list of the necessary documentation, created the missing testing instructions and refined the existing ones taking into account things that are not obvious to developers.
Then we gave developers access to all the testing tools and test environments and began to jointly conduct testing. At first, the developers did not want to take responsibility for the test results, because nobody else checked anything for them, it was unusual for them. Each developer had only a test script, the functionality developed by him and the Merge button. But gradually they got used to it.
Experimental results
Six months have passed since the start of the experiment. The graph displays statistics of the number of bugs by week in our team. The red color indicates the number of all bugs found by the team, green - the number of corrected.
It can be seen that the level of the red line remained at the same level, except for a surge at the beginning of the experiment. There are no more bugs, which means that we have kept the current level of quality.
At the same time, the average work time on the tasks decreased by only 2%: before the start of the experiment it was 12 hours and 40 minutes, after - 12 hours and 25 minutes. So we were able to maintain the current speed of work on the tasks.
As a result, our team now has no hard dependence on QA. If I get sick or go on vacation - the team will continue to work without loss in speed and quality.
Do we consider the experiment successful, despite the fact that the indicators of time and quality remained the same? Yes, we believe, because we have kept the speed and quality of work and freed up QA time for more conscious work on product quality and development processes. For example, for conducting research testing, increasing the coverage of functional tests and describing the principles and rules for developing test documentation in all teams.
In other teams we sowed the grain of the experiment. For example, in one of the QA teams, it is no longer testing what is described by the developer in pull-quest, because the developer can see it himself. In another team, QA analyzes the changes of the request and checks only complex and non-obvious cases.
What should be remembered before the start of the experiment
This is a difficult and long experiment. It is not implemented by clicking the fingers, you need to carefully prepare for it and not wait for quick results.
Be prepared for negativity from the team. You can't just say that the developers are now checking the functionality. It is necessary to prepare them for this, develop approaches, explain the value of the experiment for the team and the product.
Loss of speed at the start is inevitable. Time to train developers, write missing documentation and reorganize processes takes time, which will have to be sacrificed in favor of the experiment.
There is no ready solution. Similar processes are being implemented, for example, in Atlassian , but this does not mean that you will also be able to implement them as is. The adaptation to the company's culture and the specifics of the teams is important.
Source: https://habr.com/ru/post/449082/
All Articles