Testing under the SCRUM. Thorns, rakes and successes
SCRUM is a project management framework in which our team tried to implement the testing phase. In this article we want to help not make our mistakes to those teams that are just starting to get acquainted with SCRUM. Participants of the same experienced SCRUM teams are invited to comment to share comments and successes. And as illustrations - racetracks, marathon runners, obstacles. They are here for a reason. Yes, and the Olympics, in the end.
Let's start by listing the key features of SCRUM for those who are not familiar with it:
Live Tipping's use of the SCRUM methodology in some products subsequently yielded positive results: the team achieved discipline without supervision from the manager; the tasks were assessed more precisely because their evaluation did not occur at the beginning of the project, but in the course of when it was already known about some pitfalls; From the customer’s side there was a fast and high-quality feedback. But along with its successful implementation, the question arose of how to embed an existing testing process into the framework of this methodology. Reflections of the testing department of our company formed the basis of everything that is written below.
')
At the start, attention ...
The company's priority vector is mobile application development. This often creates specific working conditions: the deadlines are tight, there is no normal documentation (as is the case with startups), and customer requirements change often and dramatically. Because of this, a number of difficulties arise in the organization of testing, the main of which is time. If you compare a large enterprise project and a mobile startup, then in the first case there is a sea of ​​requirements for the project, but it’s simpler to work - there are often approved documentation and the deadlines are not so tight. In mobile development, projects are not always as large-scale, but because of the specifics of testing mobile platforms, special requirements for applications and the frequent lack of technical documentation, the scope of work is comparable, and less time is spent on testing.
Hence the following problems of testing mobile applications:
Changes in the requirements occur "on the fly", so there is no point in making TK - more time will be spent on updating it than on developing a product. There are also problems in communication and awareness within the team: the developer clarified the details of the customer, but did not share with the others, or the tester was not at the rally, where important decisions were made that were not documented due to lack of time. Etc. As a result, the team does not have a clear understanding of the project. Hence the following problem:
Often the work on the product is organized in such a way that the assembly is given for testing when the lion's share of functions is ready. If the tester did not follow the development of the product, then before testing he should spend a lot of time on understanding his work.
When your application, for example, provides calendar integration, uses self-developed chat rooms, gets data from social networks and shares information through them or interacts with cloud services like Google Drive, the complexity of testing increases exponentially due to various interaction options.
The main problem with the implementation of testing in SCRUM was the prevailing testing practice at that time. It involved lengthy development stages, then transferring the assembly to tests, receiving bug reports, editing, testing again, editing again, and so on until the functions declared at this stage started to work correctly. The meaning of the SCRUM is in relatively short sprints, at the end of which you have some decorated and working increment of the product. As a result, we have two schemes.
Scheme 1 . Let's call it "Single test + check fixes."
Key points:
It turned out that the bugs found in the current sprint were fixed only in the following. But since the customer was aware of the status of the project, and the final acceptance was carried out only at the end of the stage, more time is spent on the sprint for tests and fixes, and everything seems to be normal.
Advantages of this approach:
This approach was very similar to that prevailing in the company prior to the transition to SCRUM, which made it possible to become familiar with the methodology relatively painlessly. Also, by the end of the stage, fully tested functions were surrendered, which freed me from the pangs of conscience.
Cons of this approach:
We illustrate this drawback with a case in the company. The tester is used to being able to contact the manager for any explanation. But it is more profitable for the manager to view the project himself and quickly send the assembly to the client, telling about possible shortcomings, than to give it to the tester, who can not miss this assembly and delay the deadlines. Problems with communication were evident, and later they turned into bugs, about which the tester found out almost on the day the project was completed. In SCRUM, it is initially declared that each member of the team on an equal footing with everyone is voluntarily responsible for the product. After receiving the lesson, the tester began to discuss the situation directly with the client.
We did not fully understand SCRUM and thoughtlessly followed the rule that “tasks are not added to the current sprint”. The same attitude to the bugs appeared: the critical ones were corrected on the next sprint, while the less serious ones were postponed until later, right up to the sprint of fixes at the end of the stage.
Sometimes the customer himself liked to test the product, and the team did not have a hard list of the tasks to be performed in this sprint, except for the list of tasks for the stage. It turned out as follows:
Do not walk our thorny path - just take it as a rule to rigidly determine the tasks for the sprint and eliminate critical bugs in the current sprint.
The result of the work on the scheme : it turned out a quality product, but the production process could not be completely called SCRUM. The customer was often dissatisfied with the fact that he himself found bugs, although the final result suited him completely.
We breathe smoothly, we run further.
Scheme 2. We call it "streaming testing."
Key points:
This scheme is more like a real SCRUM, although there is an additional task in the sprint, whose duration has changed several times.
Advantages of this approach:
If all these conditions are met, the productivity of work grows: every day at about the same time, the tester receives the assemblies and gives a report so that the team corrects critical things if necessary. This allows him to better plan his work time and be sure that he will not be pulled out to another project.
Cons of this approach:
A characteristic feature of this scheme is that if a bug associated with a task is detected, but this bug is not critical and not blocking, the task is still marked as completed. However, in the comments, the bug is supplied with a description (or it is referred to in the bug tracker), and the VP looks at it and finally decides whether to accept or reject this task. Plus, there is a sense of embarrassment in the background that you give the customer a build with bugs.
The result of the work on the scheme: much better than the first scheme, but not without flaws. To smooth them, we have identified some additional rules.
Now you have a second wind - we are entering the home straight.
These techniques in the conduct of projects on the SCRUM-methodology, we brought to ourselves in the process of work and continue to hone them. We hope that they will be useful to you too:
The only tasks that can and should be added are fixing blocking bugs. It is also permissible to add tasks on top of the original ones, but only on condition that the initial tasks are completed, tested, and everything works as it should.
Each assembly must be an increment of the product. If, after adding another feature, everything breaks down, and the solution to the problem turns out to be long and complicated, you can send the VP to one of the previous builds - only one of them will be working.
If you successfully test the build on the Nexus 5, Galaxy S7 and Xperia Z3, then the customer can take your LG G3, on which everything will be bad. The market of mobile devices (especially Android) is diverse, and it is difficult to adjust to the whole, so you need to set boundaries.
Bugs can come out at the last moment, and you doom yourself to work on weekends and carry a bunch of test devices with you wherever you go. This will greatly affect the quality of testing. The end of the sprint is best to install on Tuesday-Wednesday, then the final preparations for the surrender will be held in the normal mode.
We also introduced a small element of kanban inside our team. There are situations when the task list for the test grows, and the developers do not give the build to the tester, because they want to do and give more tasks to the test at a time; or when many tasks are ready for verification, and the tester does not give up the VP assembly, because he wants to give the largest possible part of the tested product. As a result, everyone suffers: a test job has a lot of time — there is little time until the end of the sprint, and there are many tasks to test; developers have gotten off - a little time until the end of the sprint, and the tester has rolled the car of bugs; VP has gotten off time - there is little time until the end of the sprint, but the team is repairing everything and testing, repairing and testing. To avoid this, each team member can not simultaneously be in operation, for example, more than two tasks, and the total number of tasks in the test and ready for testing cannot exceed (also for example) six in each case if there are three developers in the team.
First of all, SCRUM smoothes the problem of missing documentation and establishes team communication: tasks are set gradually and formulated clearly, communication with the customer’s representative is constantly maintained, and it is much easier to understand the project. In parallel with this, test documentation can be compiled.
Testing begins a little later than development, which allows you to be in the context of the project from its very beginning and greatly simplifies the work. All the rapidly changing wishes of customers are promptly discussed with the team and are accepted (or not accepted) collectively, since the VP is in close contact with the team and over time understands what is really done and what is not.
Although following these techniques does not guarantee the successful completion of the project, it will at least help in this. We hope that they will be useful. If you have your own examples of SCRUM implementation, we will be happy to see them in the comments.
Let's start by listing the key features of SCRUM for those who are not familiar with it:
- iterative incremental development. The word “iterative” means that the development is divided into equal periods of time - sprints. One sprint takes from one to four weeks. The word "incremental" implies that as a result of the iteration a new, potentially working product is obtained that solves a business problem. This product is called the product increment;
- self-organizing team, in which there is no project manager (hereinafter - PM);
- there is a SCRUM-master in the team (hereinafter CM);
- in the team there is a person on the part of customers - the product owner, or the owner of the product (hereinafter - VP)
- the entire development period is divided into intervals - sprints. The length of the sprint is set at the beginning of the project and changes only if unaccounted details emerge that interfere with keeping within the specified limits;
- tasks (functional requirements, bugs, edits of the customer, etc.) form a pool of works - backlog. Initially, it only includes customer requirements;
- At the beginning of the sprint (and in any of its places, if necessary), Backlog Grooming is performed - processing tasks from the backlog. The result is a developed backlog for 2-3 future sprints. Then the VP and the SCRUM team formulate the goal of the sprint and the expected result, and the team makes up the backlog of the sprint;
- after planning a sprint, its composition is not changed. If the addition of new tasks still occurs, then old and value-comparable tasks are excluded from the sprint. If these changes lead to a change in the sprint goal, then the sprint is canceled and re-scheduled;
- daily short SCRUM rallies. They make it clear how the process is moving, and the team is aware of whether they are going to the sprint goal or not;
- at the end of the sprint, completed tasks are either confirmed or rejected and returned to the backlog;
- according to the results of the sprint, the team gets the product increment.
Live Tipping's use of the SCRUM methodology in some products subsequently yielded positive results: the team achieved discipline without supervision from the manager; the tasks were assessed more precisely because their evaluation did not occur at the beginning of the project, but in the course of when it was already known about some pitfalls; From the customer’s side there was a fast and high-quality feedback. But along with its successful implementation, the question arose of how to embed an existing testing process into the framework of this methodology. Reflections of the testing department of our company formed the basis of everything that is written below.
')
At the start, attention ...
Problems
The company's priority vector is mobile application development. This often creates specific working conditions: the deadlines are tight, there is no normal documentation (as is the case with startups), and customer requirements change often and dramatically. Because of this, a number of difficulties arise in the organization of testing, the main of which is time. If you compare a large enterprise project and a mobile startup, then in the first case there is a sea of ​​requirements for the project, but it’s simpler to work - there are often approved documentation and the deadlines are not so tight. In mobile development, projects are not always as large-scale, but because of the specifics of testing mobile platforms, special requirements for applications and the frequent lack of technical documentation, the scope of work is comparable, and less time is spent on testing.
Hence the following problems of testing mobile applications:
- TK is not due to its rapid obsolescence. Consequently, it is impossible to immediately create and the current set of test cases.
Changes in the requirements occur "on the fly", so there is no point in making TK - more time will be spent on updating it than on developing a product. There are also problems in communication and awareness within the team: the developer clarified the details of the customer, but did not share with the others, or the tester was not at the rally, where important decisions were made that were not documented due to lack of time. Etc. As a result, the team does not have a clear understanding of the project. Hence the following problem:
- Difficult entry into the project when some of the functions are already implemented.
Often the work on the product is organized in such a way that the assembly is given for testing when the lion's share of functions is ready. If the tester did not follow the development of the product, then before testing he should spend a lot of time on understanding his work.
- If an application interacts with other applications or services, the complexity of its testing increases.
When your application, for example, provides calendar integration, uses self-developed chat rooms, gets data from social networks and shares information through them or interacts with cloud services like Google Drive, the complexity of testing increases exponentially due to various interaction options.
Implementation experience
The main problem with the implementation of testing in SCRUM was the prevailing testing practice at that time. It involved lengthy development stages, then transferring the assembly to tests, receiving bug reports, editing, testing again, editing again, and so on until the functions declared at this stage started to work correctly. The meaning of the SCRUM is in relatively short sprints, at the end of which you have some decorated and working increment of the product. As a result, we have two schemes.
Scheme 1 . Let's call it "Single test + check fixes."
Key points:
- daily meetings with a customer who knows about bugs;
- at the beginning of the sprint, each developer in the team recruits a pool of backlog tasks, including bugs;
- The tester usually turned on somewhere in the middle of the sprint. This was due to the fact that we misunderstood the SCRUM, and as a result, in the course of the whole sprint, the entire team was not always involved;
- found bugs are sent to backlog;
- after the start of the sprint, tasks are no longer added to the backlog;
- the final acceptance by the customer is based on the results of the stage, not the sprint;
- the team was PM and there was no VP and SM.
It turned out that the bugs found in the current sprint were fixed only in the following. But since the customer was aware of the status of the project, and the final acceptance was carried out only at the end of the stage, more time is spent on the sprint for tests and fixes, and everything seems to be normal.
Advantages of this approach:
- no new tasks are added to the sprint;
- the client understands why there are shifts in the timing of completion of the stages;
- The last sprints of the stage are mainly devoted to fixes and testing.
This approach was very similar to that prevailing in the company prior to the transition to SCRUM, which made it possible to become familiar with the methodology relatively painlessly. Also, by the end of the stage, fully tested functions were surrendered, which freed me from the pangs of conscience.
Cons of this approach:
- it turned out ScrumBut - defective SCRUM, especially in terms of communication in a team. The processes were heavily taxed by the PM, the team had no VP, and the team members (including the tester) worked the way they were used to with the manager in the team.
We illustrate this drawback with a case in the company. The tester is used to being able to contact the manager for any explanation. But it is more profitable for the manager to view the project himself and quickly send the assembly to the client, telling about possible shortcomings, than to give it to the tester, who can not miss this assembly and delay the deadlines. Problems with communication were evident, and later they turned into bugs, about which the tester found out almost on the day the project was completed. In SCRUM, it is initially declared that each member of the team on an equal footing with everyone is voluntarily responsible for the product. After receiving the lesson, the tester began to discuss the situation directly with the client.
- Bugs were corrected only on the next sprint, and sometimes later.
We did not fully understand SCRUM and thoughtlessly followed the rule that “tasks are not added to the current sprint”. The same attitude to the bugs appeared: the critical ones were corrected on the next sprint, while the less serious ones were postponed until later, right up to the sprint of fixes at the end of the stage.
Sometimes the customer himself liked to test the product, and the team did not have a hard list of the tasks to be performed in this sprint, except for the list of tasks for the stage. It turned out as follows:
- developers recruit a certain number of tasks;
- completed tasks are passed for testing;
- tester finds bugs;
- developers fix some bugs and take on new tasks, since the remaining bugs have a lower priority, and there will still be a stage of fixes and final testing;
- the customer sees that some of the functions are partially ready, starts testing, finds bugs and gets upset.
Do not walk our thorny path - just take it as a rule to rigidly determine the tasks for the sprint and eliminate critical bugs in the current sprint.
The result of the work on the scheme : it turned out a quality product, but the production process could not be completely called SCRUM. The customer was often dissatisfied with the fact that he himself found bugs, although the final result suited him completely.
We breathe smoothly, we run further.
Scheme 2. We call it "streaming testing."
Key points:
- daily meetings with the VP, which monitors the status of the project;
- the tester receives assemblies with partially ready-made functions every day;
- found bugs are sent to backlog;
- at the beginning of the sprint, the team recruits itself a backlog of tasks from the sprint backlog;
- after the start of the sprint, tasks are not added to it, but blocking bugs are repaired and checked in the same sprint, and their priority is increased;
- the customer accepts the sprint build, and there is a hard task list (Sprint Backlog) that you definitely need to complete in this sprint;
- PM in this scheme becomes SCRUM-master and mainly establishes communication between developers and VP or parts of the team, protects the team from excessive pressure of VP, proposes changes in the work process, etc., that is, plays the role of facilitator in the process;
- assembly gets to the VI only through the tester;
- at the end of the sprint, a retrospective is held and a demo is prepared for the customer’s test group (Stakeholders).
This scheme is more like a real SCRUM, although there is an additional task in the sprint, whose duration has changed several times.
Advantages of this approach:
- daily testing of new features. The number and complexity of the work are increased more smoothly;
- daily short SCRUM rallies. This allows its members to better understand the context of the product, which is especially useful when a team is distributed across cities;
- the fastest possible correction of blocking bugs;
- handing over functions to work units that approximate the achievement of a sprint goal.
If all these conditions are met, the productivity of work grows: every day at about the same time, the tester receives the assemblies and gives a report so that the team corrects critical things if necessary. This allows him to better plan his work time and be sure that he will not be pulled out to another project.
Cons of this approach:
- far from always assemblies come daily, which leads to a sharp carrock at the end of the sprint;
- Adding tasks after a sprint start may result in delays. This happens if at the beginning of the sprint the developer took a good pace, did a lot of tasks and decided to get new ones without finishing all the current ones. In this case, a problem will surely come out that will not allow you to complete all the work on time.
A characteristic feature of this scheme is that if a bug associated with a task is detected, but this bug is not critical and not blocking, the task is still marked as completed. However, in the comments, the bug is supplied with a description (or it is referred to in the bug tracker), and the VP looks at it and finally decides whether to accept or reject this task. Plus, there is a sense of embarrassment in the background that you give the customer a build with bugs.
The result of the work on the scheme: much better than the first scheme, but not without flaws. To smooth them, we have identified some additional rules.
Now you have a second wind - we are entering the home straight.
Bug work
These techniques in the conduct of projects on the SCRUM-methodology, we brought to ourselves in the process of work and continue to hone them. We hope that they will be useful to you too:
- take on a sprint a fixed number of tasks.
The only tasks that can and should be added are fixing blocking bugs. It is also permissible to add tasks on top of the original ones, but only on condition that the initial tasks are completed, tested, and everything works as it should.
- Break large tasks into smaller ones and issue test assemblies as regularly as possible. This way you can avoid idle testers at the beginning of the sprint and crashing at the end.
- To do several versions of assemblies during the sprint. For example, three.
Each assembly must be an increment of the product. If, after adding another feature, everything breaks down, and the solution to the problem turns out to be long and complicated, you can send the VP to one of the previous builds - only one of them will be working.
- For mobile development: define a list of devices that guarantee good product performance.
If you successfully test the build on the Nexus 5, Galaxy S7 and Xperia Z3, then the customer can take your LG G3, on which everything will be bad. The market of mobile devices (especially Android) is diverse, and it is difficult to adjust to the whole, so you need to set boundaries.
- NEVER put an assembly deadline on Friday, Saturday or Sunday.
Bugs can come out at the last moment, and you doom yourself to work on weekends and carry a bunch of test devices with you wherever you go. This will greatly affect the quality of testing. The end of the sprint is best to install on Tuesday-Wednesday, then the final preparations for the surrender will be held in the normal mode.
- Configure the bug tracker so that the generated bugs pass through the tester.
- So the bug reports will become unified, which facilitates understanding, and the tester himself is aware of how things are wrong with the project.
- Critical decisions must be made collectively.
- If someone needs to discuss this or that serious change with the VP, the participation of the whole team is desirable. Firstly, it gives an outside view, secondly, the change can affect the workflow of other team members, and thirdly, the team understands the project more deeply.
- Limit the number of tasks in the work, ready for testing and verification.
We also introduced a small element of kanban inside our team. There are situations when the task list for the test grows, and the developers do not give the build to the tester, because they want to do and give more tasks to the test at a time; or when many tasks are ready for verification, and the tester does not give up the VP assembly, because he wants to give the largest possible part of the tested product. As a result, everyone suffers: a test job has a lot of time — there is little time until the end of the sprint, and there are many tasks to test; developers have gotten off - a little time until the end of the sprint, and the tester has rolled the car of bugs; VP has gotten off time - there is little time until the end of the sprint, but the team is repairing everything and testing, repairing and testing. To avoid this, each team member can not simultaneously be in operation, for example, more than two tasks, and the total number of tasks in the test and ready for testing cannot exceed (also for example) six in each case if there are three developers in the team.
Solvable problems
First of all, SCRUM smoothes the problem of missing documentation and establishes team communication: tasks are set gradually and formulated clearly, communication with the customer’s representative is constantly maintained, and it is much easier to understand the project. In parallel with this, test documentation can be compiled.
Testing begins a little later than development, which allows you to be in the context of the project from its very beginning and greatly simplifies the work. All the rapidly changing wishes of customers are promptly discussed with the team and are accepted (or not accepted) collectively, since the VP is in close contact with the team and over time understands what is really done and what is not.
Conclusion
Although following these techniques does not guarantee the successful completion of the project, it will at least help in this. We hope that they will be useful. If you have your own examples of SCRUM implementation, we will be happy to see them in the comments.
Source: https://habr.com/ru/post/307860/
All Articles