Scrum From chaos to order and high productivity
Millions of articles are written about what Scrum is and what it is eaten with. However, most of them assume that there is a vacuum before the screen, or vice versa a tough environment with processes conducted by PMBOK and others. Many authors write about the “zero sprint” in the beginning of the project, the selection of the ideal team, At the time I did not find a large number of articles about the introduction of Agile methodologies into the existing environment, in which there were no methodologies before, but there were already established traditions.
Two and a half years ago, when the team and I (or rather, it wasn’t there yet) started developing our product, we didn’t think about methodologies, processes and other bureaucratic issues that seemed unnecessary at the time. As time went on, products became more, the team grew. Gradually, everyone began to realize that some kind of chaos was being formed, which was becoming more and more difficult to control, and most importantly, which seriously limited our possibilities. In fact, unnoticed by us, the situation was approaching a critical one.
Under the cut there is a long real story of Scrum's introduction into the development process, which was going through hard times. I hope this story will be interesting to you and, perhaps, will help you to solve or solve some problems.
')
To better assess the situation:
Autumn 2014. The team supports two small systems and is actively developing three others, combined into one complex with a common architecture. In fact, this is not even a team. In the development process, historically formed the people responsible for each system and only those involved in it. One programmer is engaged in supporting two small systems and developing one, two more systems are distributed among two other team members, and finally one front-endender is developing a single interface that all systems can use.
The code base at that time was already hundreds of thousands of lines, there were a minimum number of tests, no automation. In Git-e, two master branches were used in production and dev in development, naturally there were many problems with large merges, etc. Gradually, the speed of introducing new features dropped to almost zero. The management asked to somehow assess the time, but in fact, any assessment was a finger to the sky.
The big problem was also the fact that we could not even somehow balance between projects and priorities. When trying to allocate more people to system A, having suspended, for example, development of system B, we were faced with the fact that the programmer absolutely did not know someone else's code, poorly represented the business tasks of the system and, in fact, retarded the development for a long time.
Why was everything so bad? It's simple. When we started, the main thing was to do everything quickly, and as you know, in the pursuit of speed, quality always suffers. Deep down everyone understood that it was wrong, but it seemed that there was no time to think about it. Yes, we are to blame, but what is, that is.
It was clear that something needs to be changed.
The idea of ​​using Scrum has long been in the air. Its main advantages were exactly as we need: a short release cycle, flexible backlog, fast feedback. However, there were problems. First, we have at least one complex, but different systems, each has its own backlog, its own priorities, and there is no opportunity to provide each system with its own team. Secondly, programmers are a very inert class of employees, they are used to working as before and, although they themselves understood that not everything went smoothly - they were afraid to change something drastically. Thirdly, Scrum is not only project management, it is auto-tests, CI, clean code and much of what we didn’t have at that time.
But the time has come, it was clear that we can no longer delay, and we began to prepare to globally change the way we work.
To begin with, it was necessary to solve the main problem, without which no further steps would be possible.
Knowledge Exchange
For this we have been preparing for a long time. They completed all the major tasks that were started at that time, made descriptions of the business functions of the systems, and each developer prepared a story about thin or non-obvious places in his code.
When everything was ready, we suspended any new developments for two weeks, only fixing bug and only in extreme cases. For two weeks, each team member presented his code, and I told about user needs in the smallest detail.
For two weeks, people studied the code, tried to fix "alien" bugs and asked many questions.
When this was over and everyone at least had some idea of ​​what they were talking about in each project - it was time to change the process.
It should be mentioned here that at the beginning of everything I was both a Scrum master and a Product owner. I perfectly understood that it was not right, but at that moment, it was simply impossible to work otherwise. I am the only one of the team who knew the principles and rules of scram qualitatively, but I also had to present the product.
Process
If up to this point everything went relatively well, then problems started. Programmers generally supported Scrum, but when it came to planning, the objections began like, “Why waste so much time talking?”, “We're no use for a whole day in negotiation,” but of course everyone had the most shock when I gave a hint about Daily Scrum - “It’s terrible to talk every day,” “Everyone can see everything in the grease / beatback”, “If anyone is interested, he will ask.”
It was clear that it was necessary to introduce something gradually, and the point here was not that the process was imposed “from above,” no, people wanted changes, but this was beyond their power.
We started with the introduction of sprints and, yes, planning.
We merged backlogs of all subsystems into one and tried to set priorities. Our first user-stories were old tasks, they could be purely technical, and so on. Evaluated in story points, for a basis they took the scale of powers of two, but of course the initial estimates differed significantly. Our first sprints lasted a week, some didn’t approve of such short sprints, and such frequent “chatter”, but those who supported me understood that even for a week we couldn’t plan everything exactly, we couldn’t even talk about big sprints .
We started our first sprint on Wednesday evening and decided that the retrospective will be in a week on Wednesday morning, so it will take half a day, then planning and a new sprint. Of course, there could be no talk about any demos, we really had nothing to show, and besides this is another point for increasing the emotional load on programmers.
So, on Wednesday morning, having come to the retrospective, we unanimously understood that we have a few more sprint tasks in our work that “we have just a little more to finish” (in fact a little bit could mean a whole day of work). There was another misunderstanding, in retrospect, we had to discuss problems and opportunities to improve our process, but programmers unanimously declared that "there are no problems." They did not feel that they did not have time, but today they are discussing everything, and then they will finish. There was no sense of cyclical process.
For several weeks we continued to work in this mode, I tried to draw attention to the fact that the sprint is an iteration, and not just a segment on the way. In the end, a less-than-good solution was found. We divided the retrospective and planning and smashed them on different days, on Tuesday retro, on Wednesday planning, between these meetings, the programmers solved different RND tasks, those that were still too early to be evaluated, since it was not clear what to do. But most importantly, between the retro and planning it was forbidden to work on the tasks of the last sprint. This, too, did not like at first, because the unfinished code remained and was forgotten, but this measure bore fruit. The team began to understand that the tasks that were not completed in the sprint were not really completed, they could be forgotten, left indefinitely, etc.
In the meantime, it was necessary to pay attention to working with the code. Previously, tests were written only on key functions and, in fact, they covered an extremely small part of the code. From this point on, we decided that every piece of code that affects the problem to be solved should be covered with tests. Here, too, there were moments, for example, when planning the first sprints five, it was often necessary to remind you when evaluating "And with the tests this task will take how many points?" But it was still implemented quite simply, everyone understood too well why this was necessary.
Another big misfortune was the fact that despite two weeks of study, people still didn’t know "alien" projects.
First, the planning task for project A is the person who developed it was rated at 4 points, and the others at 16 and 32. It took a long time to explain that the assessment should take into account that the task may be in a person who used to have this project did not work.
Secondly, the developers began to actively use the fact that Scrum invites the team members to choose their own tasks. Thus, the programmer usually tried to take on the task of his “own” project in order not to bother with someone else's code. For several weeks, we had to keep the rule that you are forbidden to take a task from “your” project, if there are others on the board. There was dissatisfaction, but, I must say, it quickly faded away as soon as people had studied the entire code enough.
But of course the main thing, and certainly the best thing that happened during this transition, is that a few good guys who used to simply work at the same table, finally began to become a team. Before that, they, perhaps, went together for lunch, and even then rarely, and then real mutual assistance, teamwork and responsibility began to arise.
Slowly but surely, we moved in the direction of a correct and good process that helped us work.
After a couple of months, we entered the Daily Scrum (stand-up), appointed it at 12:00, because by that time everyone was usually at work (we have a very flexible schedule). For a long time at the stand-up, everyone just called the tasks they were working on, and it was not very useful, but some adaptation period passed and the team really began to exchange useful and relevant information about the problems and progress.
Sprints became two weeks, we learned to better evaluate the tasks and were able to afford it. Estimates on planning began to coincide in most cases, and when they did not coincide, we could discuss everything and, more often, come to a common opinion.
The percentage of purely technical problems in the backlog gradually began to fall. And increasingly, the task was the User Story.
A weak point in the process was still a retrospective, on most of them the team could not offer topics for discussion. “We're fine” is a fairly common statement. Performing the role of Scrum Master, I read a large amount of materials on this topic and began to use some techniques, however there is still a big difference between theory and practice, for example, the standard method of finding the root problem is very good, but in practice, he showed that We have a problem in the indivisibility of some tasks, about which we already knew, but no cunning method could help us solve this problem. However, over time, the guys began to take the initiative, which of course was extremely useful.
We began to use Git Flow, set up Jenkins to run autotests at every commit, began to measure the code coverage and try to improve it. The number of hotfixes, which at the beginning could occupy half of the sprint, began to decrease. The quality of the product, and most importantly, the mood of our users has improved significantly. We analyzed where we have weak spots in the development process and figured out how to deal with the long Code Review.
More than half a year has passed since we began this epic. Today, we are still developing and we are going to do it always. However, we can already say about the results.
We still have a lot of work ahead. Recently, we finally reached the point that it was time to introduce a demo, let's see what this will lead to. In addition, we understand that we are just far from ideal and can work an order of magnitude faster and try to achieve this. However, the main thing that I can say is that no one from the team regrets that we started this, and despite many difficulties, many of which were possibly related to our inexperience in agile processes, we did reach order in processes and really escaped from the situation in which we were led by the pursuit of speed and fear of bureaucracy.
Thanks for reading, I will be happy to answer any questions, as well as accept any criticism.
Two and a half years ago, when the team and I (or rather, it wasn’t there yet) started developing our product, we didn’t think about methodologies, processes and other bureaucratic issues that seemed unnecessary at the time. As time went on, products became more, the team grew. Gradually, everyone began to realize that some kind of chaos was being formed, which was becoming more and more difficult to control, and most importantly, which seriously limited our possibilities. In fact, unnoticed by us, the situation was approaching a critical one.
Under the cut there is a long real story of Scrum's introduction into the development process, which was going through hard times. I hope this story will be interesting to you and, perhaps, will help you to solve or solve some problems.
')
To better assess the situation:
Little about who we are
We, the technical team of Sputnyx, are developing a set of systems for automating online advertising. At the moment, we fully cover the needs of clients in managing contextual and targeted advertising, and also offer a number of small tools for automating routine tasks.
The team consists of five people (1 front, 3 backups and I am the product manager) + one DevOps engineer who provides us with first-class server setup.
There is also a separate team to develop our RTB platform, but it does not appear in this story.
The team consists of five people (1 front, 3 backups and I am the product manager) + one DevOps engineer who provides us with first-class server setup.
There is also a separate team to develop our RTB platform, but it does not appear in this story.
So, the situation:
Autumn 2014. The team supports two small systems and is actively developing three others, combined into one complex with a common architecture. In fact, this is not even a team. In the development process, historically formed the people responsible for each system and only those involved in it. One programmer is engaged in supporting two small systems and developing one, two more systems are distributed among two other team members, and finally one front-endender is developing a single interface that all systems can use.
The code base at that time was already hundreds of thousands of lines, there were a minimum number of tests, no automation. In Git-e, two master branches were used in production and dev in development, naturally there were many problems with large merges, etc. Gradually, the speed of introducing new features dropped to almost zero. The management asked to somehow assess the time, but in fact, any assessment was a finger to the sky.
The big problem was also the fact that we could not even somehow balance between projects and priorities. When trying to allocate more people to system A, having suspended, for example, development of system B, we were faced with the fact that the programmer absolutely did not know someone else's code, poorly represented the business tasks of the system and, in fact, retarded the development for a long time.
Why was everything so bad? It's simple. When we started, the main thing was to do everything quickly, and as you know, in the pursuit of speed, quality always suffers. Deep down everyone understood that it was wrong, but it seemed that there was no time to think about it. Yes, we are to blame, but what is, that is.
It was clear that something needs to be changed.
Start
The idea of ​​using Scrum has long been in the air. Its main advantages were exactly as we need: a short release cycle, flexible backlog, fast feedback. However, there were problems. First, we have at least one complex, but different systems, each has its own backlog, its own priorities, and there is no opportunity to provide each system with its own team. Secondly, programmers are a very inert class of employees, they are used to working as before and, although they themselves understood that not everything went smoothly - they were afraid to change something drastically. Thirdly, Scrum is not only project management, it is auto-tests, CI, clean code and much of what we didn’t have at that time.
But the time has come, it was clear that we can no longer delay, and we began to prepare to globally change the way we work.
Implementation
To begin with, it was necessary to solve the main problem, without which no further steps would be possible.
Knowledge Exchange
For this we have been preparing for a long time. They completed all the major tasks that were started at that time, made descriptions of the business functions of the systems, and each developer prepared a story about thin or non-obvious places in his code.
When everything was ready, we suspended any new developments for two weeks, only fixing bug and only in extreme cases. For two weeks, each team member presented his code, and I told about user needs in the smallest detail.
For two weeks, people studied the code, tried to fix "alien" bugs and asked many questions.
When this was over and everyone at least had some idea of ​​what they were talking about in each project - it was time to change the process.
It should be mentioned here that at the beginning of everything I was both a Scrum master and a Product owner. I perfectly understood that it was not right, but at that moment, it was simply impossible to work otherwise. I am the only one of the team who knew the principles and rules of scram qualitatively, but I also had to present the product.
Process
If up to this point everything went relatively well, then problems started. Programmers generally supported Scrum, but when it came to planning, the objections began like, “Why waste so much time talking?”, “We're no use for a whole day in negotiation,” but of course everyone had the most shock when I gave a hint about Daily Scrum - “It’s terrible to talk every day,” “Everyone can see everything in the grease / beatback”, “If anyone is interested, he will ask.”
It was clear that it was necessary to introduce something gradually, and the point here was not that the process was imposed “from above,” no, people wanted changes, but this was beyond their power.
We started with the introduction of sprints and, yes, planning.
We merged backlogs of all subsystems into one and tried to set priorities. Our first user-stories were old tasks, they could be purely technical, and so on. Evaluated in story points, for a basis they took the scale of powers of two, but of course the initial estimates differed significantly. Our first sprints lasted a week, some didn’t approve of such short sprints, and such frequent “chatter”, but those who supported me understood that even for a week we couldn’t plan everything exactly, we couldn’t even talk about big sprints .
We started our first sprint on Wednesday evening and decided that the retrospective will be in a week on Wednesday morning, so it will take half a day, then planning and a new sprint. Of course, there could be no talk about any demos, we really had nothing to show, and besides this is another point for increasing the emotional load on programmers.
So, on Wednesday morning, having come to the retrospective, we unanimously understood that we have a few more sprint tasks in our work that “we have just a little more to finish” (in fact a little bit could mean a whole day of work). There was another misunderstanding, in retrospect, we had to discuss problems and opportunities to improve our process, but programmers unanimously declared that "there are no problems." They did not feel that they did not have time, but today they are discussing everything, and then they will finish. There was no sense of cyclical process.
For several weeks we continued to work in this mode, I tried to draw attention to the fact that the sprint is an iteration, and not just a segment on the way. In the end, a less-than-good solution was found. We divided the retrospective and planning and smashed them on different days, on Tuesday retro, on Wednesday planning, between these meetings, the programmers solved different RND tasks, those that were still too early to be evaluated, since it was not clear what to do. But most importantly, between the retro and planning it was forbidden to work on the tasks of the last sprint. This, too, did not like at first, because the unfinished code remained and was forgotten, but this measure bore fruit. The team began to understand that the tasks that were not completed in the sprint were not really completed, they could be forgotten, left indefinitely, etc.
In the meantime, it was necessary to pay attention to working with the code. Previously, tests were written only on key functions and, in fact, they covered an extremely small part of the code. From this point on, we decided that every piece of code that affects the problem to be solved should be covered with tests. Here, too, there were moments, for example, when planning the first sprints five, it was often necessary to remind you when evaluating "And with the tests this task will take how many points?" But it was still implemented quite simply, everyone understood too well why this was necessary.
Another big misfortune was the fact that despite two weeks of study, people still didn’t know "alien" projects.
First, the planning task for project A is the person who developed it was rated at 4 points, and the others at 16 and 32. It took a long time to explain that the assessment should take into account that the task may be in a person who used to have this project did not work.
Secondly, the developers began to actively use the fact that Scrum invites the team members to choose their own tasks. Thus, the programmer usually tried to take on the task of his “own” project in order not to bother with someone else's code. For several weeks, we had to keep the rule that you are forbidden to take a task from “your” project, if there are others on the board. There was dissatisfaction, but, I must say, it quickly faded away as soon as people had studied the entire code enough.
But of course the main thing, and certainly the best thing that happened during this transition, is that a few good guys who used to simply work at the same table, finally began to become a team. Before that, they, perhaps, went together for lunch, and even then rarely, and then real mutual assistance, teamwork and responsibility began to arise.
Development
Slowly but surely, we moved in the direction of a correct and good process that helped us work.
After a couple of months, we entered the Daily Scrum (stand-up), appointed it at 12:00, because by that time everyone was usually at work (we have a very flexible schedule). For a long time at the stand-up, everyone just called the tasks they were working on, and it was not very useful, but some adaptation period passed and the team really began to exchange useful and relevant information about the problems and progress.
Sprints became two weeks, we learned to better evaluate the tasks and were able to afford it. Estimates on planning began to coincide in most cases, and when they did not coincide, we could discuss everything and, more often, come to a common opinion.
The percentage of purely technical problems in the backlog gradually began to fall. And increasingly, the task was the User Story.
A weak point in the process was still a retrospective, on most of them the team could not offer topics for discussion. “We're fine” is a fairly common statement. Performing the role of Scrum Master, I read a large amount of materials on this topic and began to use some techniques, however there is still a big difference between theory and practice, for example, the standard method of finding the root problem is very good, but in practice, he showed that We have a problem in the indivisibility of some tasks, about which we already knew, but no cunning method could help us solve this problem. However, over time, the guys began to take the initiative, which of course was extremely useful.
We began to use Git Flow, set up Jenkins to run autotests at every commit, began to measure the code coverage and try to improve it. The number of hotfixes, which at the beginning could occupy half of the sprint, began to decrease. The quality of the product, and most importantly, the mood of our users has improved significantly. We analyzed where we have weak spots in the development process and figured out how to deal with the long Code Review.
Our days
More than half a year has passed since we began this epic. Today, we are still developing and we are going to do it always. However, we can already say about the results.
- We work as a team on all tasks, which means that if someone gets sick, not a single project will be left without support.
- The speed of development in the number of embedded features over the period of time has increased several times when compared with last fall.
- The number of bugs and hotfixes, at the moment we have from zero to one per sprint, which suggests that the quality of the system has increased by an order of magnitude.
- During these six months, we raked backlog from the accumulated tasks, removed the superfluous, and now this is a full-fledged dynamically changing list of features that we will actually implement in the next quarter.
- In addition, thanks to realistic estimates, we, knowing the speed of the team, can build more long-term plans and calculate what we will do in the next six months.
We still have a lot of work ahead. Recently, we finally reached the point that it was time to introduce a demo, let's see what this will lead to. In addition, we understand that we are just far from ideal and can work an order of magnitude faster and try to achieve this. However, the main thing that I can say is that no one from the team regrets that we started this, and despite many difficulties, many of which were possibly related to our inexperience in agile processes, we did reach order in processes and really escaped from the situation in which we were led by the pursuit of speed and fear of bureaucracy.
Thanks for reading, I will be happy to answer any questions, as well as accept any criticism.
Source: https://habr.com/ru/post/295062/
All Articles