This post is based on the master class of Boris Wolfson (HeadHunter Development Director), dedicated (surprise!) To the basics of Agile. The material will be useful to anyone who is not at all familiar with this methodology for developing complex software, or has a vague idea about it.

Waterfall model

Dr. Royce created the so-called waterfall model of software development. She quickly gained popularity in the West, and some time ago the vast majority of development companies worked on this model. What is she like? Product development goes through a series of steps:

• collection of requirements;
• their analysis;
• creation of architecture;
• creation of system design;
• coding;
• testing;
• display;
• exploitation.

In an ideal world, we would go down through these levels as a waterfall flows, and in the end we would have a good system. What was Dr. Royce's decision? He suggested writing a lot of documentation, handling risks, repeating some stages several times. The result was a heavy waterfall model. The entire industry of commercial software was formed in the 1990s. And if in Europe and the United States there were heavy methodologies, then we did not have any. There was a group of people who were just trying to make good software. Both problems - the lack of methodology and heavyweight schemes in the West - are well solved by Agile.
')

What is agile development methodology for?

So what is the Agile methodology used for?

• Accelerate product launch . If you want to do something faster, you need to do it in accordance with Agile. Very simple example. There are two companies, they have approximately the same business. One writes TK, then designs the system and draws the design - this is a waterfall model, which can take several months to develop. In the second company working on Agile, by this time the site may already be launched, software released, it will start making money and seize the market, which is the most important thing.
• Managing changes in priorities . This is perhaps a very painful problem for almost all companies. If you are doing a project that lasts at least a few months, then your requirements will surely change. Of course, if this is not software, for example, for a satellite or a rover. Although even satellites and rovers are usually flooded with the latest version of the software when they arrive at their destination. If we talk about commercial development, the problem is that we, programmers, analysts and designers, never know what not only a customer who pays for us needs, but also users. Usually, everyone approaches the question like this: until the user tries the functionality of the site or application, you do not know if it is needed or not.
• Improving the interaction between IT and business . This is a headache, especially for large companies, because the business periodically changes the requirements, each speak their own language. As a result, the parties do not understand each other.

The answer to all these challenges was the Manifesto of flexible software development.

Flexible development manifest

It consists of several parts. The first part is called Values. These are four "weightings":

• If you want to build a flexible process, you need to interact and communicate with each other. What is it expressed - consider below on the example of Scrum. In this case, you can (and definitely will) use some tools and processes, for example, trackers - JIRA, Redmine, etc. But your work should rely on various meetings, meetings and interaction, and not on the settings of trackers or TFS (if we talk about the Microsoft stack).
• The working product that we make is much more important than the documentation on it. Above was given an example with two companies: one has a finished product that can be given to users, the customer, by capturing the market; and the second writes TZ, draws layouts, etc. All this documentation, which the user cannot apply due to the product’s non-availability, does not bring value to this user. If we learn to work by minimizing these steps, or making them in small pieces, then we will have a more flexible process.
• Cooperation and interaction with the customer is more important than rigid contractual restrictions. Usually a contract is signed in which it is specified that by a specific date for a certain amount the developer undertakes to fulfill the agreed scope of work. Naturally, the terms of reference are applied to the contract. That is, fixed time, scope of work and deadlines. This is called Fixed Price. This approach is not very good if you want to work for the long term and be flexible. In this case, it is more correct to build partnerships with the customer. If we talk about contract clearance, then this usually results in contracts using the “time - materials” scheme, when the developer is simply paid for the time spent. The most important thing is that the search for partnership and the Win-Win situation begins here, when both the customer and his contractor win.
• Willingness to change in weighing with following the original plan.

Agile has a plan, estimates and projections. But if you have some initial plan for the annual project, and after three months you have already provided some version of the product, users felt it, you removed the metrics, looked at what and how they use, learned something new, then after that, the original plan can be almost completely changed.

I will give a fresh example. We rolled out a small feature on HeadHunter, when anyone can confirm your skills - he gave you a plus sign, and an inscription appears that so many people confirmed your skill. I have Scrum confirmed by 18 people. We specifically launched it in a very simple way to see how users would react to this. We proceeded from the logic that there would be interaction a la LinkedIn or Habr, where users of each other are plus. They took the statistics, and it turned out that we put these pluses, probably, after HR interviews. That is, we can further develop this feature in the direction of HR, or modify it somehow, so that it is more useful for applicants. Thus, in Agile there are four values:

• people and interaction between them;
• work product;
• cooperation and building partnerships with the customer;
• willingness to change.

12 principles of Agile

Values ​​entail 12 Agile principles:

1. The highest value is customer satisfaction thanks to the regular and early delivery of valuable software. If a customer wants to get a big elephant from us, but we can give him a part of this elephant not in a year, but in three months, then in another three months another part, and then monthly produce pieces, then the more often we will do it and what the sooner the better.
2. We are always ready to change the requirements, even in the later stages of the project, if we learn something new. Thus, we create a competitive advantage for a business or external customer. Suppose two companies are working: one wrote a TK and in one year made a product, and we made the concept of a product (no matter in what form) and gradually roll it out and roll it out. Then our product will more meet the requirements of customers, users and the market as a whole.
3. When using Agile, the working product is released as often as possible. The manifesto contains terms from a couple of weeks to a couple of months. This is actually a week / month if you use Scrum. In Russia, most often something new is released every two weeks. And if they make a web project, they usually use one of the variations of Kanban, which means that releases can be done every day. HeadHunter usually releases several releases every day, which creates major problems for our competitors. These can be bug fixes, but we can add something valuable to our users.
4. Business must work together with programmers to help them understand the specifics of this market. For example, HeadHunter programmers need to understand how HR works, and how job seekers look for jobs. If you work in a bank, you need an understanding of the principle of the bank as a whole and very detailed information about the area for which you are responsible. The most frequent problem, in my experience, is the inaccessibility of the business - when the developer cannot obtain the necessary information from the employee. When using Agile, it is important to avoid such situations.
5. The team is one of the cornerstones of Agile. The best results are achieved by a team of motivated professionals. There is a brilliant remark that the effectiveness of Scrum depends on the leader. In Agile, a manager must first of all create conditions for the team and provide comprehensive support, conduct coaching, monitor the atmosphere in the team.
6. There are many studies that show that the best communication is face to face. Moreover, it is desirable that there is some kind of visualization tool on which you can write: a sheet of paper, a board with stickers, etc. The easiest and most effective way to find out the requirements of a client, customer or user is to talk with them.
7. A working product. I will not repeat about him. The degree of readiness of the project should not be measured in words, that the TK has already been written and 50% of the layouts are drawn, but by the amount of functionality released in production.
8. In Agile, rhythm, constant improvement is important. Business and programmers should always be able to make the process sustainable, constantly improve it.
9. About this point, managers usually do not like to talk to developers, but Agile will not work at all if you have written cattle code. You should have a good flexible architecture, in which you can add different elements and change them if necessary. And if the team does not pay maximum attention to technical quality (write good code, use engineering practices, automate processes), then you will not have any Agile.
10. Simplicity. It is manifested in the technical component in the design. This is one of the principles of extreme programming. Simplicity is also very important in terms of product release: when you want to chop up that elephant, it’s better to start with the simple part.
11. The manager (manager, Scrum-coach, Agile-coach) in the team changes its role: he is not so much involved in the organization of the process as the team teaches, so the team must be self-organized. There are special strategies for making a self-organized team out of a group of people.
12. The team should constantly analyze their work, processes: what happened, how they achieved it, and constantly improve the organization of work.

As a result, we can say that Agile is a series of approaches to the development of software products by continuously and quickly delivering valuable functionality to a self-organized team of professionals in collaboration with the customer. This is not a canonical definition, but my own understanding of Agile.

So, we have a pyramid consisting of four values, on which 12 principles are built. Now there are specific practices.

Practices

One of the values ​​of Agile says: we must build a working process, establishing interaction and communication between people. This translates into practical steps. For example, in the morning stand-ups, when the team verbally synchronizes its activities for the upcoming day. It is possible, instead of stand-ups, to each write a report on the work done yesterday, but this will no longer be Agile, because a stream of irrelevant documentation occurs. What practices are most often used in companies practicing agile development?

1. In the first place standdapy . In Russia, even if a company has completely started implementing, using and adapting Agile, standadapes usually still remain. If you cannot use them, it means that you have a completely unorganized team. The practice is simple: every day at a certain time, the team gathers and synchronizes its activities.
2. In the second place is the planning of iterations , when the team plans the amount of work for the next iteration. This is to the question that in Agile there is no planning. If we have an iteration of two weeks, then we are trying to make a plan, decomposition, maybe, break the business tasks into technical tasks.
3. Unit testing is one of the simplest engineering practices that can be quickly started to use. If there are problems with quality, about 60-70% of the bugs can be caught by unit-testing.
4. Release Planning . Here we are already planning in large chunks - what and when we will release. If we are talking about Scrum, then we measure with large strokes, in sprints.

Let's see how you can graphically present the iterative performance of work.



We need to get from point A to point B. “Reach” means “get feedback”. Suppose I have a GPS, I can get to a point and check if I’ve got it right. The waterfall model is, in fact, one big step. That is, I came somewhere, launched a product on the market, and only after that I can understand whether I did. An iterative model is a series of steps. I take the first step, remove the metrics that I use in the product, then adjust the next steps. May I not come to point B, but I will be in its vicinity.

With commercial development, conditions are constantly changing: new laws come out, business requirements change, competitors suddenly release something that we must do better than they do. The result is that we should go from point A not to point B, but to point B. The iterative model implies that after the release of the first release we remove the metrics, rework something, make the next release, etc. And at some stage we understand that a competitor has released some feature, and we need to go in the wrong direction. At this moment, we can stop, accept other requirements and start moving to point B. With high variability of conditions in the process of creating a product, you will always learn something new. And this is one of the advantages of an iterative model.

Another important point: when managers (and even developers) are not very deeply immersed in the technical part, it seems to them that it is possible to make a program iteratively, put it into pieces, like a puzzle. It's a delusion. The developer must represent in full and in detail each element of the picture, and begin to take it in five steps. It should be borne in mind that conditions may change. This is called the incremental approach (“increment” - adding something).

To work iteratively and incrementally, you need to act a little differently. The developer should have some kind of concept in his head that he is gradually working on.

If you are working on a "waterfall", then usually you have a fixed project content. You are told: “I want you to do this. I know exactly what I want and my users. Estimate how many people you need for this and how long it will take. ” In Agile, we act differently: we have a team and fixed periods of time (I'm talking about Scrum), for example, two-week iterations. Based on this, we plan the amount of work that we can perform during this time.

If you take the classic approach and ask: “Have you done this project successfully or not? How to understand this? I have to do it on time, without exceeding the budget, and completely make the content. If we look from the Agile position, then our project is the more successful, the more we put values ​​to the customer.

Scrum

Henrik Knberg has such a metaphor - an Agile umbrella. These are the methodologies that are flexible. The largest is Scrum, Extreme Programming (XP), DSDM, Crystal, FDD, Kanban. More than half of Agile companies use Scrum. In the second place is the combination of Scrum and XP, when management practices from Scrum are taken and engineering ones are added. Separately, I note that the combination of Scrum and Kanban is used in approximately 8% of companies. Now it is one of the trends, fashion. The name Scrum came from rugby and is translated as “scramble”. Simply put, when a dispute arises, teams line up, a ball is thrown, and you need to push each other. This product was first applied to product development in the 1980s by two Japanese - Hirotaka Takeuchi and Ikujiro Nonaka. These are good management researchers. In particular, they wrote the document "The New New Product Development Game". Here the use of the word “new” 2 times is not an error. Just the name translates as "New game to develop new products." What did these two Japanese do? They analyzed how different companies create their products. And not even software, but all kinds of equipment and electronics. The authors divided the identified approaches into three types.

• The first type (A): we have a development phase, everything is tough, consistent and good.
• The second type (B): the links intersect, it is possible to get feedback. I programmed something, I program it again, I give it to the tester, he gives his comments, I manage to process them before I finish my work.
• Third type ©: each phase intersects with more than one other phase. I program something else, and my first tasks are tested, laid out in production, metrics are removed from them, I can make changes.

Type C two Japanese compared with the situation in rugby. Why? The meaning of the game is to take the ball and bring it to the line of the opponent's field, overcoming resistance. But in rugby you can not throw the ball forward. When you run and realize that now you will be put to the ground, the ball can be given back to a member of your team. He catches him and runs on. If you can not overcome the resistance, then runs back.

Modern Scrum was created by two IT specialists - Ken Schwaber and Jeff Sutherland. On the official website www.scrumguides.org you can find the official description of Scrum. This is the general scheme of Scrum:



So we want to make some kind of product. It is cut into separate pieces, called backlog of the product. This is a requirement. That is, we do not have a technical assignment or some extensive document that describes everything in advance. Usually, the more important some requirements, the better they are broken and described. This is done by the product owner.

The average team consists of seven people (plus or minus two people). If it is much less, then, most likely, the team lacks some specialists, because it means that the Scrum-team can independently make back-product a finished product with a new functionality. For example, a team may not have a front-end programmer. If you use Scrum and the project involves front-end development, then this specialist should be included in the team.

Scrum components

Roles
A scrum team consists of a development team, a product owner and a scrum master.

• Why do we say Scrum is a flexible Agile framework? Because it directly implements the values ​​and principles described at the beginning of the publication. The team in Scrum should be self-organizing, and the Scrum-master helps it to become such, teaches how to create a product increment — a new version of the software — as quickly and efficiently as possible from the backlog elements. The scrum master is responsible for ensuring that all processes work, so that participants understand why and how this is done.
• The product owner, product manager is the person responsible for maximizing the value of the product, he is responsible for the product (for example, Steve Jobs).
• The development team should consist of motivated professionals who can supply during each sprint, that is, based on the requirements to create a finished product.

Processes

• Daily scrum is a panel meeting where team members tell what has been done, what problems they face, what they plan to do in the near future, so that everyone understands who does what.
• Sprint is an iteration with a fixed term, that is, there must be a rhythm in Scrum.
• Sprint planning. Before the start of the sprint, everyone gathers and determines what needs to be done during the sprint and in what form.
• Sprint review or demonstration: everyone gathers and demonstrates what's new in the product increment, watch, record comments, maybe change the nearest plans.
• Retrospective: everyone gathers and thinks they can improve in the next sprint. For example, there were a lot of bugs, users are unhappy. Let's try in the next sprint to use unit testing. We try, at the next retrospective look, it helps or not, we invent something else.

The sprint backlog is the top of the backlog. The number of elements is usually determined by the speed of the team at the event, which is called “sprint planning”, and is carried out before the start of the sprint stage itself. Sprint lasts 1-4 weeks (usually 2 weeks). The faster and cheaper you can release, the shorter the duration of this stage.

Every day, the team holds a 15-minute stand-up, Scrum-meeting, at which all team members synchronize their actions. Often these meetings are called simply “scraps”.

After the sprint is completed, a demonstration is held - “Review”, the meaning of which is that the team shows the work done to the product owner or someone else. Then a retrospective is held, at which the team discusses what has happened and what has not, the workflow, the atmosphere in the team are analyzed, attempts are made to improve something. After that, the next sprint starts. As a result, the work of the Scrum team can be represented as a chain of multiple sprints.

Artifacts
These can be cards on a board with a brief description of what constitutes a specific functionality. In this case, the content of the card can be a discussion with the owner of the product. Usually this translates into tickets in the tracker that you use - JIRA, Redmine, etc.

• Product backlogs are all tickets, cards or other requirements in the form of backlog items that are in your product.
• Backlog sprint - the most valuable and priority requirements.

Examples of Scrum Practices: Estimates

There will be no canonical / kosher / vanilla Scrum that is described by Schwaber and Sutherland, and about which you can read here: www.scrumguides.org . I'll tell you about the real practices used by the teams. There are heavy, ostensibly exact methods to calculate and do everything “correctly”. But almost all big projects are out of date. And this concerns not only IT. For example, there was a case when the airport could not be put into operation due to the fact that the software responsible for the automatic distribution of baggage was not ready. If you use flexible methodologies, as well as what I’ll tell you now, you can safely launch projects without heroism. One of our big projects, which ended in September, was actually ready for production two weeks before the official launch. I watched a lot as the team, team leaders and managers try to predict when they will have a project ready. This is about the same as approaching a team and asking for a random number.



Agile and Scrum suggest using this practice: make relative assessments, that is, “measure in parrots”. This means that I estimate the time it takes for me to complete the task, and how long it will take the parrots. They are usually called story point. These story points should not be tied to any time intervals. How to make such an assessment, why is it called relative? Usually they take some kind of task, small, standard and understandable to everyone, and declare it a measure, a standard - it takes 1 parrot, 1 story point. The next task is taken, compared with the first - it turns out to be 2 times more. How much does it take? 2 story point. And so we decompose all the tasks. As a result, we do not evaluate each task by time, but compare them with each other. If we make mistakes somewhere, then that's fine. The main thing is that in all tasks they are wrong in the same way. Evaluation is done by the team and within the team.

What can we do after this? For example, run a two-week sprint and take the top tasks without any estimates. When the sprint is over, the output will be an increment of the product, which will include some number of tasks. We calculate how many story points we made, and for the next sprint we can simply plan the same number of story points. A similar assessment is obtained relative and empirical. We do not assume, as managers like to do, that programmer Vasya works 8 hours and with equal efficiency, but actually we measure how much the team can live on a story point for a sprint. The rating scale is usually chosen to distinguish between tasks from different classes.



If you look closely, it looks like Fibonacci numbers, or the power of two. In this case, really large tasks are usually evaluated, which are further broken down into smaller ones. For the formation of estimates is usually used poker planning. This is a modification of the Delphi method. Each team member is given a deck of cards with numbers from 0 to 100. There is also a card that says “I don’t understand what this task is” and a coffee card (“Let's take a break, I’m already tortured to do this evaluation”). After that, we take the task number of such and such, we read and discuss what it is: “The user enters the login password in order to log in to the site”. We discuss how this authorization happens, where we store users. Then each team member, face-up, places a map with the grade he considers necessary. At the same time, different members of the team do not influence each other at all, because usually there is a team leader in the team, the lead, the senior developer, which is the majority. After that there is a showdown and discussion.



According to the Delphi method, the discussion takes place between those who set the highest and lowest scores. Those who put the most usually see some risks that other members of the team did not notice. He will say: “You know, in this database, where logins and passwords are stored, we have not entered for a long time. There you need to refactor something, add a column, apply another hash, ”etc. And the person who put the lowest estimate, either does not understand the task, or sees a way to do it faster, or has already done something similar. After this, the second stage of the discussion takes place: again, everyone puts the cards face down, then opens them. Usually estimates are more or less smoothed out. Again, the stage of discussion passes, they come to a consensus. As a result, it is recorded in the tracker, in the ticket system, or directly on the card, that we have this task at 3 story point.

Why is this a very good agile approach? We talked, discussed the content of the task, based our actions on the interaction of people, and not on any formal processes. If someone is interested in process things, refer to the Kokom's evaluation method. What else is good this method? Here we get some team responsibility for the size of the task. Everyone takes responsibility for the fact that the task is really of such a "size". If you measure the complexity of the tasks, say, in days, then there will be situations when someone in the team evaluates it at 8 days and it hits him, then he will do it for 8 days.

What to do if the customer wants to understand how long the project will take and asks for immediate evaluation? The ideal option is to work with the customer using the “time - materials” system. If the customer is new, then you can make one project at Fixed Price, make sure that the customer is adequate, and continue to adhere to the “time - materials” system. If the customer does not agree immediately to this system, then you can correct it: for example, if you finish the project by a certain date, you will receive some kind of bonus. There are also presentations on this topic on the net.

Examples of Scrum Practices: Team Speed

, , story point . , . « ». , .



- , , , , . , . , - , - - , - , - . : « A, B, C, , . D, , , . F, ». , , : « ? ». , , , , .

Scrum-:

, ? . Burn Down Charts.



— , 10 . story point, — . , , — Burn Down Charts. ? — , — story point. .

? Burn Down, , . story point, - 20—25. Excel , — . , Burn Down, . .



, . , Burn Down . , , , , .



, . , , , . — , .

Scrum-:

Scrum , . . , JIRA .. . , , , , - .

Scrum-:

: , , , .. , . . , , , . , , - . , , .

Scrum-:

. - , , — . — , . How does this happen? . , . , .

. 5% . — , , , , . , . — , . - . , , . , , — . : , , , , . . , , , , . , ? , - , , -. 5—10 . . code review, - . - , . . , , .

— . , , , « », « -», « Machine learning». , , , , . , .



, « ». : Plan — Do — Check (Study) — Act.

• (Plan).
• (Do).
• () (Check (Study)).
• (Act).

, . , unit- — , , . ( , Scrum), . , . , , .

: , 50% — , , — 70%. 70%, , — . 90%. : , . . - . — Real-Time Board Service.



, , , . : « . », « », « — », . .

Scrum

, Scrum, , , . , , . : . ? -, , - JIRA. . , , HipChat. — Skype, Hangout. , , .

Kanban

. Scrum Kanban, Scrumban. , . : Kanban . - - . , - , . IT - .

Kanban 2010 ., , . — . , , . Scrum , , , «, , , , », Kanban .

• , , Kanban-. , . . — , . , .
• . - . , . .
• . . , , , unit- 70%, ..
• , , .

Visualization

, Scrum. — -Kanban. . - , - , - , - , - . — WIP (Work in Progress). , . WIP, . , , . , . - , : « ».

, , . — , — . , . .



WIP, — (Definition of Done). : « » «». WIP . , Kanban-. . , , : « ». , WIP.



Scrum, . « ». , — . . , , . . — WIP, . .

— Cycle Time Lead Time. , . , Kanban-.

Lead Time , Kanban- . Cycle Time — Kanban- , . , .

Total

Kanban , - . Scrum , , , .

• «Scrum XP: ». — , Agile- Spotify. . — . — , . , , , Scrum Agile. .
• «Scrum Kanban: ». . , Scrumban.
• , Scrum «Scrum: » . , Scrum, , . .
• Kanban — «Kanban: Successful Evolutionary Change for Your Technology Business». .
• « ». « ».
• «Keystone Habits of Organizational Agility». , Agile.

Video performances

• Speeches at AgileDays and Lean Kanban conferences .
• Reports, experience reports from large companies - Alfa-Bank, Yandex, Raiffeisen, HeadHunter, etc., as well as small start-ups that tell how they used it, what worked, didn't work.