Our development process: 50 months of evolution
Our company is already 6 years old. It was based on the principles of agile and grew on them. We used Extreme Programming from day one, added a little Scrum later and eventually switched to Kanban . I would like to share invaluable experience and talk about the changes in our development process over the past 4 years.
')
The context is very important because it helps to understand what can work for you and what doesn't. We create a TargetProcess , quite a large web application. Our team is located on the same floor in the same office building - there is no distribution and no close.
One team → mini-teams. With the growth of the company, one team was replaced by several mini-teams. The mini-team is fully responsible for the design and implementation of a large feature, and usually consists of a designer, 1-3 programmers, Product Owner, 1-2 testers and a writer. The mini-team takes all the technical decisions, brainstorms ideas and finds solutions to the problem. She is very autonomous and focused. At the moment we have 6 mini-teams.
Technology . The server part is fairly stable. This is all the same .NET stack. Although during this time we migrated from .NET 2.0 to .NET 4. We added ESB for plug-ins last year, which improved the extensibility and scalability of the system. We are less and less satisfied with how NHibernate works, and we are moving towards CQRS . Front-end can not be called stable. The current UI is a rather sad cocktail from ASP.NET Ajax, ExtJS and our own framework. The current efforts are aimed at unifying all this happiness, so that in the very near future the old UI will be completely replaced by a new one. What is nice, the system can be used both in the old and in the new interface, and this should seriously simplify the migration of existing users. This is how our focus on technology has changed over the years:
Iteration → Kanban. We used iterations for 3 years, and then switched to Kanban. Overall, this was a good event. The main goal is the ability to release each feature and each fix separately and as quickly as possible. Many periodical meetings were abolished and replaced with on-demand meetings. Over time, we refused to record time and work evaluation. One negative effect of the rejection of iterations was the desire of features becomes large. When there are no iterations, there is no special need to break the work into smaller ones. We struggled with this for 3 years with varying success. In general, while the big features are winning, but this year the situation has become better.
Points → Waiver of ratings. In iterative development, estimates are necessary. When we switched to Kanban, it became clear that there was no benefit from the ratings. Now we estimate the work at the level of “a couple of days”, “approximately a week”, “not more than a month”, “very long time”. Sometimes the Product Owner misses the estimates, but when we started to do a Roadmap, such general estimates of large features turned out to be fairly accurate.
Time tracking → Waiver of accounting . We kept records of the time spent from the first day of the project’s life, even when there were only four of us working in the evenings and weekends. So the time record was in our blood, and even the salary was tied to the number of hours spent. The decision to cancel time tracking was not easy, many in the team were against it. Surprisingly, after a couple of months everything was settled and the overwhelming majority of people with a shudder recalled filling out the timeshare for the day. In fact, there is no point in taking time into account if you do not evaluate the work.
Release Planning → Nothing → Roadmaps. Such wavering from side to side may look strange (from the side). Initially, we had formal planning meetings for release releases. And then we introduced Kanban and the congregation somehow died. It turned out that without release planning it became worse. We constantly lost focus. A clear focus release is a very good and useful thing. You focus on important things and ignore the rest. If there is no focus, then there is a huge temptation to make many small and not very features that are loosely coupled to each other. This blurs effort and, ultimately, just a list of features is released. Not so long ago, we decided to create roadmaps instead of planning releases. In development we can have 3-4 big features. Each feature has a specific goal and takes from 3 to 12 months. Roadmap shows all current features and some future ones, that's all.
Crushing User Stories . It's funny, but we still can't do it well. To crush work is difficult. Yes, we have certain improvements, but from time to time, large User Stories still seep into the development without any crushing. It seems there is some kind of internal complexity in crushing.
Definition of Done . The trend is clear - DoD from year to year becomes more saturated and includes more and more things. The process is developing and I want to be sure that every feature is ready for release completely. We recently added text verification and performance tests to DoD.
Daily meetings. The old-timer of our review endured 4 years with almost no changes. With an increase in the number of people, we tried in every way to keep the meeting short and specific. In general, it was possible - for 15 people, the meeting lasts about 15 minutes.
Meetings. Now we have survived only really useful meetings that are focused to solve specific problems. The only regular meeting is the daily morning. Scrum has a lot of all kinds of meetings. This may be good when you are just starting to implement a normal development process, but over time they become boring and boring. Meeting-on-demand is much better: fewer people, clearer focus, better result.
User Experience. It was cool. 4 years ago we never heard anything about the UX . Currently, UX is practically embedded in the company's DNA. Many people in the team pay great attention to issues of usability, design and product perception by the end user. The UX planting process was quite lengthy and began with a change in product vision. We trained people, sent them to conferences, conducted seminars, read books and tried very different practices. Some good results appeared only after 2 years. And now we are working on a new big release and the intermediate results are insanely cool (until we show, we create an intrigue).
Craftsmanship. Training and professional development was not part of the company's culture 4 years ago. It all started in 2010, when the whole development strategy was completely revised and changed. Now we have a total spirit of knowledge . Seriously.
This is how the focus of the company's development changed over the years:
One brunch → Branches by features → One brunch. Curious loop. Branch fits became possible after the introduction of Kanban. We used them for 2 years. However, the desire for continuous delivery requires a return of one brunch, which we did a few months ago. The transition was not an easy one, it was much easier and safer to maintain brunches by feature, so 3-4 weeks of the trunk were red 80% of the time.
Pair programming. This practice has gone from mandatory to completely optional. Mandatory pair programming is a pretty tough thing. You have to work in a pair all day long, communicate, argue, defend your point of view. It sucks energy. Yes, you feel that you have done a lot, but it’s not easy to work at least for several months. Now we use pair programming mainly for very complex tasks.
TDD → ​​BDD. Development through testing was applied from the very first line of the code (well, almost) and was used all the time. BDD is more focused on end cases and more understandable for Product Owner. He now writes specifications in the Given / When / Then format playfully. In general, we are moving to visual specifications, where there are a lot of pictures, but few words.
And some statistics for the current moment:
That's what we have come for the last 4 years
PS If someone knows how to make Habr format the table normally - let me know.
')
Context
The context is very important because it helps to understand what can work for you and what doesn't. We create a TargetProcess , quite a large web application. Our team is located on the same floor in the same office building - there is no distribution and no close.
| 2008 | 2009 | 2011 | 2012 |
| Company size (in people) | |||
| 15 | 22 | thirty | 40 |
| Team structure | |||
| One cross-functional team | Two teams: Kernel (5 developers, Scrum Master, 3 testers) Integration (3 developers, Team Lead, tester) | Several mini-teams. We form mini-teams for each big feature. The mini-team is fully responsible for the implementation of this feature. | Several (but more) mini teams |
| Technology | |||
| C #, ASP.NET, ASP.NET Ajax, NHibernate | C #, ASP.NET, ExtJS , NHibernate | C #, jQuery , NHibernate, NServiceBus | C #, NHibernate, NServiceBus, 100% Javascript front-end, REST , Python |
Observations
One team → mini-teams. With the growth of the company, one team was replaced by several mini-teams. The mini-team is fully responsible for the design and implementation of a large feature, and usually consists of a designer, 1-3 programmers, Product Owner, 1-2 testers and a writer. The mini-team takes all the technical decisions, brainstorms ideas and finds solutions to the problem. She is very autonomous and focused. At the moment we have 6 mini-teams.
Technology . The server part is fairly stable. This is all the same .NET stack. Although during this time we migrated from .NET 2.0 to .NET 4. We added ESB for plug-ins last year, which improved the extensibility and scalability of the system. We are less and less satisfied with how NHibernate works, and we are moving towards CQRS . Front-end can not be called stable. The current UI is a rather sad cocktail from ASP.NET Ajax, ExtJS and our own framework. The current efforts are aimed at unifying all this happiness, so that in the very near future the old UI will be completely replaced by a new one. What is nice, the system can be used both in the old and in the new interface, and this should seriously simplify the migration of existing users. This is how our focus on technology has changed over the years:
Process
| 2008 | 2009 | 2011 | 2012 |
| Iterations | |||
| Week 1 | and there are no iterations, we switched to Kanban | ||
| Progress Monitoring Tools | |||
| Task Board , Iteration burn down, release burn down | Kanban Board , Cycle time | Kanban Board, Cycle time, Builds board | Kanban Board, Team Board, Cycle time, Builds board, roadmap in the kitchen |
| 2008 | 2009 | 2011 | 2012 |
| Retrospectives | |||
| every 2 weeks | We make meetings when required, there are no periods. Invented Issue Board with a limit of 3 problems. As soon as 3 problems accumulate, we make a meeting and solve them. Anyone can write problems. | We only have stop-the-line meetings with the people concerned. It does not happen often. | |
| Performance evaluation | |||
| Planning poker . Rate User Story in points | In general, we do not evaluate the work | ||
| Accounting time spent | |||
| Clear accounting for all tasks and bugs. | We do not keep records. I'm tired. | ||
| 2008 | 2009 | 2011 | 2012 |
| Discussion of requirements | |||
| Iteration Planning Meeting | "Start meeting" for User Story. Each User Story is discussed before it starts. At the meeting there is a Product Owner + Tester + Developers + Designer. | ||
| WIP limits | |||
| Iterations | Limit in 3 User Story. More in the progress of work should not be. | Scored on limits | 2 user stories per developer (recommended 1). In general, the mini-team decides for itself. |
| Release Planning | |||
| Formal collection. The release usually lasts for 2 months. | There is no planning | Create a high-level roadmap for the next few months. Revised every 3 months. | |
| 2008 | 2009 | 2011 | 2012 |
| Crushing User Stories | |||
| User Story should get into the weekly iteration | We are trying to crush, but somehow it does not work out very well. | Still not very ... | A bit better, but still difficult practice for us |
| Definition of Done ( DoD ) for User Story | |||
| Specification is relevant A set of passed test cases Unit tests | Starting meeting Specification is relevant A set of passed test cases User Documentation Demo team | Starting meeting Specification is relevant A set of passed test cases A set of automatic cases User Documentation Demo team | Starting meeting Specification is relevant A set of passed test cases A set of automatic cases User Documentation Demo team Green build Human texts Performance tests |
| 2008 | 2009 | 2011 | 2012 |
| Meetings | |||
| Release Planning (Team) Iteration planning (team) Demo Iteration (Team) Retrospective (team) Daily (team) | Launch User Story (3-4 people) User Story demo (4+ people) Retrospective (team) Daily | Launch User Story User story demo Stop-the-line (command) Daily | Launch User Story User Story demo (before testing!) Stop-the-line (right people) Daily |
| Daily meeting | |||
| Yes, at 10 am, takes about 10 minutes | Yes, at 11 am, takes about 15 minutes | ||
| Ux | |||
| What? | Wireframes | Sketches (many ideas and fast), Living prototypes, usability tests on living people , design studio | Sketches, Live Prototypes, usability tests on the working system |
| Craftsmanship | |||
| I'm sorry, what? | Seminars | Salary depends on training Mini-conferences Paid attendance of a cool conference Seminars Friday shows 5 hours of self-education per week | Salary depends on training Mini-conferences Paid attendance of a cool conference 5 hours for self-education or personal project per week |
Observation
Iteration → Kanban. We used iterations for 3 years, and then switched to Kanban. Overall, this was a good event. The main goal is the ability to release each feature and each fix separately and as quickly as possible. Many periodical meetings were abolished and replaced with on-demand meetings. Over time, we refused to record time and work evaluation. One negative effect of the rejection of iterations was the desire of features becomes large. When there are no iterations, there is no special need to break the work into smaller ones. We struggled with this for 3 years with varying success. In general, while the big features are winning, but this year the situation has become better.
Points → Waiver of ratings. In iterative development, estimates are necessary. When we switched to Kanban, it became clear that there was no benefit from the ratings. Now we estimate the work at the level of “a couple of days”, “approximately a week”, “not more than a month”, “very long time”. Sometimes the Product Owner misses the estimates, but when we started to do a Roadmap, such general estimates of large features turned out to be fairly accurate.
Time tracking → Waiver of accounting . We kept records of the time spent from the first day of the project’s life, even when there were only four of us working in the evenings and weekends. So the time record was in our blood, and even the salary was tied to the number of hours spent. The decision to cancel time tracking was not easy, many in the team were against it. Surprisingly, after a couple of months everything was settled and the overwhelming majority of people with a shudder recalled filling out the timeshare for the day. In fact, there is no point in taking time into account if you do not evaluate the work.
Release Planning → Nothing → Roadmaps. Such wavering from side to side may look strange (from the side). Initially, we had formal planning meetings for release releases. And then we introduced Kanban and the congregation somehow died. It turned out that without release planning it became worse. We constantly lost focus. A clear focus release is a very good and useful thing. You focus on important things and ignore the rest. If there is no focus, then there is a huge temptation to make many small and not very features that are loosely coupled to each other. This blurs effort and, ultimately, just a list of features is released. Not so long ago, we decided to create roadmaps instead of planning releases. In development we can have 3-4 big features. Each feature has a specific goal and takes from 3 to 12 months. Roadmap shows all current features and some future ones, that's all.
Crushing User Stories . It's funny, but we still can't do it well. To crush work is difficult. Yes, we have certain improvements, but from time to time, large User Stories still seep into the development without any crushing. It seems there is some kind of internal complexity in crushing.
Definition of Done . The trend is clear - DoD from year to year becomes more saturated and includes more and more things. The process is developing and I want to be sure that every feature is ready for release completely. We recently added text verification and performance tests to DoD.
Daily meetings. The old-timer of our review endured 4 years with almost no changes. With an increase in the number of people, we tried in every way to keep the meeting short and specific. In general, it was possible - for 15 people, the meeting lasts about 15 minutes.
Meetings. Now we have survived only really useful meetings that are focused to solve specific problems. The only regular meeting is the daily morning. Scrum has a lot of all kinds of meetings. This may be good when you are just starting to implement a normal development process, but over time they become boring and boring. Meeting-on-demand is much better: fewer people, clearer focus, better result.
User Experience. It was cool. 4 years ago we never heard anything about the UX . Currently, UX is practically embedded in the company's DNA. Many people in the team pay great attention to issues of usability, design and product perception by the end user. The UX planting process was quite lengthy and began with a change in product vision. We trained people, sent them to conferences, conducted seminars, read books and tried very different practices. Some good results appeared only after 2 years. And now we are working on a new big release and the intermediate results are insanely cool (until we show, we create an intrigue).
Craftsmanship. Training and professional development was not part of the company's culture 4 years ago. It all started in 2010, when the whole development strategy was completely revised and changed. Now we have a total spirit of knowledge . Seriously.
This is how the focus of the company's development changed over the years:
Technical Practices
| 2008 | 2009 | 2011 | 2012 |
| Source control and brunches | |||
| Subversion. One Brunch | Git Each feature or bug fix is ​​made in a separate brunch. The wizard is always ready for release. | Git Returned to one brunch, to move to continuous delivery | |
| Pair programming | |||
| Pair programming is strictly required. Developers change a couple every day to look at the problem with a fresh look, share experiences and achieve real collective ownership of the code. | Required for all user stories and complex bugs. Sometimes people work alone on simpler tasks. Pair change canceled. It turned out that it interferes with focusing and annoying | Used only for complex tasks. | Fully optional. The mini-team decides for herself how to work |
| TDD / BDD | |||
| TDD is highly recommended. Unit tests are strictly required for each user story. | The whole new code is covered in tests. On the JavaScript side, the situation is worse. Also trying to use bdd | On the JavaScript side, everything is fine too, we use QUnit . BDD is growing and growing | Clear focus on BDD. We reworked NBehave a bit for our own needs and did a VS add-in for quick writing test cases |
| Automatic Functional Tests | |||
| Several tests on Selenium | Many tests on Selenium. We made our own C # framework to write and maintain tests. | Functional tests run in parallel on 20 virtual servers | Migrating to Selenium WebDriver . Almost 40 virtual servers |
| Continuous integration | |||
| Very simple. Source code is compiled, unit tests run in Cruise Control . | Use Cruise Control with several different setups. | Switched to Jenkins | Still using Jenkins. New goal - to switch to Continuous Delivery with time. |
| Feature toggling | |||
| Nope | We use feature toggling and move to continuous delivery. | ||
| Test coverage | |||
| God knows | 50% coverage with unit tests and 30% coverage with functional tests | 70% coverage unit tests and 40% functional coverage | |
Observations
One brunch → Branches by features → One brunch. Curious loop. Branch fits became possible after the introduction of Kanban. We used them for 2 years. However, the desire for continuous delivery requires a return of one brunch, which we did a few months ago. The transition was not an easy one, it was much easier and safer to maintain brunches by feature, so 3-4 weeks of the trunk were red 80% of the time.
Pair programming. This practice has gone from mandatory to completely optional. Mandatory pair programming is a pretty tough thing. You have to work in a pair all day long, communicate, argue, defend your point of view. It sucks energy. Yes, you feel that you have done a lot, but it’s not easy to work at least for several months. Now we use pair programming mainly for very complex tasks.
TDD → ​​BDD. Development through testing was applied from the very first line of the code (well, almost) and was used all the time. BDD is more focused on end cases and more understandable for Product Owner. He now writes specifications in the Given / When / Then format playfully. In general, we are moving to visual specifications, where there are a lot of pictures, but few words.
And some statistics for the current moment:
- 5200 unit tests
- 2500 functional tests (1700 Selenium and 800 JavaScript tests)
- All tests run on 38 virtual machines.
- Short build (without installation package) takes 40 minutes
- Full assembly takes 1 hour
- 8400 commits to the git repository in 2011
Finally
That's what we have come for the last 4 years
PS If someone knows how to make Habr format the table normally - let me know.
Source: https://habr.com/ru/post/144051/
All Articles