Evidence based planning

Translator's note: the original article was written in 2007, however, in my opinion, it still remains relevant today.

Software developers do not like to plan work. Usually they try to completely give it up “I will finish when I finish!”, They say, expecting that this bold and cheerful act will cause approval from the boss, and planning will be successfully forgotten.

Most of the schedules you meet will be heartless unsubscribes. Completely forgotten, they are stored in some general directory. After the release of the product late for a couple of years, a strange guy in whose office, they say, have seen the card index, will bring to discussion the reasons for the failure of the old printout, which everyone will laugh at. “Just look! We planned two weeks to rewrite the system from scratch to Ruby! ”


Just scream! If you are still afloat.

You want to spend your time on things that will bring maximum returns. And you can not understand what kind of profit you get, not knowing how long the work will take. When you are forced to choose between an animated clip and an increase in the number of financial functions, it does not hurt to know how long it will take to complete each task.
')
Why developers do not formally plan their activities? There are two reasons for this. First, it is a big hemorrhoids. Secondly, no one believes that the plans will correspond to reality. Why waste time scheduling, if they still break with reality.

Over the past year, we at Fog Creek have developed a system with which even the dirtiest programmers have agreed to work. And, as far as I can tell, it leads to very accurate schedules. The system is called "Evidence Planning" or DP . The approach boils down to the fact that, based on the analysis of statistics of completed work, evidence is collected, which is then used to build a plan for the future. As a result, you get not just the release date of the product, but also the confidence curve of the distribution of the probabilities of completing the work at any given time. It looks like this.



The steeper the curve, the more realistic the specific completion date of the project.
Here is how it is done.

1. Separate ...

When I see a schedule, the individual works of which are measured in days or even weeks, I understand that there will be no sense from it. It is necessary to divide the planned activity into very small tasks, measured by hours. Maximum 16 hours.

This will help you understand what you really want to do. Write subroutine foo. Create a dialog box. Parse the Fizzbott file. As a result, the time for a common task will be easy to estimate, because you have already written subroutines, created dialog boxes and performed file parsing.

When you casually create three-week tasks (for example, “Implement an Ajax photo editor”), you have no idea what you really want to do. In detail. Step by step. In the meantime, an exact plan of action has not been thought out, and you cannot know how long it will take.

Setting a maximum task duration to 16 hours, you force yourself to think about this damn functionality. But if you get off with a three-week job called “Ajax Photo Editor”, then I have bad news: from this moment you are officially doomed . Without considering the specific steps of the task, you will inevitably forget about many of them.

2. Consider the time spent

It is difficult to get an accurate estimate. How can you take into account all kinds of incidents that may take you away from work, unpredictable errors, scheduled meetings, and the day of serving Windows service that comes a couple of times a year, when you have to reinstall everything from scratch on your work computer? Well, how, without knowing all this, you can say for sure how long it will take to complete a specific task?

In fact, no way.

Therefore, keep a time sheet. Keep track of how long you work on each task. So that you can then look back and calculate how much time it took compared to the initial estimate. For each developer, you need to collect data like this



Each point on the graph is one completed task with an estimated and real execution time. If we divide the estimate by the real cost, we get the speed : how quickly the task was completed compared to the estimate. Over time, you will gain speed history for each developer.

• The mythical evaluator, which exists only in your imagination, will always accurately predict the execution time. His speed history will look like {1, 1, 1, ...}
• A typical poor appraiser will demonstrate a very large variation in performance, for example {0.1, 0.5, 1.7, 0.2, 1.2, 0.9, 13.0}
• Most experts, of course, will be wrong, but on the whole their assessment will be good. Work always requires more time than planned - it is very difficult to take into account the correction of errors, night meetings, coffee breaks and the madman who constantly distracts you. This typical estimator will show fairly stable speeds, which, however, are less than one. For example, {0.6, 0.5, 0.6, 0.6, 0.5, 0.6, 0.7, 0.6}

As experts gain experience, the quality of their assessments increases. Therefore, it will be good practice to completely get rid of the accumulated statistics of speeds that are, say, six months older.

If, however, a new evaluator comes to your team who does not yet have accumulated data, assume the worst - attribute a false story to it with a large speed variation until he can cope with half a dozen real tasks.

3. Model the future

Do not give in to the initial temptation to simply add up the obtained estimates in order to calculate the exact date of product release. It only sounds right, but in reality is a completely wrong approach. It is necessary to use the Monte Carlo method to simulate a variety of possible outcomes. In the process of modeling, you can create hundreds of possible future scenarios. Each of these outcomes will correspond to one percent probability, as a result you will have the opportunity to build a graph of the probabilities of project completion for each date in the future.

To calculate the likely outcome for a given developer, you need to divide each assessment of the task by a randomly selected rate from the statistics of this developer, collected in step 2. For example



Do this a hundred times, each "Total" corresponds to one percent probability, which will allow you to calculate the probability that the project will be completed on a specific date.

See what happens

• For the case of the mythical ideal appraiser, all speeds are equal to one. Dividing by speed, which is always equal to one, does not change anything. Thus, all rounds of simulation give the same completion date, and this date has one hundred percent probability. Just like in a fairy tale!
• The ratings of a bad expert are very scattered. This can be both 0.1 and 13.0. Each stage of the simulation leads to a very different result from the others. The probability distribution curve will be extremely flat, showing equal chances to finish tomorrow or in the distant future. By the way, this is quite useful information - it tells you that you can not be sure of the designated release date of the product.
• The average typical evaluator will give a very close set of speeds to each other, for example, {0.6, 0.5, 0.6, 0.6, 0.5, 0.6, 0.7, 0.6}. By dividing by these speeds, you will increase the time allotted for performing tasks, for example, an 8-hour task at one stage of modeling can turn into a 13-hour one, while during another it will take 15 hours. This compensates for the ineradicable optimism of evaluators. Moreover, the compensation will be made accurately , based on the optimism that is inherent to this developer, proven and time-tested . And since the real speeds are quite close in values ​​(about 0.6), starting each simulation round, you will get similar numbers, and as a result you will have a fairly narrow range of possible project completion dates.

For each Monte Carlo simulation round, of course, you have to convert hourly information into calendar days, i.e. It is necessary to take into account the work schedule of each programmer, holidays, weekends, etc. You also have to accept the fact that each iteration of the project will be completed only when the most recent developer finishes his work - only this will mean the completion of the work of the whole team. Such calculations need to be done very scrupulously, but, fortunately, scrupulous calculations - this is exactly what computers do well.

Obsessive compulsive disorder not required

What to do with the endless stories of his boss about his fishing trips? Or with sales meetings that force you to attend for no apparent reason? Coffee breaks? Spent half the day to help the newcomer in setting up the workplace?

When Brett and I developed this technique for Fog Creek, we were greatly disturbed by things that were a waste of time, but at the same time could not be predicted in advance. Sometimes they took more time than direct coding. Should this be taken into account in any way and be included in the task schedule?



You can, of course, do just that if you want. And evidence-based planning will continue to work.

But you do not have to .

It turns out that DP works great even in cases when you simply add the lost time to the time of the current task. As strange as it may sound, the use of PD in this case will give even the best results.

Consider a small example. To make it as simple as possible, I’ll come up with a very predictable programmer John, all of whose work boils down to writing single-line methods for accessing fields, which are often necessary when using some programming languages. All he does every day is

private int width; public int getWidth () { return width; } public void setWidth (int _width} { width = _width; } 

I know, I know - this is an incredibly stupid example, but surely you have met programmers who are doing something similar.

Whatever the case, the implementation of each access method takes him two hours. Those. his estimates of the time to complete the tasks are as follows:

{2, 2, 2, 2, 2, 2, 2, 2, 2, 2, ...}

Also this poor guy has a boss who constantly interrupts him with two-hour conversations about catching marlin. John, of course, could add the task “Tedious conversation about marlin” to his schedule, but this would be politically short-sighted. Instead, John simply attributes the lost time to the current task. As a result, its costs are as follows:

{2, 2, 2, 2, 4, 2, 2, 2, 2, 4, 2, ...}

and accordingly speed

{1, 1, 1, 1, 0.5, 1, 1, 1, 1, 0.5, 1, ...}

Now think about what is happening. In a Monte Carlo simulation, the probability that each assessment will be divided by 0.5 exactly corresponds to the probability that John’s work will be interrupted by the boss. Thus, the DP will produce the correct schedule!

In fact, the DP is likely to take into account the interruptions of work much better than the most meticulous developer does. That’s why it works so well . This is how I explain it to people. When a programmer is interrupted, he is faced with a choice.

1. make a big fuss, bringing all the interruptions to his work schedule, so that the management can see how much time he has to spend on fishing or
2. make a lot of noise, refusing to make it to the work schedule, with the result that the current task will “hang” in the air, and all because there is nothing to correct its ideally accurate estimates, only because of a silly conversation about fishing, to which he did not even bother to invite ...

... and in any case, the DP will give similar, completely accurate results , regardless of what type of passive-aggressive developer you are dealing with.

4. Actively manage your project

Once you’ve set everything up, take over the active management of the project to deliver it on time. For example, sorting tasks according to their priority will help you plan your work in the future, allowing you to reject low priority functionality if necessary.



You can also see the distribution of possible completion dates for each developer :



Some developers (like Milton (Milton) in this figure) can be a source of problems due to the fact that their ending dates are so inaccurate: they need to learn to estimate costs for tasks better. Other developers (like Jane) have an exact completion date, which, however, delays the delivery of the project - you need to think about transferring some of their tasks to other employees. Some developers (me! Hurray!) Are not at all on the critical path - they can be left alone.

Project boundaries

DP works great when you have everything planned up to the smallest detail. However, you probably will have to implement some unplanned things. New ideas appear, sales managers promise things that are not there, and someone from the board of directors comes up with a bright idea to implement an application for a golf mobile with GPS support and the ability to observe the cardiogram of golfers moving across the field. All this leads to delays that could not be predicted during the preparation of the original plan.

Ideally, special buffers should be provided for all of this. And really - add buffers to your plans for

1. New ideas on functionality
2. Competitive response
3. Integration (combining code written by different developers into one system)
4. Time to debug
5. Usability testing (and use of test results in the final product)
6. Beta testing

Thus, when the time comes to implement an unplanned function, you can always cut a piece of the suitable buffer and use it for work.

But what to do if a lot of new functionality was added, and all buffers are used? Well, in this case, the date of delivery of the project, obtained using the DP , is shifted. You should take a snapshot of the probability distribution for the completion date at the end of each working day to be able to track changes over time:



On the X axis, mark the date of calculation, Y - the date of completion of the project. The figure shows three curves: the upper corresponds to the results with a 95% probability, the average is 50% and the lower is 5%. The closer these curves are to each other, the narrower the range of possible project dates we have.

If the completion date is constantly postponed (ascending curves), you are in trouble. If everything is postponed more than a day a day, you add a new job faster than you finish the existing one - the project will never be completed. If you see that the probability distribution becomes more dense (the curves converge), this means that you actually go out on a given date.

By the way

Here are a few things that I realized during the years of project planning.

1. Only the programmer who performs the work can estimate the time needed for it . Any system in which the management schedules tasks, not allowing developers to it, is doomed to failure. Only a programmer who will implement a specific functionality can know what steps need to be taken to achieve the goal.
2. Correct errors as they are detected, and assign the elapsed time to the original task . You cannot plan to fix every error in advance, because you do not know what problems you will face. If an error is found in the new code, attribute the time spent on correcting it to the original task, which was implemented incorrectly. This will help the PD to predict the time to get fully debugged, not just working code.

3. Do not allow managers to put pressure on developers to reduce estimates of terms. Often, insufficiently experienced software project managers think that they can “motivate” programmers to work faster, giving them good, “tight” (incredibly short) plans for completing tasks. I think this type of motivation is completely brainless. When I do not fit into the schedule, I feel doomed, depressed and devoid of any motivation. When I go ahead of the official plan, I am inspired and productive. Planning is not a place for psychological games.

Why do managers do it?

At the beginning of the project, technical managers meet with businessmen, getting a list of tasks that they think will take about three months, but in fact 12. When you decide to write a program without thinking through all the stages of its creation, it always seems that take n time, but in reality you have to spend more than 4n. When you make a real work schedule, you add all the tasks to it and you understand that the project will take much longer than originally thought. This upsets businessmen.

Incompetent problem-solving managers are trying to find ways to make people work faster. This approach has few points of contact with reality. You can try to hire additional staff, but it will take time to pick up speed, during which it will work with 50 percent efficiency, simultaneously reducing the productivity of employees forced to train them.

You can temporarily squeeze out a 10 percent increase in the speed of writing a raw code at the cost of a complete professional burnout of staff. A small achievement, which rather resembles the consumption of its own stock of seeds. Of course, when you force people to recycle, debugging time doubles and projects that lag behind are even more late. From karma not to leave.

You can’t get 4n from n, you can’t, and if you think you succeeded, send me a ticker of your company, and I’ll cut it.

interpretation of the statement about the ticker from vmarunin
"... send me a ticker so that I can short it"
Shorting (opening a short position) means borrowing stocks, selling them, and when they fall in price to buy stocks and return the debt.
In other words, Joel is ready to put money on the fact that if a company can shorten the development period from 4n to n, then it will soon fall on the exchange.

4. Schedule - a box with wooden blocks . Suppose you have several wooden blocks that you cannot place in a box, then there are two options: take a larger box or get rid of several blocks. If you want to deliver the project in 6 months, and according to the work plan, it turns out to be 12, you need to either postpone the release date, or abandon some functions. You can not just take and reduce the wooden block.

I would like to especially note that one of the main advantages of realistic schedules is that it forces you to delete functions. Why is it good?

Imagine that you come up with two functions. The first, very useful, can make your product really beautiful. The other is extremely simple, and programmers can’t wait until they can encode it (”Look! <Blink>!”), But it doesn’t have any real benefit.

If you do not make a project plan, programmers implement a simple and interesting function in the first place. Then they will break the deadlines, and you will have no other conclusion than to move the date of delivery, in order to have time to realize a useful and important function.

If the schedule is made before the start of implementation, you will immediately realize that you will have to sacrifice some functionality and get rid of a simple and fun task, having decided to do only useful and important. By forcing yourself to discard some features of the application in advance, you are helping to create a better product with a better set of functions that will sell better.

When I worked on Excel 5, our list of planned functionality was huge, which foreshadowed a very serious delay in the release schedule. “My God,” we thought, “These are all very important things. How are we going to live without a macro editing wizard? ”

As it turned out, there was no choice, and we cut out everything except the things absolutely necessary for the implementation of the project. This decision did not suit anyone. In order to somehow calm down, we have convinced ourselves that we are not abandoning the dropped functions completely, but just transfer their implementation to Excel 6.

When Excel 5 was nearing completion, Eric Michelman and I started working on the Excel 6 specification. We went through a list of functions that we didn’t have time to implement within Excel 5. And you know what? It was one of the most vulgar lists of opportunities you can imagine. None of the functions listed in it deserved implementation. The process of truncating the functionality in order to fit into the schedule was the best service to the project that we could provide it. If this were not done, the implementation of Excel 5 would take twice as long and half consist of useless crap, which would also have to be maintained for compatibility until the end of time.

Conclusion

Using evidence-based planning is fairly simple — it will take you a day or two to prepare detailed assessments before each iteration, and a few more seconds each day to record when you started working on a specific task. The benefit is huge: realistic project plans.

A realistic work schedule is the key to creating good software. It pushes you to implement the best features in the first place and helps you make the right decisions about what to include in the final assembly. This will make your product better, your boss happier, delight the customer and, best of all, allow you to go home at exactly 5:00 pm.

Thanks to truezemez for helping me translate the method name.