Scrum thoughts: Improving individual estimation
Series [ Scrum Thoughts ] Tags [ estimation, evidence-based scheduling, monte carlo simulation, Scrum ]
In my last post, I described that our team routinely overestimates the amount of work it can finish in a sprint. The amount of this overestimation is quite high at times - sometimes we only finish two-thirds of what we initially plan.
So how can we get our estimation better? I think there are two sources of estimation error: one is how many hours per day you actually have to apply to sprint tasks; the other is how many hours it will take you to actually finish something.
In his article on Evidence-Based Scheduling, Joel Spolsky describes how we can compare the history of a developer’s estimates vs. the actual time required to get an error factor in their estimates. I think this does a good job of addressing the second type of error, but you need to account for both if you are going to get an accurate estimate of how much you can do in a sprint.
For example, as a team lead, I often get interrupted while coding when someone asks me a technical question, or to participate in a discussion on the dev mailing list, etc. I usually “leave the clock running” for these things, so I think evidence-based scheduling would cover this kind of inaccuracy in my estimation. However, I also get scheduled to attend extra meetings which don’t go towards finishing sprint tasks, and I don’t leave the clock running for that. I also sometimes forget to log all my time spent against tickets, even though I update the remaining time on the tickets to make the burndown accurate. So that’s another source of inaccuracy.
Fortunately, I think there’s a pretty simple solution that covers all these things. Let’s assume we recorded in the past sprint (or past few sprints) how many working days someone was going to be available. Then we simply take the time logged against tickets that were originally planned for in the sprint and divide that by the number of days worked to get a per-day burndown figure. If someone works nights and weekends, they’ll just have a higher per-day burndown rate. If someone routinely has other duties that keep them off sprint tasks (e.g. doubling as a scrum master), their per-day burndown will be lower. If someone forgets to book time, their per-day burndown will be lower. If they routinely forget to plan for required elements of a task or if the product owner injected some new requirements mid-sprint, the per-day burndown goes down.
Now, once you know that, you can use the evidence-based scheduling trick from Joel’s article to use the actual task estimation history (here’s where we compare original estimates vs. actual estimates for finished tasks) to put an actual confidence level on how likely that developer will have a set of estimated tasks done by a certain time.
So, during planning, you let someone estimate hours as if they would get to spend uninterrupted time on it, and then you can use the EBS plus per-day burndown to monitor whether something will likely get done by the end of the sprint.
To make it concrete, let’s say we want to put a 95% confidence level on being able to finish all sprint tasks. So, for each new ticket I write, we go through the following exercise:
100 times, do:
- for each ticket so far, select a random estimate ratio from my estimation history, multiply by the original estimate to get a likely actual time
- add up total likely time, divide by per-day burndown figure to get a likely number of days
- if likely days <= days available this sprint, trial is a success
If you have at least 95 successes, you may take on that task for that sprint. Otherwise, you’re not allowed to commit to it.