The Standish group famously said that 68% of projects fail to meet estimates. But did they fail to deliver, or were 68% of estimates unrealistic?
I've worked with teams that use story points and on teams that don't. The common factor on both kinds of teams was confusion. What is a story point, and why use them when we naturally think about estimates in terms of time?
We're Terrible At Time
Human beings are terrible at time-based estimates. Consider the question “how long does it take you to drive to work?”
I might say “usually 20 minutes,” and you might take that as a "reasonable estimate". After all, I do it every day, I'm an expert in my drive-to-work process. We use these off-the-cuff numbers when estimating software all the time.
In a given month, most days it does take me 20 minutes to drive to work. But three days a month I’ll hit every green light and make it in 17 minutes! Two days a month there’ll be an accident, and it will take me over an hour. When you look at the data, my average for a month is more like 25 minutes.
It's extremely hard to avoid estimates based on our “usual” experience: an optimistic view. We don’t factor in how much worse things are when something goes wrong, versus how small the gains are when something goes a bit well.
When it comes to time-based estimates, it doesn’t do any good to "just try to estimate harder", and it won't help to lecture developers about statistics and averages… we’re just not built that way.
What Are Story Points?
I’ve seen story points defined as “an imaginary bit of time”, “an arbitrary measurement of time”, or “a relative unit of time”. Story points don’t measure time!
The most concise way to explain story points is “a unit to estimate story difficulty.” Difficulty isn’t how long a story will take. Difficulty can mean size or scope, but also should include complexity and risk. Potentially difficult stories are assigned more points; less difficult stories are assigned fewer.
Why draw this distinction between difficulty and time? After all, won’t more difficult stories take more time, and less difficult stories take less time?
Assigning a measure of difficulty means that different people on the team can agree on the number of story points for a story, but take different amounts of time to complete the task. This is important on a team; not everyone works at the same speed, and agile teams want to be flexible about who participates in a story, to work collaboratively and to share the work through rotation.
Imagine two people you know doing the same task: the difficulty would be the same but the time taken might vary according to that person’s context, focus, mood, environment, and many other factors. There are too many variables to make a dependable, repeatable estimate of time.
This is also why experts advocate consensus-based or average-based story point planning techniques like "planning poker". Having the whole team estimate time doesn't make the number better, it makes it worse, especially when an individual later has to take responsibility for the task. But when it comes to difficulty, getting everyone to weigh in just clarifies the big picture.
There are other problems with estimating how much time a story will take:
- People get distracted or interrupted during the day.
- Often we need to multi-task (context-switch) between different activities.
- Hours are too fine a unit; days too coarse.
- Complex calculations become confusing and complicated.
- No task starts at the beginning of a day and finishes at the end of the day (or hour).
- Time is an exact measure that doesn’t allow for risk or uncertainty.
Estimating with time leads to dysfunction; an estimate is not a promise of when a task will be complete, but this is often what is heard when you give an estimate.
Moving away from time based estimates helps you install best practices— limiting the maximum size of a story to a fixed number of points, or adopting a growing scale for story points estimates (1, 2, 3, 5, 10, 20, 100) to help outsiders understand that big stories are inherently uncertain.
Everyone can agree on the difficulty for each story, and no matter who works on the story the estimate remains valid.
When you switch to story points, your estimation process stops being about just fighting to stay ahead of a schedule, and starts being about predicting how much difficult work the team can do.
Adopting Story Points
The first question that comes up is “how do you know what story is a 1, and what story is a 3?”
Never start with “1 day equals 1 point”; this just confuses your team. Get off on the right foot by agreeing that 1 point equals some well understood, easy to imagine body of work: maybe “hello world” or “a program that generates 100 random numbers” or "serve a blank web page".
How many points do you target in an iteration? This is where team velocity comes from. Velocity is a measure of how many points you can do in an iteration’s worth of time. In your first iteration, just do as many points as you can, and keep track - but don’t try to predict by calculating some way to convert points into time.
In your second iteration, target the same number of points - you might do more, or you might do less. As you proceed, you can measure how much work you’re getting done, and try to improve. If the number goes up or down, discuss why. And if someone else in the company needs to know how much you can do in your next sprint, just “use yesterday’s weather” and tell them what your average velocity has been for the past two or three iterations.
The key shift in adopting story points is moving away from time based estimates and towards difficulty based estimates. Story points are an amazing tool when used correctly, but if you adopt them alongside a dysfunctional "time-based" approach you're missing the "point" in story points.