Estimating lots of work items can help agile teams dramatically: Estimates help Product Owners make trade-offs, help teams gauge capacity, help Product Owner forecast feature releases, and help team members think about architectural issues. However, in too many cases, estimating is painfully boring and slow. It doesn’t have to be.

Why Estimate?

Agile teams use estimates to help Product Owners make trade-offs. If item A is worth $1000 and item B is worth $400, which should we work on? If we don’t know the time items A and B will require, we might think we should work on item A. However, if we know item A will likely take 13 days and item B will likely take 1 day, we’d probably choose to work on item B: We earn $76 per effort-day for item A and $400 per effort-day for item B. There’s more profit having our team invest in item B.

Estimates help teams establish team velocity. Most agile teams estimate effort using a team-specific unit called “story points.” Scrum has a built-in system that correlates actual team-time with estimated effort, called velocity. It is expressed as “story points per sprint,” and is the average sum of the story points from previous sprints.

Estimates help predict the short term future, which can yield higher long-term team velocity. Teams want to limit the work they accept in a Sprint to help focus on work that can be completed. When teams work on more than can be completed in a Sprint, it turns out they end up reducing the agility of the team and the company. A work pattern where teams always start by accepting a maximum of their historic velocity into a new Sprint is called “Yesterday’s Weather”, and has been shown to yield much higher long-term productivity.

Estimates help predict the long-term future. Scrum was developed, in part, to help companies more accurately predict when teams might complete features. If a team reliably completes 15 story points per sprint, a backlog containing 45 story points can likely be completed in 3 sprints. If every Sprint deploys to users, the Product Owner can plan for delivery of all the features in 3 sprints.

Estimation Difficulties

However, long-term forecasts are sensitive to two difficult problems: 1) Teams should have estimated enough backlog items to establish a useful Forecast Horizon, and 2) The estimates should correlate well to team-time, and a few common forms of bias distort those estimates.

Estimation done poorly is painfully boring and slow, so teams don’t want to do it. To get good estimates, Product Owners have to thoughtfully specify what they want to do, so they don’t do it.

Anchor bias presents the biggest danger facing teams estimating stories together. When a person asserts at a team meeting that a backlog item is “easy,” “should only take 2 days” or “is extremely difficult,” this anchoring estimate biases the estimates from other team members. Team members are more likely to assert an estimate close to the anchoring estimate. Anchor bias is typically more pronounced when the anchoring estimator is a leader—such as an accomplished engineer, architect or manager.

Scrum relies on teamwork to complete backlog items. When a leader produces an estimate, it is likely to reflect the biases of the estimator: a high skill level, broad knowledge of the problem, or professional network likely to help overcome problems. But a team cannot productively rely on the leader to do all the work. Less experienced team members are likely to do the work, and therefore their unbiased estimated can have greater utility to the team.

Scrum Poker

To reduce anchor bias, Scrum teams usually rely on a technique called Scrum Poker or Planning Poker. Scrum Poker is based on a technique called “Wideband Delphi Estimation,” developed by Barry Boehm and the Rand Corporation. Here’s how it works:

1. The Product Owner presents an enabling specification—something that describes the desired outcome or feature from a user’s perspective, typically a user story or a use case—to the team.
2. The team asks questions to clarify the Product Owner’s specification.
3. Each team member estimates a Fibonacci number ≥ the amount of work required to complete a feature that satisfies the specification, and keeps it secret.
4. The ScrumMaster confirms that everyone has selected their secret estimate.
5. The ScrumMaster says, “Ready, Set, Go,” and all team members simultaneously reveal their estimates.
6. The ScrumMaster then selects the outliers, and asks them to explain why they selected a low or high estimate. Oftentimes, this inquiry can reveal issues that other team members haven’t considered. For example, a team member might know that freely available code can accelerate development of the feature, and thus chose a lower number. Another team member might have incorporated a Definition of Done that included thorough functional testing, and thus chose a higher number. This discussion can convince others to change their estimates.
7. The team agrees on an estimate by consensus. If a team member holds out, the maximum number expressed is the estimate for the backlog item.

The Product Owner and ScrumMaster can accelerate Scrum Poker by creating good enabling specifications, before the team meets. If the enabling specifications in the backlog item are incomplete or confusing, Scrum Poker can take a lot of time. By involving architects, user experience designers and operations engineers before a team meeting, the Product Owner and ScrumMaster can provide clearer specifications. And clearer specifications can lead to faster estimation. When multiplied by the number of team members in a meeting, this usually results in substantial cost savings.

One thing to remember: brevity can lead to greater clarity. Shorter specifications are more likely to be thoroughly read by team members.

Bulk Scrum Poker

In it’s standard form, each item is estimated individually. But if a team has a large number of backlog items to estimate, they can be estimated much faster in bulk, with a technique I call Bulk Scrum Poker. I have seen teams estimate 80 stories in 90 minutes this way.

1. Print Stories

Create a complete set of stories reflecting the release you want to forecast. Consider using a definition of Project Ready to ensure that you’ve considered all major risks. Put these in canonical story form, “As a <stakeholder>, I can <accomplish something>, so <business value accrues>.” Print each story on a separate page.

2. Discuss Stories

Spread all the stories out on a table or the floor, so all the stories are visible. Bring the product owner and all members of the development team together in the room. Let everyone know that the development team will be silently estimating the stories, but before that estimation they can ask the product owner to clarify the meaning of the different stories. Caution everyone not to use language that introduces anchor-bias, such as, “Isn’t this story really easy?” Provide at least 30 minutes for this step. If people are not asking questions, quiz them to ensure they have considered the stories. When questions have stopped, go to the next step.

3. Silently Rank

Have the team silently rank order stories in rough order of difficulty, from left to right. Stories on the left should be easy to complete. Stories on the right should be difficult or time-consuming. If team members think a story cannot be responsibly estimated—because there is not enough information or because there are significant implementation risks—team members should put it at the furthest right. If there are conflicts during ranking, such as that one person tries to put a single story further left and another further right, the outcome should place the story toward the right and they should pull the story out or mark it “to be discussed”.

4. Discuss Ranking

During the previous step, some stories will have generate conflict and questions. Let people ask questions and discuss conflicts. Why do people feel that some stories cannot be responsibly estimated? Can those stories be rewritten now to make them easier to estimate? Why do people differ in their assessment of a story’s difficulty? Can those differences be resolved? Rerank stories based on these discussions. There’s no need to be silent during this time.

5. Silently Cluster

Have the team silently cluster the stories into 5 or 6 piles, so that each pile contains stories that are roughly the same difficulty. Keep the ranking, so all stories in a particular pile are easier than the stories in the pile just to its right.

6. Revise Stories

During the previous step, team members may have disagreed. Talk about those disagreements. Are there stories that belong in a different pile? Take another look at the stories on the far right? Is there anything that can be done now to change or split those stories so they can be estimated? If the team can agree, create new stories or rewrite existing ones as needed and put the changed stories in the most appropriate piles.

7. Estimate Third Pile

Has the team worked together before? Does it have a set of reference stories from the past which can help it estimate consistently with its past? If not, assign 8 to all the stories in the third pile from the left, and write “8” on all stories in that pile.

If the team has previously estimated stories together or has reference stories, have the team play Scrum Poker on the third pile, saying this, “Thinking about all the stories in this pile, and considering the most difficult story in the pile, secretly pick a Fibonacci number equal-to or greater-than the the effort required to complete that story.” When all team members are ready, have all team members reveal their numbers. The outliers are the stars of the show. Ask the person who picked the highest number and the person who picked the lowest to justify their estimate. Then discuss among the team. If people want to change their estimates to agree on a single number, take that number as the estimate. Some teams like to play another Scrum Estimation game to find this number. Write that number on all the stories in the third pile.

8. Estimate Stories on Left

Consider the stories in the second pile. Which story is the most time-consuming or difficult? Use the most difficult story in the third pile and the estimate written on it as a reference story, and estimate that most difficult story in the second pile using Scrum Estimation.

9. Estimate Stories on Right

Then estimate stories on the right, again using Scrum Poker.

A large number of stories should now have estimates.

References

Boehm, B. W. 1981. Software engineering economics. New Jersey: Prentice-Hall.

James Grenning, “Planning Poker or How to avoid analysis paralysis while release planning,” (April 2002).

James Grenning, “Planning Poker Party (The Companion Games),” 2 March 2009.