Refine the backlog

Refining aims to prioritize requirements from the backlog and describe stories so they meet the definition of ready. A best practice to follow is to have between one and two sprints’ worth of work refined in the backlog. Think of sprint planning as laying down tracks as the delivery train rushes forward.

A few guidelines:

• Refine at least a single sprint ahead. It suggests that requirements are not being collected fast enough or that the domain is shifting faster than the team can build the product.
• Don’t refine more than two sprints of work. This action lends itself to a waterfall-like methodology, where the team makes assumptions that are not necessarily beneficial for the product.

This approach gives the team flexibility and reduces the risk of idle time. As the complexity of the product increases, refining sessions become more strategic in nature. Frequently, teams not only have to refine the backlog but also decommission stories that have lost value, introduce new requirements, and reassess the relative priority of the remaining scope. The nature of refinement sessions also changes depending on the maturity of the product—more involved early in the product life cycle, shorter and more effective in later stages. A unified cross-functional team is more effective at taking individual ownership of the refinement process.

Understanding the mechanics

Refinement sessions are typically scheduled mid-sprint. The session is one of the longest rituals in our delivery framework.

If obstacles arise mid-sprint, the team’s two choices are to either

Refinement is the responsibility of the product manager with assistance from the project team. Sessions happen as often as necessary to guarantee runway and typically take between one to one and a half hours per session. The type of project and backlog clarity determines whether refinement is necessary every sprint or every few sprints. Certain circumstances call for multiple sessions in a single sprint.

If a team requires excessive refinement, bring this to the attention of the managing director. This indicates a highly fragmented and unaligned stakeholder group that likely need an intervention, or that the product team has not gelled and worked out the cadence of their rituals.

The PM is responsible for showing up prepared, with a clear understanding of scope as well as priorities. Then each team works out the organic flow of these sessions to their own liking. If complexity is high and each story takes a long time to reach a DoR, consider using the refinement session to identify all questions and provide answers if available; or finish refining offline once the rest of the answers have been collected from stakeholders.

Think about the sprint schedule and consider how much time is necessary between a refinement session and a sprint-planning session. Try giving a little time (a day) before moving from refinement into an estimation exercise to help the whole team absorb the knowledge and identify any gaps. At the same time, sometimes a gut reaction and fresh imprint of the problem results in a more accurate estimate. Trust your instincts.

Estimating stories

The product manager facilitates the estimation session on the same day or the day after refining the backlog with the working team. Sooner is better; however, complex stories typically require technical discussion before selecting and estimating an approach. “Planning poker” is the exercise the team uses to estimate stories and discuss complexity. Consider the following flow:

• The product manager establishes the duration of the session based on the number of stories that need estimation.
• The PM presents a story and a timer starts.
• One minute is spent discussing the acceptance criteria and answering questions the team raises.
• Each team member puts down an estimate card with expected story points.
• The minimum and maximum estimates are discussed (e.g., what led the team member to believe the story is so simple or complex).
• The PM, engineering lead, and design lead provide feedback.
• The team reruns the exercise for the story and verifies that they are aligned.
• The product manager records the expected story points on the user story.

As teams gel and mature, planning poker may become unnecessary. Complexity estimation and alignment within the team improves in accuracy as the team settles into a predictable velocity with an understanding of their strengths and weaknesses.

Aligning project estimates with refined story estimates

Imagine for a moment that an authentication workflow is a single large epic in a product, estimated during the sales process at $150,000. The implementation work gets broken down into individual stories: identity management, authentication, password maintenance, account lockout, and many others. After taking the time to prioritize the stories, refine them, estimate them, and eventually ship working software—the effort actually requires a $200,000 budget. Okay, not ideal, but within a reasonable margin—estimates are guesses, after all.

But what if said deviation is closer to twice that? How about three times? I’m not making this up—these scenarios have happened in the past, and it ends up being a really tough conversation when time entries are pulled and we have to explain ourselves to the client.

Always track actual epic-level spend to original estimates and communicate with the client if deviations happen. Map changes back to assumptions and risks identified.

What if the deviation is not related to your team’s competence but to the genuine complexity, dependencies, and technical excellence necessary for said component?

To identify and resolve these issues, stop and answer some leading questions:

• Did the team know that this component had to fit within a $150,000 bucket?
• Were there opportunities to incur acceptable technical debt to stay within these constraints?
• Were there deltas affecting this component on the story level?
• Did changes to individual stories increase or decrease the overall epic size?
• Was anyone tracking the cumulative effort of the epic throughout delivery?
• Did any of the risks or assumptions change and thus lead to increased cost?

I can’t stress the importance of evaluating the validity of the estimate enough. Having gone through the estimation process enough times, we’ve learned that (a) scope changes throughout delivery and (b) accuracy in that stage of the project is likely low (especially as it relates to individual stories). Furthermore, the project estimate is measured with different units than those used for the story estimates within sprints, and there is no easy and direct conversion formula. Connect the stories and sprints for the sake of transparency and accountability, and to improve accuracy over time—measure, compare epic actuals to epic estimates, and refine the approach long term.

Estimates and actual delivery costs will deviate owing to clarification of requirements and user feedback. This is acceptable as long as you are in control of overall project outcomes.

As the team ships working products and refines the future backlog, stories are introduced, dropped, changed, and broken down into smaller stories. Even the best intentions and the most qualified individuals sometimes miss an estimate when dealing with mounting complexity, third-party dependencies, and so forth.

Agile purists say that classic Agile key performance indicators (KPIs) are sufficient to predict financial product health. These purists are certainly not wrong. However, projects often operate under severe scrutiny and constraints like time, funding, and stakeholders with limited Agile experience. Simply falling back on these metrics leads to instances of “signal sent, not received.”