In the high-stakes world of software development, the debate over story point estimates has gained unprecedented attention. Teams, particularly those aligned with Scrum and SAFe methodologies, often find themselves engrossed in the intricate art of estimation. However, what might initially appear as a path to predictability turns out to be a mirage in the desert of software development.

The Unforeseen Divergence

It’s a familiar scenario – you’ve diligently assigned 25 story points to the sprint, but as the sprint comes to a close, you realize that only 18 have been successfully delivered. Panic sets in, and the term “failed sprint” reverberates through the team. This divergence between expectation and reality can be a bitter pill to swallow, leading to disappointment, self-doubt, and even frustration.

This situation is particularly poignant when you’re working towards predefined goals. Your plans are meticulously crafted roadmaps designed for adherence. Any deviation from the plan can send shivers down your spine, putting your entire goal at risk.

The Essence of a Plan

At its core, a plan is a blueprint outlining the strategy to achieve a delivery goal. It specifies the tasks to be accomplished, allocates time for each, and prepares contingencies for foreseen hurdles. In essence, it is a well-thought-out guide for the project’s journey.

Embracing Fluidity in Requirement

In the era of waterfall projects, rigidly sticking to an initial vision was the norm. A project began with a preconceived endpoint, and every piece of the puzzle was methodically slotted into place. If anything deviated from the plan, change management panels were activated to maintain the project’s sanctity. Yet, this methodology, while appearing functional, was tailored for contract-based programming projects, not product development.

A Paradigm Shift

Take, for instance, a friend who holds a directorship in a software-driven company. A few years ago, they decided to discard the endpoint-centric approach and instead embraced incremental delivery. They learned from early versions of their features, adapting their vision based on actual customer needs. Had they stuck to the initial endpoint, the feature might have missed the mark, but by being adaptive, it unlocked new revenue streams.

In the product realm, the concept of what should be delivered is in a perpetual state of flux. The original plan may bear little resemblance to the final product. This inherent variability alone renders the waterfall process obsolete.

The Anatomy of Estimates

One of the central issues with estimation lies in the fact that functional decomposition merely scratches the surface of the actual cost of integrating a feature into an existing product. Many teams estimate effort and duration using story points, an approach inherited from XP and transplanted into Scrum. Story points aim to abstract away from hours and days, relying on relative assessments instead of absolute ones.

While this theory has its merits, it’s not without its pitfalls. Attempting to “normalize story points” into hours or days negates their inherent value. Story points were introduced to transcend the confines of clock time.

Effort is but a fraction of an estimate. One might expect that changing 15 lines of code across a few files is a minor task, but programming involves more than mere typing. An astute manager once shared a scatter graph depicting story point estimates against actual time.

The manager’s analysis revealed that the magnitude of story points didn’t necessarily correlate to the magnitude of effort. Instead, it often reflected the variation. The manager’s revelation was that effort was just one part of the estimation puzzle.

Uncertainty and Risk – The Unseen Forces

The estimator’s gut feel considers two additional elements: uncertainty and risk. Uncertainty arises when the team faces uncharted territory. They don’t know how to approach the task, how many challenges they’ll encounter, or the mistakes they might make. Risk pertains to the unforeseen consequences of an action, potentially triggering a domino effect throughout a complex system.

The Power of Gut Feel

Estimators intuitively incorporate these factors into their assessments, often with surprising accuracy. However, some less-informed managers, confronted with large estimates, attempt to break down the work into smaller chunks and re-estimate. This approach strips away the essential components of risk and uncertainty, yielding a deceptively smaller estimate. The euphoria of a seemingly more manageable estimate is short-lived, as the actual task often spirals back to approximate the original estimate.

Reality’s Unpredictable Intrusions

Even with the inclusion of risk and uncertainty in estimates, software development encounters two additional elements that elude prediction: delays and interruptions. Work may not commence when expected, and interruptions can throw a wrench into even the best-laid plans.

Reserving Capacity for Stability

Teams can mitigate these variables by reserving capacity, ensuring they can adapt to interruptions without incurring intolerable delays. Striking a balance between stability and adaptability is the key to achieving predictability.

Bridging the Chasm

The urge to predict software project outcomes accurately is understandable. However, requesting development teams to provide more reliable estimates often backfires. Teams may provide estimates with built-in contingency time, but this can create the perception of lethargy, sparking a cycle of attempts to “trim” estimates and regain predictability. It’s a no-win situation, driven by contradictory forces necessitating both reliable and ambitious estimates.

The Crux of the Issue

The real issue isn’t estimation but the complex interplay of social and technical forces beyond the scope of estimation. The work carries an intrinsic burden of risk, uncertainty, delay, and interruption.

Estimation Can’t Replace Chaos

We can’t estimate our way to predictability in this chaotic environment. Instead, we must embrace the unpredictability and work towards characterizing and reducing chaos within our system.

A Path Forward

Acknowledging the pliability of our plans, we can reduce risk by delivering complete end-to-end features incrementally. We can experiment with small, safe-to-fail iterations to minimize variability. Automated testing and frequent integration can help identify risks early, while reserving capacity ensures adaptability. Incremental delivery allows us to adjust our plans on the go, finding more efficient pathways to our goals. With time, visible and predictable work may make estimates and tracking redundant.

In the end, striving for predictability in software development isn’t about becoming clairvoyant; it’s about taming the inherent chaos in the system.