Engineering Estimates

I have met 4 engineers in my 25-year career whose estimates I would trust.

This is not a mark of how bad we are at estimating, but how hard estimating engineering work is.

Early in my career, I was taught that the ‘accurate’ way to estimate was to spend a lot of time breaking down a big problem into all its component tasks, until each task was 1–4 hours in size, and then add everything up. What I came to learn is that this just gave false confidence in the estimate and didn’t increase precision, or eliminate uncertainties.

The modern version of time estimates is Story Points. It is seductive for management to quickly focus on ‘velocity’ as a proxy for their engineering teams’ output. Here are some of the key problems with that :

• story points are typically a Fibonacci-like sequence (1, 2, 3, 5, 8, 13, 20…) — this means that any estimate has error bars of +/- 50% right out of the gate
• the correlation of story points to ‘business value delivered’ differs by engineering team i.e. you can’t add or equate story points across teams
• if an engineering team working on a given backlog changes in size or composition then the correlation of story points to ‘business value delivered’ changes
• bugs and research are typically not allocated story points — more bugs or research in a sprint == lower velocity, but not necessarily lower business value
• not everything a team does is measured in story points (client calls, interviews, meetings, etc.) — if these needs ‘spike’ in a given week then velocity will be lower, but the overall business value might be higher
• as soon as the teams know that velocity is a KPI, behaviors will change to value ‘high estimates’ on work

even before you recognize that estimating is hard.

So what’s the solution?

• supporting your team in going as fast as they can sustainably go
• ruthless prioritization — make sure the team is truly working on the most important things to the business
• iterative delivery — deliver in chunks so that value is unlocked for the business as frequently as possible
• transparency around progress — surface as soon as possible if something is off-track so that decisions can be made about how to course-correct
• removing frictions and inefficiencies — try to keep the engineering team focused on delivering business value

If you hire driven, dedicated, engaged people then this will give you the maximum output over time. In turn, you don’t need to have continuous scrutiny and low-value analysis about ‘why did velocity go down (or up) this week?’.

Instead, you should be asking “Are we prioritizing the right things?”, “What frictions or distractions does the team have?” — your engineering teams will thank you.

(Astute readers will also be asking: what use are Story Points then? The primary value I find in Story Points is exposing uncertainty and disagreement about a ticket e.g. if I size something a ‘3’ and you size it an ‘8’ then there’s clearly some missing information or context, or some hidden assumptions, that should be talked about.

I also love that Story Point sequences are typically Fibonacci-esque — it explicitly captures that the more complex something is, the less confident you can be about the precision of your estimate. If you’ve ever found yourself saying “I think it’s more than 8 story points, but not as much as 13” then I guarantee you were fooling yourself about how precisely you knew what would be involved)