How to stop wasting money on software estimations

If you’re reading this, chances are that you’re either:

A business owner or a higher-up frustrated by teams regularly underestimating efforts.
An employee affected by, and equally frustrated by, your superiors insisting on estimates, then contesting them because they seem too large.

No matter your camp, you’re not alone by any means and you’re right to be frustrated. From personal experience I’d argue that most companies run into this, and so far I haven’t seen a single one that made this topic into a non-issue.

Even more frustrating is that most attempted remediations only amplify the issue by introducing additional ceremony and commitment to an effort that is basically waste from the get-go. Let’s dissect a couple of the common issues to demonstrate why that’s the case.

”I’m not experienced enough to estimate well”

As a junior/mid, I dreaded giving estimates because, in retrospect, I felt I was going to breach them no matter what.

At the same time I also held the following beliefs:

I’ll get better at it with more experience, so by the time I’m in a senior position I’ll be able to do this with confidence and good accuracy.
It will get easier with time.
There are surely companies that have cracked this and are doing it right.
By considering more things (vacations, sick leave, national holidays, tech debt buffer, …) and putting more time into planning ahead, the estimates would get better.

But instead of getting validated, all of these beliefs evaporated as I went forward on my engineering path. Here are a couple of reasons why:

Now I’d argue that experience alone isn’t a major factor in doing estimates well. Sure, it somewhat helps, but different paths can lead to the same role, and two seniors might produce wildly different estimates based on their growth journey: past employers’ cultures, working in different teams with different processes, project knowledge, etc. This is already an indicator that estimates are highly subjective.
Maybe there are companies that have cracked this, but to this day I have yet to see one. And if somebody had really succeeded in this, we’d already be using these approaches all around the industry.
As opposed to just thinking up a number, putting more effort into estimates might occasionally give you a slightly better guess. Often enough it also won’t. And now you’re paying for days (or weeks) of work that doesn’t produce any consistent or reliable results. The longer the estimated period, the larger the error margin and the uselessness of the estimation become. And even if all of the produced estimates were accurate enough, there’s still the big question of whether putting this amount of money and effort in them couldn’t be much better spent elsewhere.

An estimate was given, then got hugely contested

The PM asks for an estimate on the upcoming work. The developer goes away, crunches the numbers for 3-4 hours, then comes back saying: “8 weeks”. The PM, shocked, says “I expected something more like 2, can’t we cut anything?”. After some discussion and 1-2 additional hours lost, they agree on something between 2-4 weeks. The real work ends up being 12 weeks anyway. Sound familiar? All this time, meetings have taken place with constant inquiry into how this could have happened, delaying things even further and burning more cash. The answer is always a variation on “we have discovered something we hadn’t initially accounted for” anyway.

There’s an endless list of potential reasons for this, some common ones being:

The work introduces dependencies on other teams. A lot of conversation and priority shifting is necessary to get it done.
In typically stressful environments, a whole bunch of “done is better than perfect” code needs to be cleaned up to support feature development.
The “minimal” scope is actually not really minimal and could be delivered in smaller chunks.
Hidden work: the devs working on the feature were pulled in to help with other estimates, incidents, etc. An hour or two here and there compounds over months. Split attention and context-switching take a large toll on delivery.
The developer has both more skin in the game and less leverage at the same time, so it’s easier to pressure them into uncomfortable numbers.

”We just need to estimate better”

This has partly been tackled above, but it deserves its own section.

Everybody sees that estimates get breached often and that they simply don’t work as intended. But “we need some number to be able to plan our work”, so the instinct is to, e.g., create a working group with seniors, architects, PMs, etc. to devise a thorough estimation methodology, resulting in a meticulous and complicated Excel sheet that work items get entered into with risk levels etc.

Now you’ve used a whole lot of time from a bunch of highly talented people to basically devise the project-management analogue of an astrological birth chart: something that looks very intricate and professional, yet doesn’t mean anything. Once you start the actual work, you’ll run into many things that haven’t been foreseen by your ~~chart~~ estimation sheet and the delivery date will shift many times over.

”Just give me a rough number now and we’ll adjust later”

This number is almost certainly off by a large margin. Now you’re either putting somebody in a bad position, or you’re being put in one. It’s a common joke (but also practice) that “rough numbers” immediately become commitments. If your plans depend mainly on such hastily given estimates, then they aren’t worth much anyway.

Why did we get into this mess?

Estimates are simply a very obvious and intuitive way to, e.g., request a price from a contractor or a deadline from a developer. You can then hold them accountable if the dates get breached, so at the start it all seems predictable and low-risk.

Also, requesting estimates takes basically no effort and requires no process. Had they worked as expected, management would easily just ask for them, put the numbers into their neatly organized Gantt chart and plan further in the future while everything gets delivered on time.

Yet we know things don’t work this way by any means. In many cases it’s hard to even plan for a month ahead because changes in business requirements come in on a daily basis. The deadlines then get moved and future work gets reshuffled and reprioritized. Regularly.

And when we notice that a workstream isn’t going according to plan, we want to know how long it will “really” take now, so we push people who are already under stress to give us another estimate. It might happen a few more times before the work ultimately gets delivered.

“But again”, you might be thinking in an already irritated voice, “we need to be able to plan and we need numbers!”.

Yes, and I even agree. But you want these numbers to actually tell you something useful, and you want your plan to be worth something. You’ll also need to put a bit of effort into it.

Projections

Some of the insights we could infer from above are:

Requirements change regularly, so ideally we’d update our timeline on every change and track it just as often.
Ideally the timeline updates would happen instantaneously and with little effort to save costs.
Giving estimates based on experience is unreliable and not of much value. Maybe there’s a better source of data to deduce durations from.
Estimates cost real effort and money, and don’t produce the value to warrant that cost. Something automated would let people spend their time on valuable work instead, with less meeting overhead and stress.

Projections, while not completely precise, will give you much more useful input on when to expect the work to be done. They track the actual cadence of resolving tickets over a period of time, then project a date (or a date range) based on that data. That’s it!

A screenshot of a projection in Linear showing a completion date range

A projection in Linear: instead of a single committed deadline, it shows a completion date range derived from the team’s actual ticket-closing cadence.

If new tickets are introduced or existing ones are removed, or when the average ticket resolution time changes as some tickets take longer or less time to close, the projections change accordingly. This all runs autonomously and practically requires no team involvement.

If somebody goes on sick leave, or on vacation, or if there’s a national holiday, the timeline will reflect it.

There are a few things to keep in mind, though:

It takes a while, e.g. 2-3 weeks, before you start seeing any meaningful results. Your data is coming from measuring the team’s actual workflow, so some time is needed to collect enough of it.
The first projected completion date will likely change often. It will, however, get updated continuously, and it’s usually easy to give all project stakeholders access to the always-up-to-date value. This also better reflects reality, as the dates are expected to change with new findings and scope updates anyway.
Your team needs to exercise a bit of discipline, taking care that work is correctly tracked in tickets and that they keep getting updated. The simpler you make your project management, the more likely you’ll get your team’s buy-in. E.g. write small tickets, don’t put documentation in there, write user stories only and count on team interactions to figure things out, etc.
Some project management tools haven’t been built with this in mind and actively encourage the “old way”. It’s unlikely your company will be migrating away from them soon, but they can very likely be extended to calculate projections via plugins or additional tools.
This is a mentality shift as much as it is a workflow one. The reality of sliding completion dates might not be as easy to accept for some decision makers as a comforting set-in-stone commitment (that is nevertheless also a lie). If you are a decision maker reading this, I encourage you to help the cause and stop the practice of software estimation in your company.

Where to go from here: the #NoEstimates movement

Hopefully I have piqued your interest in projections and maybe even convinced you to give them a try. There is much more to be learned, so I highly encourage you to get started with the same great sources that have taken me down this path: