Earlier this year I was asked to estimate a project. A proper one. An integration between an ERP system, a warehouse platform, and a customer portal, for a UK business turning over around £8 million. They wanted a number. A single number. Something to take to the board.
I gave them a range instead. Their faces fell. You could see them mentally filing me under “difficult”. A range is not a number. A range is uncertainty. And nobody wants to pay for uncertainty.
Here is the thing though. I have been building software for 27 years. I have estimated hundreds of projects, from two-week internal tools to multi-year enterprise rollouts. And I am here to tell you the most useful thing any honest engineer will ever say to you.
I cannot accurately estimate a software project. Neither can you. Neither can anyone.
That is not pessimism. It is maths. And once you accept it, you get better projects. I promise.
The Numbers Are Brutal, and Everyone Has Them
The most-cited study in this field is the McKinsey-Oxford work on large IT projects. They analysed 5,400 projects with budgets above $15 million. The findings: large IT projects run 45% over budget on average, 7% over schedule, and deliver 56% less value than predicted. A third of software projects came in more than 50% over budget. And 17% were what the researchers called black swans, overrunning by more than 200%. Not slipping. Becoming existential threats to the businesses that commissioned them.
You might think that only applies to big enterprise programmes. Too big, too political, too many vendors blaming each other. Your modest SME integration cannot possibly suffer the same fate.
The Project Management Institute’s 2024 Pulse of the Profession report says otherwise. Across its global survey, average project performance was 73.8%, meaning roughly a quarter of completed projects missed their business goals. Around 30% of projects experienced scope creep. And when projects failed, organisations lost on average 25.7% of their budget.
These are averages. Across every industry, every methodology, every toolchain. Agile did not fix it. Cloud did not fix it. JIRA definitely did not fix it.
So when a supplier hands you a single confident fixed number for a bespoke build, ask yourself a question. Are they better at estimation than McKinsey’s entire dataset? Or are they quoting to win the work and managing the overrun later?
Why We Are All So Bad at This
In my experience, and I have thought about this a lot, usually at 2am, there are four reasons.
Reason one: you are estimating the wrong thing. When a client says “build me a stock management system”, what they have in their head is a feeling, not a specification. They know their pain. They do not know what the software should do about it, in detail, in every edge case. Nobody does, until the software exists. The estimate is for the feeling. The build is for the detail. The gap between the two is where budgets go to die.
Reason two: the unknown unknowns dominate. Estimation models, and estimators, work well on the work you can see. The trouble is that in any bespoke build the expensive parts are the things you cannot see in week one. The legacy API that returns dates in a format invented in 1987. The “simple” export the sales director mentions in month two. The supplier whose documentation was written by an intern. I have watched projects where the visible work was two weeks and the invisible work was six months.
Reason three: software is not additive. If a wall takes a builder a week, two walls take two. Software is not like that. Features interact. Every addition touches everything that already exists. A project twice the size is not twice the work, it is more, and the growth is not linear. Humans estimating “twice” systematically underestimate the coupling.
Reason four, and this is the big one: the estimate changes what is being estimated. A confident number creates a fixed scope. A fixed scope creates pressure to hit the number. That pressure creates the very shortcuts, deferred testing, skipped edge cases, and mounting rework that blow the number. The estimate is not a passive forecast. It is an active participant in the outcome.
The Fix Is Not Better Estimates. It Is Different Conversations.
For years I thought the answer was sharper estimation. Better breakdown structures. More reference-class forecasting. Story points. Those things help, marginally. But the real gains came when I stopped trying to predict the unpredictable and changed what I promised instead.
Here is what I do now, and what I would advise any UK SME commissioning software to insist on.
Ask for ranges with a shape, not a number. “Between 40 and 90 days” is honest. It also tells you something a single number never can: the width of the range is a risk signal. A supplier quoting £80,000 plus or minus 5% on a bespoke build is either extremely lucky in their client base or not being straight with you. A supplier quoting £60,000 to £120,000 with the conditions of each outcome spelt out is doing you a service, even though it looks worse.
Buy a discovery phase before you buy the build. Spend 10% of the budget to turn unknown unknowns into known unknowns. You would not accept a builder’s quote for an extension without a soil survey and drawings. Software is identical, except the survey is cheaper and people skip it anyway. A real discovery phase does not eliminate uncertainty. It reveals it. Revealed uncertainty is negotiable. Hidden uncertainty is explosive.
Cap the money, not the scope. This is the single most useful inversion I know. Instead of fixing scope and letting cost float, fix the budget and let scope flex. Decide what the first £40,000 buys: the smallest version of the system that a real user could run a real business process on. Build that. Ship it. Then decide, with evidence, whether the next £40,000 should go on the features you originally imagined or the ones you have since learned actually matter. In 27 years I have almost never seen the second list match the first. That mismatch, discovered cheaply, is the entire game.
Measure by delivered value, not percentage complete. Percentages lie because they are self-reported and back-loaded. “90% done” often means the remaining 10% is the hard 40%. Insist instead on walking-skeleton deliveries: something demonstrable every few weeks, ideally in front of the actual users. A fortnightly demo either produces working software or it does not. There is no ambiguity to hide in.
Budget for the overrun you are statistically owed. If the research says average overruns sit between 30% and 50% on anything bespoke, then plan for it. If the board signs off £100,000, set the internal expectation at £135,000 and be delighted if it comes in under. This is not defeatism. It is what mature engineering outfits call reference-class forecasting, and it is how you avoid the uglier alternative, which is discovering the gap in month seven when the money is gone and the software is not.
The Uncomfortable Truth
There is a reason this industry is still arguing about estimation after 70 years, and it is not that we are all idiots. It is that software development is product discovery, and discovery is not schedulable. You are not paying someone to type. You are paying them to find out what your business process actually is, because no document you could write in advance captures it, including the one in your head.
The suppliers who win on price are often the ones willing to compress the range into a number. The clients who win on outcome are the ones who make it safe to be honest about uncertainty, then manage it deliberately instead of discovering it at the worst possible moment.
So if you are planning a bespoke build this year, do one thing for me. When someone hands you a precise figure, ask what it assumes, and what is not included, and what happens when the first assumption breaks. How they answer that question tells you more about the next six months of your life than the number ever will.
After 27 years, the best estimate I can give you is this: the projects that succeed are never the ones that were estimated well. They are the ones that were structured to survive being estimated wrong.