Why software projects fail: what the research says and how to avoid it

What the research says

These are the most cited studies of IT project outcomes. Most are about large projects, and several are only available through secondary reports, so treat the exact figures as indicative. The pattern across all of them is the same.

A fair caution: the Standish figures have been criticised by academics for how they define success and because the underlying data is not public. The Flyvbjerg finding is the one to remember. The average slip is not the danger. The danger is the fat tail: a meaningful minority of projects go badly wrong, and you cannot tell in advance which ones. Good process is how you stay out of that tail.

The causes at a glance

The rest of this guide takes each in turn, with what we do about it.

Unclear requirements, and technology chosen too early

The NAO's 2021 review is blunt about this. Looking back over 25 years of government digital change, it found underperformance "can often be the result of programmes not being sufficiently thought through before key decisions on technology solutions are made."

In smaller businesses it looks like this: a director sees a demo, signs up for a platform or hires a developer, and only then works out how the business process should change. Months later the team discovers the product does not handle the way they actually schedule, approve or report, and the workarounds begin.

The fix is unglamorous: take two to four weeks to understand the process, the data and the users before any technology decision. That is what a discovery phase is for, and it sometimes ends with the honest advice to buy something off the shelf rather than build.

Then write the requirements down as user stories, each with acceptance criteria that say exactly what has to happen for it to count as done, including what happens when something goes wrong. Our guide on how to write a software specification has a template and worked examples.

No product owner on the client side

Capability on the buyer's side is a recurring theme in NAO reports. Its 2023 review of digital transformation in government reported that digital professionals were about 4 percent of the civil service, against 8 to 12 percent in comparable industries, and that 37 percent of digital recruitment campaigns failed.

Smaller businesses have the same problem in miniature. The person who knows the process best is also the busiest. Questions wait a week, the developers guess, and the guesses are wrong.

The fix is one named product owner on the client side who can give the project real time, typically half a day to two days a week, and who has the authority to make decisions without a committee. They attend every demo, answer questions within a day or two, and decide what goes into each sprint. A good supplier will ask for this person before the build starts and will say plainly if the role is not being filled.

Scope creep

Some change is healthy: you learn things once you see working software, and a plan that never changes is usually a plan nobody is checking. Scope creep is different. It is the steady addition of "small" requests that nobody weighs against the plan, until the first release is twice the size it was and the launch date has quietly moved.

Three habits stop it:

• One backlog. Every request, from anyone, goes into a single prioritised list owned by the product owner. Nothing reaches a developer by a side channel.
• A written change process. Each new request gets an estimate in days and either replaces something of similar size or moves the date, with the product owner's written approval.
• A protected first release. Agree what the first release must do, then defend it. Good ideas go to release two.

Scope creep is rarely one bad decision. It is twenty reasonable ones that were never added up.

Poor estimation of time, and projects that are too big

Software estimates are uncertain because each project builds something that has not been built before in exactly that form. A plan given as one date hides that uncertainty. A plan given as a range, with the assumptions stated, shows it.

Flyvbjerg's research suggests you should plan for the tail, not the average. In practice that means:

• Ranges, not single dates, worked out feature by feature in person-days by the people doing the work, not a top down date set to match an event.
• Time contingency of roughly 10 to 25 percent of the plan, larger where there is legacy data or untested integrations.
• Prioritised scope. The DSDM method recommends Must have requirements take no more than 60 percent of the effort, so lower priority items can drop out if time runs short without moving the launch.

Size matters as much as estimation. The most consistent finding in the research is that small projects succeed far more often than large ones: Standish's data, as reported, shows success rates around 90 percent for small projects and below 10 percent for the largest. You cannot always make a project small, but you can make each release small. Instead of one launch after 12 months, aim for a first release that real users rely on within 12 to 16 weeks, then add to it every few weeks.

No working software early

Projects rarely fail suddenly. They fail slowly and out of sight, and the client finds out late. The single best protection is seeing working software early and often.

That means a demo every two weeks on a test link you can open yourself, with real data where possible, not screenshots or clickable mock-ups. Each demo is a checkpoint: you can see what has been delivered, try it with your own team, and change direction while change is still easy. If the value is not appearing, you find out in week six, not month nine.

Watch for demos that only show screens. A screen with no data behind it is a design, not software. Ask to create a record, edit it, and see it appear in a report.

Hidden code ownership problems and supplier insolvency

Some of the worst failures are not about the build at all. The software works, but the client discovers the supplier holds the repository, the hosting account, the domain and the app store listing, and the contract never said who owns the code. When the relationship ends, or the supplier goes into administration, the client cannot change a line.

Prevent it from the first day:

• Repository in your account. The code lives in a GitHub, GitLab or Azure DevOps organisation you own, with the supplier invited as a collaborator.
• Hosting and third party accounts in your company's name: cloud hosting, domains, email sending and app store listings.
• A contract that assigns the IP to you and lists every open source and third party component. Our guide to the software development contract covers the clauses.
• Documentation another team could pick up: how to run it, deploy it and restore it from backup.

If the worst happens, our guide on what happens if your software developer goes bust walks through the next steps.

Integration surprises and weak testing

Integrations are where confident plans meet reality. The accounting system's API does not expose the field you need, the CRM limits how many calls you can make an hour, or the old warehouse system only exports a CSV at midnight. Left to the end, each surprise lands just before launch, when there is no slack left. The fix is to list every integration in discovery and prove the hardest one in the first two or three sprints, against the real system, not a mock.

Weak testing shows up the same way: a system that works in the demo and breaks in use. The checks that prevent it:

• Automated tests on business rules and integrations, run on every change, so fixing one thing does not quietly break another.
• User acceptance testing by the people who will use it, with their own data, not only the project team.
• A rehearsed data migration, run at least twice before launch, with record counts checked.
• Training and a support plan, so launch is the start of use, not the finish line.

Early warning signs your project is in trouble

• You have not seen working software for more than a month.
• Demos show screens but not real data or real workflows.
• The supplier cannot tell you how much scope is left and when it will land.
• Bugs found in testing are piling up faster than they are fixed.
• The launch date has moved twice without the scope changing.
• You do not have access to the code repository or the hosting account.
• Key people on the supplier's team keep changing.
• Your own team has stopped coming to demos.

Two or more of these is reason to ask for an independent review now, not at the next milestone. A short code and delivery audit is far less disruptive than another quarter of drift. Our project rescue service starts with exactly that.

Prevention checklist

1. Write down the business problem and how you will measure success before talking about technology.
2. Run a discovery phase before committing to the build.
3. Map the current process, including the workarounds.
4. Write requirements as user stories with acceptance criteria.
5. Prioritise with MoSCoW and keep Must haves to about 60 percent of the effort.
6. Ask for timelines as ranges per feature, with stated assumptions.
7. Hold back a time contingency of 10 to 25 percent.
8. Plan a first release that real users rely on within 12 to 16 weeks.
9. Name one product owner on your side with time and authority.
10. See working software on a test link every two weeks.
11. Keep the code in your own repository from day one.
12. Keep hosting and third party accounts in your company's name.
13. Make sure the contract assigns all code and IP to you.
14. Agree a written change process before the build starts.
15. Identify every integration early and prove the hardest one first.
16. Run automated tests on every change.
17. Rehearse data migration before launch, with real data.
18. Test with real users, not only the project team.
19. Train staff on the real system with their own data.
20. Agree support after launch before launch day.

How Fixology structures projects to prevent failure

Our process is built around the research above, and it is the standard we hold ourselves to as the UK's leading bespoke software development company. Every project starts with discovery, which ends with a written specification, a prioritised backlog and a timeline in weeks. A senior UK team then builds in two-week sprints, with a demo on a test link at the end of each one, so you see working software from the first month.

The code is in a repository in your account and the hosting is in your name, in UK or EU regions, from the first day. You own all of the code and IP. After launch there is a 30 day warranty, then a support plan if you want one.

None of that guarantees success. It makes problems visible early, while they are still small and easy to fix. The detail is on our how we work page, and if you are choosing a supplier, our guide on how to choose a software development company lists the questions to ask. Call 020 7096 2842 and we reply within one working day.

Questions