Why AI Pilots Don't Reach Production: One Statistic, Four Meanings

You will have seen the statistic. It is usually quoted as a single devastating number and it is usually quoted wrong.

There are at least four of them and they measure four different things. RAND is cited for an 80% failure rate on AI projects, roughly twice that of ordinary IT projects. A Forrester and Anaconda study is cited for 88% of enterprise AI agent pilots never reaching production. MIT is cited for 95% of generative AI pilots failing to deliver measurable profit-and-loss impact. Gartner is cited projecting 60% of AI projects abandoned through 2026 for want of AI-ready data.

Those are not the same claim. Never shipped, shipped but unmeasurable, abandoned for a specific cause, and failed against an undefined standard are four separate outcomes. One review of the field puts it plainly: the published rates vary because each study defines pilot, deployment and success differently.

Which means the headline number tells you nothing actionable, and the useful question is narrower. Not "do pilots fail" but "which of those four things is about to happen to mine".

The honest reading is that something in the region of 80% to 90% of pilots do not become production systems, and that the causes are mostly not technical. The model usually works. What is missing is a definition of done, an owner, data that holds up outside the demo, and a path to the systems where the work actually happens.

Three questions predict the outcome better than any technology choice. Was a success criterion written down before the pilot started? Is there a named person with a budget who owns it? And did the pilot run against real data in real conditions, or against a curated sample?

The pilot was built to be seen, not to run

Most pilots are demonstrations, and a demonstration is an entirely different artefact from a system.

A demonstration optimises for the best case. Clean inputs, a cooperative audience, the happy path, a narrator explaining what is about to happen. It exists to secure permission to continue, which is a legitimate purpose, and it tells you almost nothing about whether the thing can run unattended.

A system optimises for the worst case. Malformed inputs, nobody watching, the model provider having a bad afternoon, a user doing something nobody considered. One published view puts it as pilots being sized for the demo rather than for the operating environment, which is exactly the gap.

The costs sit almost entirely in the second version, which is why pilot effort is a poor predictor of production effort. A pilot built in three weeks routinely needs three months to become something you can leave alone, and that ratio surprises people who read the first number as progress toward the second.

The tell is how the pilot handles failure. If nobody can answer what happens when the model is wrong, when the API times out, or when a user asks something out of scope, the pilot has not started on the hard part.

This is not an argument against pilots. It is an argument for calling them what they are and budgeting the production build as its own project rather than as a finishing touch.

Write the success criterion before you start

Omdia's 2025 AI Market Maturity Survey, fielded in the third quarter of 2025 across 448 enterprises, puts a number on the spread. Under a third of enterprises get fewer than 5% of their proof-of-concept projects into production. At the other end, 10% get more than 40% of them there. Same technology, same year, an order of magnitude between them.

Eden Zoller, Omdia's chief analyst for applied AI, locates the cause outside the pilot itself. Most of the critical effort happens before a pilot begins and continues after it finishes, and failures usually come from enterprises and vendors misjudging what deployment involves rather than from a defect in the technology being tested. The distribution above is mostly a distribution of how well the work either side of the pilot was done.

The failure mode is obvious once named. A pilot without a written threshold cannot fail, and anything that cannot fail cannot succeed either. It simply continues, consuming attention, until interest moves elsewhere.

A usable criterion has a number, a comparison and a date. Not "improve support efficiency" but "resolve 40% of tier-one tickets without human involvement, at a customer satisfaction score no worse than the current one, measured over four weeks ending on a stated date".

The comparison is the part people skip and it is the one that matters. Against what? The current manual process has an error rate, a cost and a cycle time, and almost nobody measures them before starting. Without that baseline, any result can be argued either way, and it will be.

Write the kill criterion in the same document. The threshold below which you stop. Pilots of promising technology run indefinitely on the strength of the next improvement, and the only reliable defence is a number agreed before anyone was invested in the outcome.

Somebody has to own it, with a budget

The common failure is a pilot owned by everybody and nobody. Innovation sponsors it, a data team builds it, an operations team is meant to use it, and when it comes to putting it into the live process, no one person is accountable for the integration, the training, the exception handling or the ongoing cost.

Production needs a budget line, not enthusiasm. Somebody has to pay for the inference, the monitoring, the person who answers when it misbehaves and the work of changing the process around it. Pilots are often funded from an innovation pot that does not extend to any of that.

The owner should be the person whose problem it solves, not the person who finds the technology interesting. An operations manager who wants their queue shorter will push a mediocre system into production. A technologist who wants the architecture right will keep improving a pilot.

Name them before the pilot, not after. Appointing an owner at the end means handing a finished thing to someone who did not shape it and does not want it.

The data problem is the one Gartner named

Of the four statistics, the Gartner projection is the most specific about cause: 60% of AI projects abandoned through 2026 for lack of AI-ready data. That is worth more than the bigger numbers precisely because it points somewhere.

Pilots run on curated data and production does not. Somebody exported a clean set, fixed the obvious problems and handed it over. The live system meets the same data with the problems still in it: missing fields, inconsistent formats, duplicates, records that contradict each other, and the free-text field where everyone puts everything.

Access is a separate problem from quality. The pilot used a spreadsheet somebody emailed. Production needs a live connection, which needs permission, which needs a conversation with whoever owns the system, which in some organisations is the longest pole in the project.

The data that describes outcomes is usually the missing piece. You can almost always get the inputs. Getting what happened next — which prediction was right, which ticket was actually resolved, which recommendation was taken — requires instrumentation nobody built, and without it you cannot measure whether the system works.

Audit the real data before the pilot, not between pilot and production. A day spent querying the production tables tells you more about feasibility than a month of modelling.

Accuracy is not the metric the business cares about

Pilots report accuracy. Production is judged on something else entirely, and the gap between the two is where a successful pilot gets quietly rejected.

Accuracy averages over cases that are not equally important. A model that is 92% accurate overall may be 99% accurate on the easy majority and 55% on the cases that carry the money. The headline figure conceals exactly the segment the business is paying attention to, and nobody discovers this until an expensive case goes wrong.

The costs of the two error directions are almost never equal. Flagging a good transaction as fraud annoys a customer; missing a bad one costs the money. Approving a claim wrongly is not the mirror image of rejecting one wrongly. A single accuracy number averages across a trade-off that someone in the business has a strong opinion about, and that opinion is the actual specification.

So ask for the cost of each error type before tuning anything. Once you have two numbers, the threshold stops being a technical choice and becomes arithmetic, and the conversation with the business gets much shorter.

Coverage usually matters more than accuracy. A system that handles 60% of cases at 97% accuracy and passes the rest to a person is frequently better than one handling 100% at 85%. Pilots optimise the second shape because it demos better, and operations wants the first.

Report the metric the owner recognises. Tickets resolved, hours saved, errors prevented, days of cycle time removed. Precision and recall belong in the appendix, and a pilot presented in those terms is asking a business audience to do a translation they will not do.

Pilot fatigue is real and it has a cost

Organisations that run many pilots and ship few acquire a specific institutional problem, and it is worth naming because it changes what the next project has to overcome.

People stop engaging. Operational staff asked to test a fourth system that will probably be shelved give it the attention a fourth system deserves. The adoption problem that sinks the deployment was created by the three pilots before it.

Sponsors become sceptical of the category rather than the project. Once "AI project" is associated with effort and no change, a genuinely good proposal has to overcome a reputation it did not earn.

The sunk attention is larger than the sunk cost. The money spent on a shelved pilot is usually modest. The engineering and analyst months, and the operational goodwill, are not, and neither appears in any budget line.

The fix is fewer pilots, finished. One project carried properly into production, measured against a real baseline, changes the organisation's belief about what is possible. Three concurrent pilots at 70% complete change nothing and cost more.

If there is a history here, address it directly. Naming it in the kickoff — the last three did not ship, here is what is different about the ownership and the criteria this time — buys more cooperation than pretending the history does not exist.

Integration is where the months go

The pilot sat beside your business. Production has to sit inside it, and that is a different kind of work entirely.

Somebody has to receive the output. A prediction in a notebook is not a business outcome. It has to arrive in the system where the work happens, in a form the person doing the work will act on, at the moment they are deciding.

The surrounding process usually has to change, and that is organisational work rather than engineering. Who reviews the uncertain cases, what the exception path is, what people stop doing, how the new step is trained and audited. None of this appears in a pilot and all of it appears in a rollout.

Authentication and permissions arrive as a surprise. The pilot ran with someone's credentials. Production needs per-user access, which means the system must know who is asking and what they may see, which frequently means a conversation with a security team that has not heard of the project.

Legacy systems set the pace. If the output has to land in software that has no API, you have inherited an integration problem larger than the AI one, and it should have been scoped first.

Budget integration as its own phase with its own estimate. Teams that treat it as the last 10% routinely find it is half the work, and the half that cannot be hurried because it depends on other people.

What it actually costs to go from pilot to production

Production work routinely exceeds the pilot by several times, because it adds error handling, retries, monitoring, authentication, audit logging, an administrative interface, a rollback path and the integration. At our published $40 to $100 an hour, blending to $60 to $70, the larger surprise is the permanent running cost starting at launch.

The uncomfortable arithmetic is that the pilot's cost tells you very little about the production cost.

The build multiplies. Error handling, retries, monitoring, authentication, audit logging, an administrative interface, a rollback path and the integration above. Our published rate band is $40 to $100 an hour by role, blending to $60 to $70 for a mixed team, and the production work routinely exceeds the pilot by several times.

Running cost begins and never stops. Inference at real volume, infrastructure, monitoring and the human review that nearly every deployed AI system needs. This is a new permanent line in somebody's budget and it should be modelled before the decision, not discovered afterwards.

How many pilots you can afford to run at once is where this bites. Omdia found 40% of enterprises running 6 to 20 proof-of-concept projects and 18% running 21 to 50. It also found 43% of firms with revenue under $100 million running fewer than five. If you are the smaller company you get a handful of attempts, each carrying the production cost above if it works. That is the real argument for the kill criterion: not tidiness, but that you cannot afford to leave one of your five running on sentiment.

The thing to compare against is the baseline you measured. Cost of the current manual process against cost of the automated one including review, over a realistic period. If nobody measured the baseline, this comparison cannot be made, which is the single most common reason a working pilot cannot be justified.

Pilots that were never going to work

Some projects fail for a reason that was knowable on day one, and recognising them early is worth more than any delivery improvement.

No baseline exists and nobody will create one. If the current process is unmeasured and the organisation will not measure it, success cannot be demonstrated and the project will end in an argument about whether it helped.

The output has nowhere to go. The prediction is interesting and no decision changes because of it. This is the commonest version of a technically successful, commercially pointless pilot.

The error rate is not survivable. Some processes tolerate being wrong one time in ten and some do not, and the tolerance is a property of the business rather than something to be engineered away.

The volume does not justify the build. Automating something that happens fifty times a month rarely repays a production system, however satisfying the demo.

Nobody wants it. If the people who would use it were not involved and do not believe in it, the deployment will fail at the adoption step regardless of accuracy. Pilots run by a central team for an operational team that was not consulted are the classic shape of this.

What we would do differently

Measure the baseline first, before anything is built. Error rate, cycle time and cost of the current process. It is unglamorous, it takes days, and it is the only thing that makes the eventual result arguable in your favour.

Write the success and kill criteria on one page and get them agreed. Number, comparison, date, and the threshold below which you stop.

Name the owner, with a budget, before the work starts, and make it the person whose problem it is.

Run the pilot on real data in real conditions. A worse result on honest data is more useful than a better result on a curated extract, because only the first one predicts anything.

Scope production as a separate project with its own estimate, presented at the same time as the pilot so nobody mistakes one for a down payment on the other.

We would also say no more often. Several of the pilots we are asked about fail the tests above before any technology is chosen, and the cheapest possible outcome is finding that out in a conversation rather than a quarter.