Why Most Corporate AI Pilots Never Ship

You are probably here for a percentage. There is one circulating, it has been circulating for years, and it is not worth having. What is worth having is the reason a pilot ends without producing a decision, because that reason is structural and you can remove it in an afternoon.

The number you came for

Every few months a figure appears saying that some large share of corporate AI pilots never reach production. It is repeated widely, usually without a link, and where a link exists it points at a press summary rather than at a method. This page will not print one, and the reasons are worth stating because they also say something about your own pilot.

• There is no denominator. A “pilot” can be one engineer spending a fortnight on a prototype, a department trialling a licensed assistant, or a funded programme with a steering committee. Those three things fail at wildly different rates and no survey separates them, so the ratio is between two quantities nobody has defined.
• There is no numerator either. “Reached production” can mean deployed to one team, deployed to everyone, still running a year later, or simply not formally cancelled. A pilot folded into a larger platform is a success or a failure depending on who is counting.
• The respondents select themselves. Anything built from a survey of executives measures what executives are willing to say about projects they sponsored, which is a different quantity from what happened.
• The number is recycled. The same round figure gets attributed to different sources in different years, which is the signature of a statistic that has detached from whatever produced it.

None of this means pilots do not stall. They plainly do. It means the rate is not the useful object, because you cannot act on a rate. The useful object is the list of conditions under which a pilot is incapable of ending in a decision — and those are mechanical, checkable before you start, and mostly free to fix.

What a pilot is actually for

A pilot is an experiment that buys a decision. If you cannot name the decision, you do not have a pilot; you have a demonstration, and a demonstration has no success condition, so it cannot end. It can only be extended, admired, or quietly abandoned when its sponsor changes role.

There are three decisions a pilot can legitimately buy, and they need different designs. The first is does the capability clear a quality bar on our data — that is an evaluation, it needs a frozen set of real examples and a written definition of failure, and it is the same procedure as building an eval set for your own use case. The second is what does it cost at our volume — arithmetic on measured token counts, not an impression. The third is will the people whose work it changes use it — a behavioural question that needs a comparison group, and the one most often answered by asking six enthusiastic volunteers.

A pilot that tries to answer all three at once answers none of them, because the design that makes one credible ruins the others. Enthusiastic volunteers give you a fast quality signal and a useless adoption signal. A randomised rollout gives you a credible adoption signal and takes two months. Choose one, and say in the charter which one you chose.

Five structural ways a pilot cannot ship

Not cultural diagnoses. Each of these is a property of the pilot as designed, visible on day one to anyone who looks for it.

• 1. No decision is attached to the result. Nobody wrote down what happens if it works. There is no approver, no budget line to move into, no named owner on the other side. The pilot succeeds and then sits, because succeeding was never connected to anything.
• 2. The success criterion was written after the results. Whatever the pilot produced gets described as promising. This is not dishonesty; it is what happens when a threshold is set by looking at the output. A criterion written afterwards cannot fail, and a test that cannot fail carries no information.
• 3. There is no comparison condition. The team using the tool got faster. So did everyone else that quarter, because the backlog changed. Without a group that did not get the tool, or at minimum a stable baseline measured before it, nothing observed can be attributed to the tool and any figure produced is a story with a number attached.
• 4. The pilot ran on data production will not have. A hand-cleaned export, one tidy document set, one customer’s records, no permissions model, no stale rows, no second language. The pilot measured a capability against a dataset that does not exist outside the pilot, which is why data readiness is assessed per use case rather than declared once for the company.
• 5. Nobody owned the production budget. Pilot money is discretionary, small and available now. Production money is a line item, larger, and approved on an annual cycle. A pilot that concludes in October and needs funding from a cycle that closed in September does not fail on merit; it fails on the calendar, and it will be described afterwards as a technology problem.

The pilot charter

One page, agreed before any code is written, signed by the person who will approve production. Six fields. Every one of them is a question the pilot cannot answer for itself later.

PILOT CHARTER

1  DECISION      The single decision this pilot buys, and who makes it.
                 Bad:  "explore whether AI can help support."
                 Good: "decide whether to route tier-1 email drafting
                        through a model for the whole support org in Q1."

2  SUCCESS       The threshold, as a number, with the metric defined and
                 the measurement method named. Written now, not later.

3  ABANDON       The threshold below which we stop. Also a number. If
                 there is no such number, the pilot cannot fail.

4  COMPARISON    What the result is compared against: a control group, a
                 pre-period baseline, or the current manual process
                 measured the same way.

5  TIME BOX      End date. Not "about a quarter". A date, after which the
                 pilot reports whatever it has.

6  NEXT BUDGET   Which budget line production comes from, whose it is,
                 and when that budget's approval window opens.

Field 6 is the one people leave blank and the one that decides most outcomes. Fill it in first: if the answer is “we would have to ask”, ask before the pilot rather than after, because the answer changes what the pilot must prove and to whom. A programme whose funding window opens in six weeks needs a result in five, and that constraint should shape the design rather than ambush it.

The stopping rule

Two thresholds rather than one, because a single threshold produces a long argument about a result that landed just underneath it. Set a success line and an abandon line, and name the action for the band between them in advance — normally a single extension of stated length with one stated thing changed. One extension, not a habit.

Write down who decides, and write down that the decision happens on the end date whether or not the result is comfortable. The purpose of a pre-committed rule is not rigour for its own sake. It is that on the end date the room will contain whoever sponsored the pilot, and the sponsor’s entirely reasonable enthusiasm is the input the rule exists to neutralise.

Two practical additions. Record token counts and per-request cost from the first day, because the cost question is answered by arithmetic on logs and cannot be reconstructed later from memory. And keep the frozen evaluation set and its runner in the repository: the model you tested will be superseded, and re-running the decision should be an afternoon rather than a second pilot. Almost everything separating a pilot that ships from one that does not is decided before the first request is sent, which is why this page is a charter rather than a list of encouragements.