Cost Curves: Training and Inference, Frontier and Fixed

“Training costs are rising and inference costs are falling” is the usual summary, and it is half of a better statement. Each of those two curves splits into two, the four move independently, and nearly every argument about whether AI is getting cheaper is two people pointing at different ones.

Four curves, not two

The two axes are what is being measured — training or inference — and against what standard: the best available capability, or a capability level held fixed.

Any statement about “the cost of AI” that does not say which cell it means is ambiguous between four claims, two of which are rising and two falling.

The frontier rises by construction

The frontier is not a fixed target that gets cheaper to hit. It is defined as whatever is furthest along, and while additional scale keeps producing additional capability, the way to be furthest along is to spend more than the last attempt.

Suppose capability improves with compute in a
strongly diminishing way — each increment of
capability costs a multiple of the last.

Then a frontier run costs more than the previous
frontier run for as long as anyone believes the
increment is worth buying.

The curve stops rising not when compute gets cheap,
but when either
  (a) the increment stops arriving, or
  (b) the increment stops being worth its price.

Note what this means: a *falling* cost per unit of
compute does not lower the frontier cost. It raises
the capability the same budget buys, and budgets are
set by expected value, not by cost.

That last observation is the one that trips people up. Cheaper compute does not make frontier training cheaper any more than a cheaper cost per metre makes the tallest building cheaper. It makes the frontier further out for the same money.

Which gives a clean test for when the frontier curve stops rising, and it is not a technical test. Frontier spending continues while the expected value of the increment exceeds its cost, so the curve turns over when either the increments stop arriving or buyers stop paying for them. The first would be visible as successive releases that are hard to distinguish on tasks people care about; the second as a shift in purchasing toward the cheap tier even where the expensive one is affordable. Both are observable from outside, neither requires access to anyone’s accounts, and either would be a more meaningful signal about this industry’s direction than a spending total.

Fixed capability falls, twice over

Against a fixed bar, both training and inference costs fall, and they fall for independent reasons that compound:

• Hardware. More throughput per unit of cost, from silicon and from the software that drives it.
• Method. Better data curation, better training recipes, and architectures that reach a given quality with less compute. Sparse architectures in particular decouple total size from work per token.
• Distillation. Once a capability exists in a large model, it can be transferred into a much smaller one. This is the mechanism that most directly connects a frontier advance to a collapse in the price of yesterday’s capability.
• Serving. Batching, caching, quantisation and better kernels raise tokens per accelerator-hour, which is the T term in the price decomposition.

The relationship between the two columns is the interesting part: frontier spending is what produces the fixed-capability decline. Techniques discovered at enormous expense at the frontier become the cheap way to hit a modest bar a year or two later, and distillation moves the capability itself downward. The two curves are not independent phenomena; the expensive one is the input to the cheap one.

The multiplier that offsets the fall

None of the falling curves reach your bill directly, because your bill is denominated in tasks and the number of model calls per task is not fixed:

cost per task = calls * tokens_per_call * price_per_token

Assume over some period:
  price per token   falls to 0.35 of its old value
  tokens per call   rises 2.5x  (reasoning tokens,
                                 longer context, tool
                                 definitions)
  calls per task    rises 4x    (an agent loop replaced
                                 a single call)

  cost per task = 0.35 * 2.5 * 4.0 = 3.5x

The price of the input fell by nearly two thirds and
the cost of the job rose by a factor of three and a
half.

Two of those three multipliers are yours, not the market’s. That is the practical conclusion of the whole page: the falling curve is real, and whether it reaches you depends on architectural choices that are entirely within your control. An agentic redesign can consume a decade of price deflation in a single release.

This also explains a pattern that looks like a contradiction in reporting. A provider can truthfully say the price of its capability has fallen dramatically while a buyer truthfully says their bill has grown every quarter, and neither is spinning. They are quoting different factors of the same product. The only way to settle such a disagreement is to fix the denominator: agree on a task, count the calls and the tokens it takes end to end today versus a year ago, and multiply. Almost nobody does this, which is why the argument recurs.

Note too that the three factors are not independent. Cheaper tokens are what make it reasonable to spend more of them — nobody would run a twelve-call agent loop at a price that made each call precious. The falling price is the cause of the rising call count, not a coincidence alongside it, which is the same rebound mechanism as the elasticity condition operating inside a single application rather than across a market.

What each curve implies about who competes

• Frontier training is capital-gated, and the gate gets higher as long as the curve rises. Few participants, and the constraint is capital and physical capacity rather than ideas.
• Fixed-capability training is not gated. Reaching a useful bar gets cheaper every year, which is why capable models keep appearing from organisations that could not attempt a frontier run. This is the engine behind commoditisation at any fixed requirement.
• Frontier inference is a premium product and should be budgeted as one. If your product requires it, price accordingly rather than assuming deflation will rescue the margin.
• Fixed-capability inference is where the deflation you can actually spend lives, and collecting it requires re-testing cheaper models against your bar on a schedule. Nothing collects it for you.