Token Prices Over Time: How to Build a Series Worth Reading

Everyone has seen the chart: price per million tokens, falling steeply. It is not wrong so much as under-specified, and every problem with it is a problem statistical agencies have spent a century on. Their vocabulary is the fastest route to reading it properly.

That chart is a price index

An index tracks the price of a basket of goods over time so that two dates can be compared. To do that it has to hold something constant — either the basket or the quality of what is in it — because otherwise a change in the number is not a price movement at all. Three difficulties recur, and a price-per-token chart has all three, generally unaddressed:

• Quality change. The good is not the same good at the two dates.
• Basket change. Items leave and arrive, and the index has to be linked across the switch.
• The wrong unit. The thing being priced is not the thing being bought.

Problem one: the good keeps changing

A token bought two years ago and a token bought today are not interchangeable. The output produced per token differs in accuracy, instruction-following, context handling and reliability. Charting them on one axis prices two different goods as one, which overstates the fall in exactly the way that a chart of computer prices ignoring performance would.

The standard treatment is quality adjustment: hold a fixed quality level and ask what it costs at each date. In this setting that means fixing an acceptance bar on a task you care about and pricing the cheapest model that clears it. Note what this does to the direction of the error — unadjusted charts of a fixed named model understate the fall, because they miss that a cheaper model now clears the same bar, while unadjusted charts across whatever is newest overstate it, because they compare unlike goods. The two common ways of drawing the chart are biased in opposite directions.

Problem two: the basket keeps changing

Models are deprecated, renamed, superseded and quietly repointed behind an alias. A fixed-basket index breaks when its constituents are retired, and a chained index — comparing each period only to the one before it, then multiplying the links — is the standard answer, with the standard caveat that chaining accumulates drift.

There is a specific version of this worth naming: an alias that resolves to a different model over time is a basket change disguised as a constant. The name in your code did not change; the good behind it did, along with its price and behaviour. If you are building a series, pin exact model identifiers, not aliases.

Problem three: the denominator is wrong

This is the largest of the three and the one that reverses conclusions rather than merely biasing them. Nobody buys tokens. They buy completed tasks, and the number of tokens in a task is not fixed:

cost per task = calls * tokens_per_call * price_per_token

Assume across two dates:
  price per token       falls 60%   ->  0.40
  reasoning tokens      rise  3x    ->  3.00
  calls per task (agent) rise 2x    ->  2.00

  cost per task ratio = 0.40 * 3.00 * 2.00 = 2.40

Price per token fell by more than half and the cost
of getting the same job done more than doubled.

Both facts are true simultaneously and only one of them is on the chart. This is not a hypothetical accounting trick: reasoning models bill hidden thinking tokens, and agentic patterns turn one user request into a sequence of model calls. Any series meant to describe what things cost has to be denominated in tasks.

The same denominator problem has a subtler form worth guarding against. If quality improves, the same task may need fewer retries, less human correction and no second opinion — so cost per completed task can fall faster than any price. A per-token series misses this in the opposite direction from the one above, understating the improvement. Both errors come from the same source: the token is an input, and only the task is a good.

There is a fourth, quieter problem: list price is not paid price. Cached input rates, batch tiers, committed-use discounts and free allowances all separate the two, and the gap is not constant across providers or over time. A series built from published list prices is a series of list prices, and it should say so.

Building a series that survives contact

The construction is straightforward and the discipline is in doing it the same way every time:

1. Fix a task set and an acceptance bar.
     Freeze it. Changing the bar mid-series
     destroys comparability, and it is tempting
     to raise it as models improve — resist that,
     or start a second series.

2. Each period, evaluate the candidate models
     against the bar. Record which ones pass.

3. Index value:
     I_t = min over passing models of
           (cost to complete the task set on that model)

     including reasoning tokens, retries and any
     verification pass your pipeline actually runs.

4. Record alongside each observation:
     date, exact model identifier, provider,
     list prices used, whether discounts applied,
     and the URL you read them from.

5. Report I_t / I_0, and separately report the
     number of models that passed the bar. The
     second series is a commoditisation measure
     and is often more interesting than the first.

Step five is the part worth stealing. The count of suppliers clearing your bar tells you about your negotiating position and your switching options, and it is the quantity that the drift argument predicts will rise for any fixed requirement.

What the series can and cannot tell you

• It can tell you whether your own cost of doing a fixed job is falling, and at what rate, which is the only version of the question a budget needs.
• It can tell you how many suppliers you could plausibly move to, which is worth more than the price at renewal time.
• It cannot tell you anything about the price of the frontier, which is a different good moving for different reasons — see the two prices.
• It cannot forecast. Extrapolating a deflation series assumes the three underlying terms keep moving, and two of them have floors.

No numbers are printed on this page deliberately. Any figure for today’s prices would be stale within a quarter, and the whole point of a constructed series is that you own the observations and know their date and provenance. Look the current values up, write them down with their source, and the chart you end up with will be the only one you can defend.