Procurement: Budgeting for Usage-Based AI Costs

An annual budget is a single number approved once, in advance, by somebody who will be held to it. A metered AI bill is a number that moves every day and rises fastest precisely when the project is going well. Those two instruments are in direct conflict, and the conflict has a structural fix.

Why the two instruments fight

Three properties of metered inference make ordinary budgeting behave badly, and none of them is a forecasting failure.

• Success increases spend. Adoption is the outcome you wanted and it is also the mechanism by which you exceed the line. A budget that punishes over-spend without distinguishing growth from waste is a budget that punishes the goal.
• The unit is invisible to the buyer. Finance approves dollars; the system consumes tokens. Between them sit prompt length, retrieval size, conversation depth and retries — none of which the budget owner controls and any of which can change the bill by a large factor after a routine deploy.
• The distribution has a long tail. Per-request cost is not symmetric around its mean; a small fraction of requests costs many times the median. Planning from an average therefore under-provisions in exactly the months you notice.

The instinctive response — one large annual number with a contingency — fails in a specific way. It removes the signal. A pot large enough to absorb growth also absorbs a cost regression, and the two look identical on the ledger until the quarter ends.

A three-part budget

Split the number into three parts with different behaviour, different owners and different approval rules. This is the artefact; everything else on the page serves it.

The reserve is the part that makes the structure work politically. It gives the budget owner a pre-agreed answer to “usage is above plan” that is neither an emergency meeting nor a blank cheque, and it makes the difference between growth and regression an explicit test rather than an argument.

Deriving the band

The band is not a guess with a percentage stapled to it. Build it from the two quantities you can measure — active users and cost per active user — and let the uncertainty live where it actually is.

monthly = A * q * c

  A   active users in the month
  q   requests per active user per month
  c   cost per request

Take q from logged usage, not from a guess. Build the band from
the spread of q across users, because q varies far more than c:

  low   = A * q_p25 * c
  mid   = A * q_median * c
  high  = A * q_p90 * c

Worked, with every figure an assumption to replace with your own: 500 active users; requests per user per month at the 25th, 50th and 90th percentiles of 40, 120 and 400; cost per request $0.012.

low   = 500 *  40 * 0.012 = $   240 / month
mid   = 500 * 120 * 0.012 = $   720 / month
high  = 500 * 400 * 0.012 = $ 2,400 / month

band = $240 - $2,400, centred on $720
annual variable band  =  $8,640  (mid)
                         $28,800 (high)

A factor of ten between the low and high case is normal and it is the most important thing to communicate. It is not imprecision; it is the real shape of usage, where a small group of heavy users generates most of the volume. Presenting a single number instead hides that, and the conversation you avoid in January is the one you have in July.

Two refinements. Model any conversational feature separately, because resending history makes cost grow faster than request count. And rebuild c whenever the prompt, the model or the retrieval changes — a prompt change is a price change, and it will not announce itself.

Alerts are not controls

A budget is only a number until something enforces it. An alert notifies a human, who may be asleep, and the spend continues at machine speed in the meantime. Three layers of actual limit, each answering a different failure.

• Per request. A ceiling on the size of a single call, set from the p99 of your observed distribution rather than from the mean. This is the limit that catches a runaway loop or a pathological input, and it is the cheapest one to add.
• Per key, per team, per customer. The limit that makes a shared budget divisible. Without it, one team’s experiment is drawn from everyone’s allocation, and there is no way to say whose.
• Organisation-wide, hard. A stop, not a warning, with a decided behaviour when it is reached — degrade to a cheaper model, queue the work, or refuse with a clear message. Designing a cap that actually stops spending is a real piece of architecture, because the naive version checks a counter that is updated after the money is spent.

Two further items belong in this section because they are enforcement in disguise. Anything unauthenticated that reaches a model is a way for a stranger to spend your budget, and the per-request and per-key limits are its primary mitigation. And attribution is what makes any of the above reviewable: if a monthly bill cannot be split by team and feature within a day, the budget has no owner in practice, whatever the spreadsheet says.

The reforecast trigger

Set a rule that fires a reforecast before the year-end surprise rather than after it. A simple, defensible one:

if  cumulative_spend / annual_plan  >  1.25 * (elapsed / period)
then reforecast this month

At 25% through the year, a plan-consuming pace above 31%
triggers the review. Not an escalation — a rebuild of the
band with current numbers, and a decision about the reserve.

When it fires, the diagnostic question is which term moved: A, q or c. More users is growth and the reserve exists for it. More requests per user is either deepening engagement or a retry loop, and those are distinguishable in the logs. A higher cost per request with flat usage is almost always a change somebody deployed — a longer prompt, a larger retrieval, a reasoning setting left on — and it is the one case where the right answer is to fix it rather than fund it.