Skip to content
Multigrid

LLM cost engineering

What an inference bill is actually made of, and the arithmetic for deciding which change to it is worth an engineer's week.

An inference bill is one of the few things in a software system that is genuinely calculable in advance. It is a small number of prices multiplied by a small number of token counts, and every technique in this cluster — caching, cascading, batching, shortening the output — is a change to one of those terms. The reason bills surprise people is not that the arithmetic is hard. It is that nobody wrote it down, so nobody noticed which term was doing the damage.

So these pages are formulas with their variables defined, each followed by one worked example. The prices in the examples are inputs, not facts: per-token prices move every few months and any figure printed here would be wrong by the time you read it. What does not move is the structure — output costs several times input, a cached read costs a fraction of an uncached one, latency tolerance is worth a discount — and the structure is what the formulas are built on. Substitute your own numbers.

How LLM Pricing Works: Every Line on the Bill

Every meter a provider can bill you on — input, cached input, cache writes, output, reasoning, images — and the formula that turns them into the cost of one request.

6 min read

Cost per Request: Building a Model You Can Forecast With

How to build a per-request cost model out of token distributions rather than averages, and which variable in it actually deserves your attention.

5 min read

The 12 Levers That Cut an LLM Bill

Twelve ways to make an inference bill smaller, and a scoring formula that tells you which of them is worth doing first on your bill rather than on somebody else's.

5 min read

Model Cascading: Cheap Model First, Expensive on Failure

The design of a two-stage cascade, and the escalation rate above which it costs more than simply calling the expensive model every time.

5 min read

When a Small Model Is Genuinely Good Enough

A procedure for deciding whether a cheaper model can take a task, built around what a failure costs you rather than around benchmark scores.

5 min read

Prompt Caching: Where the Savings Actually Come From

What a prompt cache can and cannot reuse, how to order a prompt so the reuse happens, and how many hits it takes to pay back a cache write.

6 min read

Batching for Half Price

Which workloads can tolerate an asynchronous completion window, the one question that decides it, and the payback period on doing the migration.

5 min read

Output Length Control: The Most Ignored Cost Lever

How much of a request's cost is the answer, and five ways to make the answer shorter without making it less useful.

5 min read

Cost of Retries, Failovers and Timeouts

What a retry policy adds to an inference bill, why timeouts are the expensive kind of failure, and how to avoid paying twice for one answer.

5 min read

Streaming vs Non-Streaming: Any Cost Difference?

Whether turning on streaming changes what you pay, what happens to billing when a stream is abandoned, and the test that answers it for your provider.

6 min read

Free Tiers and Free Models: What They Really Cost

The four different things people mean by a free LLM, the questions each one has to answer before you build on it, and when free is the wrong choice arithmetically.

5 min read

Committed Spend and Volume Discounts: When to Sign

How to value a committed-spend agreement against an uncertain forecast, why the right commitment level is your pessimistic case, and the non-price terms that matter more than the discount.

6 min read

Self-Hosting vs API: The Real Break-Even Point

The GPU-hour arithmetic for serving your own model, including the ops cost, and why the break-even turns out to be a utilisation figure rather than a volume one.

5 min read

Budget Alerts and Hard Spend Caps

Why an alert is not a cap, how much an alert-only system can lose before a human reacts, and the architecture of a cap that actually stops spending.

5 min read

Forecasting LLM Spend as You Grow

A per-user cost model for inference spend, the three terms that make it grow faster than your user count, and how to keep it honest as the product changes.

5 min read

Unit Economics of an AI Feature

Gross margin arithmetic for a product with inference in its cost of goods, and why the margin has to be computed as a distribution rather than a number.

5 min read

Charging Users for AI: Credits, Seats or Usage

The four ways to price a product with a variable cost of goods, what each does to your margin as usage grows, and the statistic that tells you which one fits.

5 min read

The Hidden Costs: Embeddings, Storage and Egress

An inventory of the AI costs that are not inference — embeddings, vector storage, logs, egress, evaluation runs and human review — with how to size each one.

6 min read

Cost Regression: Catching a 10× Bill Before It Lands

The categories of change that multiply an inference bill, the multiplier each one carries, and a deterministic CI check that fails the pull request instead of the invoice.

5 min read

Currency, Rounding and Why Money Should Be Integers

Why floating point breaks a metered billing system, which integer scale to pick for costs that are fractions of a cent, and where rounding is allowed to happen.

6 min read

Other topics