The 12 Levers That Cut an LLM Bill

Lists of cost-saving tips are ordered by how interesting the tip is. The order you want is by money saved per hour of work, and that order is different for every bill — so here is the list, and here is the arithmetic that sorts it.

How to rank a lever

A lever is worth doing in proportion to three things and inversely to one:

score = (B * f * r) / (h * w)

  B  monthly bill, in dollars
  f  fraction of B this lever touches      (0..1)
  r  reduction it achieves on that portion (0..1)
  h  engineer-hours to build it
  w  ongoing hours per month to keep it alive, +1 so w >= 1

The numerator is dollars saved per month; the denominator is the cost of having done it. Score is dollars per hour, so it is comparable across levers and comparable against whatever else that engineer would have built. Two rules make it behave: f and r are estimates, so round them to the nearest 0.1 and do not pretend otherwise; and w exists because a clever routing layer that needs an hour of babysitting a week is not free, however good its r is.

One worked score, entirely from assumed inputs: a $4,000/month bill, prompt caching applied to the 80% of spend that has a stable prefix, achieving a 40% reduction on that portion, costing 6 hours to implement and about an hour a month to maintain. score = (4000 × 0.8 × 0.4) / (6 × 2) = 1280 / 12 = $107 saved per hour of work. Compare that with a model-swap evaluation that touches 100% of spend at a 25% reduction but costs 40 hours of evaluation: (4000 × 1.0 × 0.25) / (40 × 1) = $25/hour. Both are worth doing; only one of them is worth doing this week.

The four structural levers

These change what work happens at all. They have the largest r and the largest h.

• 1. Do not call the model. The cheapest inference is the one a lookup, a regex, a cached answer or a deterministic code path handled instead. Classification with twelve fixed labels and a thousand examples is a classifier, not a prompt. This lever is skipped because it is unglamorous, and it has the highest ceiling of anything on this list.
• 2. Use a smaller model where it holds. The price gap between tiers is often an order of magnitude, so the question is never “is the small one as good” but “what is a failure worth, and how often does it happen”.
• 3. Cascade. Cheap model first, expensive model only when a verifier says the cheap answer failed. Pays only below a computable escalation rate.
• 4. Cut the number of calls per task. Agent loops, multi-step chains and self-critique passes multiply the whole per-request cost by the step count. A step cap is one line and one of the highest-scoring changes available.

The five token levers

These leave the architecture alone and shrink the terms in the per-request formula. Small h, and unusually good scores.

• 5. Prompt caching. Move the stable prefix into the cached-input line, which is billed at a fraction of the uncached rate. Mostly a matter of ordering the prompt so the volatile parts come last.
• 6. Shorten the output. Output is the expensive half per token. Asking for a shape rather than a length, banning preambles, and using structured output all reduce it without reducing information.
• 7. Trim the input. Retrieval that returns ten chunks when three suffice, a system prompt that has accumulated three years of “also, never” clauses, tool schemas for tools this route cannot call. All input, all billed, all resent on every turn.
• 8. Truncate history. Conversation cost is quadratic in turns if you resend everything: turn n pays for all n−1 before it. A sliding window plus a rolling summary turns that back into a linear cost.
• 9. Match the reasoning budget to the task. Thinking tokens are billed at the output rate. Leaving a high reasoning effort on for routine calls is the most expensive default in modern inference.

The three commercial levers

• 10. Batch what nobody is waiting for. Providers with an asynchronous tier discount it substantially in exchange for a completion window measured in hours. Nightly enrichment, evals, backfills and classification jobs all qualify.
• 11. Route on price where quality is equivalent. The same open-weight model hosted by several providers is the same weights at different prices; picking the cheaper one that meets your latency requirement is a configuration change with h close to zero.
• 12. Negotiate, but only on real volume. Committed spend is worth considering once the bill is large and your forecast is reliable, and the break-even is a genuine calculation rather than a discount you simply accept.

Working out your own order

Fill in f and r from your own per-request model, guess h honestly and let the formula sort. Three things fall out of it almost universally, and they are worth naming because they are counterintuitive:

• Levers with tiny h win even at unimpressive r. A 10% reduction that takes an hour beats a 50% reduction that takes a fortnight, on this month’s numbers.
• A lever with f near zero cannot be worth doing however elegant it is. Check what fraction of the bill you are aiming at before you start.
• Anything with a high w should be viewed with suspicion. The ongoing cost is paid forever and the saving usually is not.

Re-run the sort after every change. Each lever you pull alters B and the f of everything else, and the second-best lever on the original list is frequently not the second one you should do.