Agent Cost Control: Capping Spend Per Task

The surprising thing about agent cost is not that it is high. It is that it is superlinear in the one parameter people tune most casually — the step limit — so a configuration change that looks like “let it try a bit longer” is a multiple, not an increment.

The shape of the bill

Each step re-sends the entire transcript. With a fixed prefix P (system prompt plus tool schemas) and g tokens added per step, the input tokens over an N-step run are N·P + g·N(N−1)/2. The second term is quadratic, and past about ten steps it dominates. Output is linear and, despite the higher per-token price, is usually the smaller line.

This is why the cost of an agent is not “a few chat completions”. It is one chat completion whose prompt grows every time you send it.

Two corollaries people get backwards. Output tokens being three to five times the price of input does not make output the expensive half of an agent run: at 300 output tokens against 15,000 input tokens in a mid-run step, input still dominates by a wide margin, and “answer more briefly” is a latency fix rather than a cost fix. And a cheaper model does not reduce cost in proportion to its price if it needs more steps, because steps enter quadratically while price enters linearly — a model at half the rate that takes 40% more steps is roughly break-even, and one that takes twice as many is worse.

Doubling the steps triples the cost

Assumptions, illustrative: P = 6,000 tokens (a system prompt and about 25 tool schemas), g = 1,500 tokens per step, 300 output tokens per step, billed at $3 per million input and $15 per million output. Substitute your own rates:

max_steps = 12
  input   12*6,000 + 1,500*(11*12/2 = 66)  = 171,000 tok -> $0.513
  output  12*300                           =   3,600 tok -> $0.054
  run cost                                                  $0.567

max_steps = 24                       (the "let it try longer" change)
  input   24*6,000 + 1,500*(23*24/2 = 276) = 558,000 tok -> $1.674
  output  24*300                           =   7,200 tok -> $0.108
  run cost                                                  $1.782

2x the steps  ->  3.14x the cost
100k runs/month:  $56,700  ->  $178,200

Two things follow. First, a step budget is a cost control, and the most effective one you have — more effective than switching models, in many cases. Second, anything that reduces g pays quadratically: truncating tool output from 6,000 to 1,500 tokens does not save 75% of one step, it removes 4,500 tokens from every subsequent request in the run.

Check before, not after

Accounting after the fact tells you what you spent. It does not stop you spending it, and with a growing context a single late call can overshoot a budget by a wide margin. Estimate the next call before making it:

class CostGuard:
    def __init__(self, cap_usd, price_in, price_out, max_out=1024):
        self.cap, self.pin, self.pout = cap_usd, price_in, price_out
        self.max_out = max_out
        self.spent = 0.0
        self.est_in = 0            # tokens, from the last response

    def affordable(self):
        """Worst-case cost of the call we are about to make."""
        worst = (self.est_in / 1e6) * self.pin \
              + (self.max_out / 1e6) * self.pout
        return self.spent + worst <= self.cap, worst

    def commit(self, usage):
        self.spent += usage["prompt_tokens"]     / 1e6 * self.pin
        self.spent += usage["completion_tokens"] / 1e6 * self.pout
        # next request resends this prompt plus what this step added
        self.est_in = usage["prompt_tokens"] + usage["completion_tokens"] \
                    + EXPECTED_TOOL_RESULT_TOKENS

    def remaining(self):
        return self.cap - self.spent

# in the loop
ok, worst = guard.affordable()
if not ok:
    raise Halt("cost_budget",
               "next call could cost $%.4f, only $%.4f left"
               % (worst, guard.remaining()))
msg, usage = call_model(messages)
guard.commit(usage)

est_in is the honest part. The next request will contain this request’s prompt, plus the assistant message just produced, plus whatever the tool returns — so the estimate is derivable from the response you just received, and it is an upper estimate paired with max_tokens as the output ceiling. That combination means the guard cannot be surprised: the worst case it computes is a real worst case, because max_tokens is enforced by the provider.

A refinement worth adding once this works: as remaining() falls below the cost of two more steps, inject a system-level note into the next request — “budget nearly exhausted, wrap up and call finish”. An agent given warning usually produces a useful partial result. An agent killed mid-thought produces nothing.

Aborting cleanly

“Cleanly” has a precise meaning: the abort must not land between a mutating tool call and its completion, and it must not leave the model believing a write succeeded when the run was killed before it ran. Two rules cover it.

• Abort at step boundaries, never inside dispatch. Once a tool has started, let it finish and record its result. A budget overrun of one tool call is cheaper than a half-applied migration.
• Check the budget again immediately before any mutating tool — the class from parallel tool calls. If the remaining budget cannot fund the follow-up step that would verify the write, do not perform the write. An unverified mutation is worse than no mutation, and this is the single check that separates a clean abort from a mess.

There is a third rule for anything long-running: checkpoint the message list at the same boundary. An abort that discards the transcript throws away tokens you have already paid for, and a resumed run that starts from nothing pays for them a second time. A write per step converts a budget overrun from a loss into a decision — top the budget up and continue, or stop with what you have.

Then return a structured outcome, not an exception: {status: 'aborted', reason: 'cost_budget', spent: 2.04, steps: 19, artefacts: [...]}. The artefact list is what makes an aborted run resumable — the next attempt starts from the files that were written rather than from nothing.

Three layers of enforcement

• In the loop — the guard above. Precise, aware of the task, and completely bypassed by any code path that does not use it. Necessary, not sufficient.
• At the key — a hard spend cap enforced by whatever issues your credentials, so a bug in the loop, a runaway retry or a recursive sub-agent cannot spend more than the key allows. This is the layer that saves you, because it holds when the code is wrong.
• At the org — alerting on daily spend by key, with an expectation for each. The failure this catches is the slow one: average cost per run drifting from $0.60 to $1.90 over a month because a prompt grew.

One habit ties the layers together: give every run its own key or its own tag, so a runaway is bounded to one task rather than to your account. It costs a little plumbing and it converts “an agent spent the month’s budget overnight” from a possible incident into an impossible one.