Multi-Agent Systems: When Two Agents Beat One

The case for multi-agent systems is usually made with an org chart. The honest case is made with arithmetic, and the arithmetic has a surprising shape: splitting genuinely reduces the dominant cost term, and then one assumption you cannot control usually gives the saving back with interest.

Everything below is a model, not a measurement. The assumptions are stated so you can substitute your own; nobody here has benchmarked your workload.

Why one agent gets expensive

A stateless API re-sends the whole conversation every step. Let P be the fixed prefix (system prompt plus tool schemas), g the tokens each step adds to the transcript (an assistant message plus a tool result), and N the number of steps. Total input tokens over the run:

input(N) = sum over steps i = 0..N-1 of (P + g*i)
         = N*P  +  g * N*(N-1)/2
           ^^^^     ^^^^^^^^^^^^^
           linear   quadratic in step count

The quadratic term is the whole story of agent cost. It is why a 20-step run does not cost twice a 10-step run — with P = 4,000 and g = 1,200 it costs about 2.7 times as much — and why raising a step budget is never the cheap fix it looks like.

What splitting does to that term

Now split the same N steps across k workers, each doing n = N/k steps with its own fresh context:

k workers:  k * ( n*P + g*n*(n-1)/2 )
         =  N*P  +  g * N*(N/k - 1)/2

Compared with one agent's  N*P + g*N*(N-1)/2 :
the linear term is unchanged, the quadratic term is divided by ~k.

That is a real and under-appreciated argument for multi-agent designs, and it has nothing to do with specialisation or role-play. Ten short contexts are cheaper than one long one because each avoids carrying the others’ history. Context isolation is a cost optimisation before it is anything else.

The worked numbers

Assumptions, all illustrative: P = 4,000 tokens, g = 1,200 tokens per step, 250 output tokens per step, and a rate of $3 per million input and $15 per million output. Substitute your own.

SINGLE AGENT, 20 steps
  input   20*4,000 + 1,200*190           = 308,000 tok  -> $0.924
  output  20*250                         =   5,000 tok  -> $0.075
  TOTAL                                                    $1.00

ORCHESTRATOR + 3 WORKERS, 8 steps each, orchestrator 6 steps
  worker  8*4,000 + 1,200*28  =  65,600 tok  -> $0.197 + $0.030
  3 workers                                  -> $0.681
  orch.   6*4,000 + 1,500*15  =  46,500 tok  -> $0.140 + $0.036
  TOTAL                                         $0.857   (-14%)

SAME SPLIT, but each worker needs 15 steps instead of 8
  worker  15*4,000 + 1,200*105 = 186,000 tok -> $0.558 + $0.056
  3 workers                                  -> $1.842
  orch.                                      -> $0.176
  TOTAL                                         $2.02    (+102%)

Same architecture, same prices, same orchestrator. The only thing that changed between the second and third block is how many steps a worker needs, and the answer swung from a 14% saving to a 100% overrun. Wall clock moves the other way — the three workers run concurrently, so the second and third blocks both finish in roughly the time of the longest worker rather than the sum. If latency is what you are buying, that is the column to look at.

What the table does not price is worth naming. That wall-clock advantage assumes the workers genuinely run concurrently, which requires concurrency in your executor and enough provider rate limit to sustain three simultaneous streams. Hit a per-key limit and the workers serialise: you pay the cost of the split and get the latency of the single agent. That is the worst square of the matrix, and it is easy to land in without noticing, because nothing errors — the requests simply queue.

The assumption that decides it

The load-bearing assumption is that total steps are conserved — that 20 steps of one agent become 8 steps each of three. It usually is not, for reasons that are structural rather than fixable:

• Workers rediscover shared context. Each starts cold and re-establishes what the single agent already knew — re-listing the directory, re-reading the config, re-deriving the schema. This inflates both P (you brief them) and n (they explore anyway).
• Summaries are lossy in both directions. The worker compresses its findings for the orchestrator, which compresses its instructions for the next worker. Information lost at the boundary is re-derived by somebody, at full price.
• Decomposition is itself a task. The orchestrator spends steps deciding how to split, and a bad split is discovered only after the workers return.
• Overlap is invisible. Two workers searching the same corpus with different phrasings duplicate work that no component can see, because neither has the other’s context — which was the point.

This is consistent with what practitioners publishing on the topic report. Anthropic’s engineering write-up on their multi-agent research system is explicit that the design consumes dramatically more tokens than a single-agent conversation and is only worth it for tasks whose value justifies that — they argue it suits open-ended research with genuinely parallel branches, and not much else. That is the qualitative version of the third block above.

When it is genuinely worth it

• The subtasks are independent and you want wall clock. Search across five sources, review twelve files, check a claim against three databases. The work genuinely fans out, so the step conservation assumption actually holds.
• Contexts must not mix. One agent handling untrusted web content and another with database write access is a security boundary, not an efficiency choice, and it is worth paying for.
• Tool catalogues are irreconcilable. If a single agent would need sixty tools, splitting is one of the four fixes in how many tools is too many and pays for itself in catalogue tokens.
• Role heterogeneity buys a cheaper model. A planner on an expensive model and workers on a cheap one can beat one expensive agent outright — this is the one case where the multiplier in the cost model works in your favour.

And when it is not: sequential dependent work, anything under about ten steps, anything requiring shared mutable state, and anything you cannot yet debug as a single agent. Multi-agent multiplies your existing observability problem by k. If a single-agent failure is currently hard to diagnose, three agents and a message bus will not make it easier — pick a topology deliberately in orchestrator-worker vs peer agents rather than arriving at one.