How Many Tools Is Too Many?

Nobody can tell you the number, because it depends on how similar your tools are to each other. But half of the problem is not about accuracy at all — it is arithmetic, it is calculable in advance, and it is usually the half that hurts first.

The question has two halves

“Too many tools” conflates two failures with different shapes. The first is selection degradation: with more candidates the model picks wrong more often, especially among near-synonyms. The second is context tax: every tool schema is input tokens on every request, forever, whether or not it is ever called.

The second is deterministic. You can compute it today, before writing any evaluation, and for most agents it turns out to be the binding constraint — teams cut their tool count for cost reasons long before they can prove an accuracy problem.

The arithmetic half

A serialised tool schema — name, description, three or four parameters with their own descriptions — runs roughly 100 to 200 tokens. Rich ones with enums, boundary clauses and examples run 250 to 400. Take 130 as a working average for a well-written but not enormous tool, and take a catalogue of 60:

catalogue          60 tools x 130 tokens        =   7,800 tokens
one 12-step run    7,800 x 12 requests          =  93,600 tokens
                   (the tool block is resent in full on every step)

at $3 per million input tokens
per run            93,600 / 1e6 x $3            =   $0.28
10,000 runs/day    $0.28 x 10,000               =   $2,808 / day
                                                =  ~$84,000 / month

...before a single message, tool result or answer is paid for.

The prices are illustrative — take a rate typical of a mid-tier model and substitute your own; the structure of the calculation is what matters, not the constant. What the structure says is that catalogue cost is multiplied by loop depth. A 60-tool catalogue in a single-turn classifier is 7,800 tokens and nobody notices. The same catalogue in a 40-step agent is 312,000 tokens per task and dominates the bill.

There is a second cost with no line item. Those 7,800 tokens occupy context that would otherwise hold file contents, search results or conversation. In a long run this is what pushes you into compaction early, and compaction is where agents lose the thread.

What prompt caching does to that number

Tool schemas have the two properties that make prompt caching work: they sit at the very top of the context, and they do not change during a run. Where a provider bills cache reads at a fraction of the input rate — an order of magnitude cheaper is typical, with a smaller surcharge on the write — the same 12-step run looks like this:

uncached   93,600 tokens at full rate                    = $0.281
cached     7,800 full  + 85,800 at ~1/10 of the rate     = $0.049

~5.7x cheaper, for one configuration change, IF the prefix
is byte-identical on every request.

The last clause is the catch and it is where implementations fail. Anything that varies invalidates everything after it, so a timestamp in the system prompt, a tool list built from a Python set with non-deterministic ordering, or a per-request user id injected above the tools will silently cost you the entire discount. Order the block deterministically — static system prompt, then tools, then everything variable — and assert on the prefix hash in tests.

The accuracy half, honestly

There is no universal curve of accuracy against tool count, and any page that shows you one has measured a specific tool set on a specific model. What generalises is the mechanism: selection errors concentrate between tools whose descriptions do not distinguish themselves. Ten tools that each do an obviously different thing are easier than four that all search something.

If you want published numbers, they exist and are maintained by people who run the experiments. The Berkeley Function-Calling Leaderboard (Gorilla, UC Berkeley) scores function selection and argument construction across multiple, parallel and irrelevant-query categories — that last category, where the correct action is to call nothing, is the one most internal evals omit. Sierra’s τ-bench evaluates tool use inside multi-turn task completion rather than single-shot selection. Both move as models ship, which is why the numbers belong on a leaderboard and not in an article.

For your own catalogue, the harness in tool description design gives you the curve that actually applies: run it at 10, 25 and 50 tools and find your own knee.

Four architectures

1. Split by phase

Most agents do not need every tool at every step. A coding agent exploring needs read tools; the same agent applying a patch needs write tools. Swap the tools array when the phase changes and the catalogue halves. The cost is that you now have a state machine around the loop — which, given that you have just made the tool set a function of state, you had anyway.

2. Two-stage selection

Expose a handful of coarse tools, one of which returns a finer catalogue. search_tools(query) returns the full schemas of the three or four tools matching a description, which the model then calls in the next step. You pay one extra round trip and carry a small constant catalogue instead of a large one. This is the pattern behind tool-search features in agent runtimes, and it is worth building by hand if yours does not have one.

3. Retrieval over descriptions

Same idea without the extra round trip: embed the tool descriptions, embed the user’s turn, and include only the top-k tools in the request. Cheap and effective, with one nasty failure mode — a task that needs a tool the retriever did not surface fails in a way that looks like the model being stupid, because from the model’s point of view the tool does not exist. Always include a small always-on set, and log which tools were withheld so the trace can explain the failure.

4. Sub-agents with disjoint tool sets

Give each sub-agent ten tools and a narrow brief, and let an orchestrator route. This bounds catalogue size structurally rather than by policy, and it composes with prompt caching because each sub-agent’s prefix is stable. It also multiplies your token bill in ways worth calculating first — see when two agents beat one.

A blunt heuristic to close on, offered as a starting point rather than a finding: if you are past about twenty tools, the interesting question has stopped being “how do I describe these better” and become “why is my agent expected to hold this much scope”. Catalogue size is usually a symptom of an unscoped agent, and the architectures above are all ways of admitting that.