Error Recovery: What an Agent Should Do When a Tool Fails

An agent that crashes when a tool fails has thrown away the one capability that made it worth building. The model can read an error and try something else — but only if the error reaches it, in a form it can act on, with the retry decision already made by code.

The tool result is the error channel

The default instinct from ordinary software — let the exception propagate — is wrong here. A tool result is a message to a reader. Failure is information that reader needs, and the reader is capable of responding to it. So: catch, format, append as a tool message, continue.

But the message has to be written for the reader, and this is where most implementations lose most of the benefit. Compare:

BAD    "Error"
BAD    "KeyError: 'user_id'"
BAD    Traceback (most recent call last):
         File "tools.py", line 118, in get_account
       ...42 lines of stack...

GOOD   "ERROR FileNotFoundError: 'src/retry.py' does not exist.
        The workspace root contains: src/, tests/, README.md.
        Try list_dir('src') to see what is actually there."

GOOD   "ERROR ValidationError: 'plan' must be one of
        basic|pro|scale; you sent 'Pro Plan'. Retry with the exact
        enum value."

GOOD   "ERROR PermissionDenied: this key cannot write to production.
        Do not retry -- this will fail identically. Report the
        limitation to the user and stop."

The three good ones share a structure: what failed, why, what is true instead, and what to do next. That last clause is the highest-value text in your entire tool layer. A traceback contains none of it and costs several hundred tokens to say so.

That cost is not a one-off, which is the second principle. The error text joins the transcript and is re-sent on every subsequent request, so a 400-token traceback at step 3 of a 30-step run is paid for twenty-seven times. Concision in error strings is not a style preference; it is the same quadratic term that drives everything in agent cost control.

Six error classes

The class that surprises people is empty-result. By any technical definition it is not an error — the call succeeded — and it is nonetheless the single most common input to a runaway loop, because “no results” reads to a model as a near miss worth another try. Handle it explicitly at the tool boundary by returning what was searched, how large the candidate set was before filtering, and one concrete way to widen it. “0 of 1,284 open issues matched ‘timout’; this index does not correct spelling” ends the loop that a bare “No results.” would have started, and it is three lines in the tool rather than a paragraph in the prompt.

The decision procedure

tool raised or returned an error
 |
 +- transient?      -> RETRY in code: exponential backoff + jitter,
 |                     max 3 attempts, honour Retry-After. The model
 |                     never sees it. If attempts exhaust, downgrade
 |                     to 'forbidden' and tell the model the service
 |                     is unavailable.
 |
 +- malformed-args? -> REPAIR: return the exact validation message.
 |                     Allow ONE repair per (tool, step). A second
 |                     failure of the same shape is a schema problem,
 |                     not a model problem -- escalate.
 |
 +- not-found /     -> REPLAN: return the fact plus an enumeration
 |  empty-result       hint. Never auto-retry. Track repeats: the
 |                     same (tool, args) three times is a loop.
 |
 +- forbidden?      -> STOP THIS BRANCH: return an explicit
 |                     'do not retry' instruction. If the whole task
 |                     depends on it, halt with reason='forbidden'.
 |
 +- corrupt-state?  -> HALT: do not return to the model at all.
                       Emit the run state and escalate to a human.
                       An agent asked to fix a half-written state
                       will finish writing it.

The asymmetry is deliberate. Transient errors are handled entirely in code because the model has no information that helps and every retry it conducts costs a full context re-send. Semantic errors go to the model because it has the task context and your code does not. Corrupt state goes to a human because both of the others will make it worse.

The classifier

import random, time
import requests

RETRYABLE_STATUS = {408, 425, 429, 500, 502, 503, 504}

def classify(exc):
    if isinstance(exc, requests.HTTPError):
        code = exc.response.status_code
        if code in RETRYABLE_STATUS:               return "transient"
        if code in (401, 403):                     return "forbidden"
        if code == 404:                            return "not-found"
        if code in (400, 422):                     return "malformed-args"
    if isinstance(exc, (requests.Timeout, ConnectionError)):
        return "transient"
    if isinstance(exc, (ValueError, TypeError, KeyError)):
        return "malformed-args"
    if isinstance(exc, FileNotFoundError):         return "not-found"
    if isinstance(exc, PermissionError):           return "forbidden"
    return "malformed-args"        # unknown: let the model try once

def invoke(fn, args, attempts=3):
    """Returns (text_for_model, halt_reason_or_None)."""
    for i in range(attempts):
        try:
            return str(fn(**args)), None
        except Exception as e:
            kind = classify(e)
            if kind == "transient" and i < attempts - 1:
                retry_after = getattr(getattr(e, "response", None),
                                      "headers", {}).get("Retry-After")
                wait = float(retry_after) if retry_after else \
                       (2 ** i) + random.uniform(0, 0.5)   # jitter
                time.sleep(min(wait, 30))
                continue
            if kind == "corrupt-state":
                return "", "corrupt_state"
            return format_for_model(kind, e), None
    return format_for_model("forbidden", "service unavailable "
                            "after 3 attempts; do not retry"), None

Two details that repay attention. Jitter is not decoration — without it, a step with eight parallel tool calls retries all eight in lockstep and reproduces the burst that caused the 429 in the first place. And Retry-After is honoured when present, because a provider telling you when to come back is more accurate than your backoff curve.

When recovery becomes a loop

The failure mode error handling creates is the one it was meant to prevent: the model retries forever because every attempt returns something that reads like “try again”. Two counters stop it, and both belong in the loop rather than the prompt.

• Identical-call counter. Hash (tool_name, canonical_json(args)). On the third identical call in a run, do not execute it — return You have called this exact tool with these exact arguments 3 times. The result will not change. Try a different approach or stop. This one intervention resolves a large share of runaway runs.
• Consecutive-error counter. Five errors in a row from any tools, with no successful call between, means the agent has lost the plot. Halt with reason=“error_cascade” rather than letting it spend the rest of the budget flailing.

Both counters are per-run state, not per-tool, and both should record what tripped them. The most useful line in an incident review is never “the agent looped”; it is the exact repeated call, which almost always turns out to be a tool whose output the model could not interpret as an answer. That makes it a tool-layer fix, and tool-layer fixes hold across model upgrades in a way prompt fixes do not.

Both counters, plus the budget checks in stopping conditions, form the complete set of reasons a run may end. Enumerate them, name them, and return the name to the caller. “The agent finished” is not an outcome you can operate on.