ReAct: Interleaving Reasoning and Tool Calls

Nearly every agent framework is a re-implementation of one 2022 paper. Knowing the original makes the modern APIs legible, because the API replaced the formatting and left every hard part exactly where it was.

What the paper proposed

Yao et al. (2022), ReAct: Synergizing Reasoning and Acting in Language Models, observed that chain of thought reasons without checking anything, while action-only agents act without planning. ReAct interleaves the two: the model emits a thought, then an action, then receives an observation from the environment, and repeats. The paper evaluated it on knowledge-intensive question answering against a Wikipedia API, and on interactive decision-making environments, and reported that grounding each step in a retrieved observation reduced the fabricated intermediate facts that pure chain of thought produced.

Two comparisons in that paper still matter. Against chain of thought alone, ReAct traded some fluency for grounding: reasoning that has to survive an observation cannot drift as far from the world. Against acting alone, the interleaved thoughts gave the model somewhere to plan, which mattered most when the next action depended on interpreting the last observation rather than on following a fixed script. Both results came from prompting only, on models much weaker at instruction following than anything current — the pattern transferred; the specific numbers should not be quoted as though they describe today.

The original was pure prompting. The format lived in the text and was parsed with string matching:

Question: Which magazine was started first, Arthur's or First for Women?
Thought 1: I need to find when each magazine was started.
Action 1: Search[Arthur's Magazine]
Observation 1: Arthur's Magazine was an American periodical published in
               Philadelphia in the 19th century.
Thought 2: That is vague. I need the founding year.
Action 2: Lookup[founded]
Observation 2: Founded 1844.
Thought 3: Now the other one.
Action 3: Search[First for Women]
Observation 3: First for Women is a women's magazine launched in 1989.
Thought 4: 1844 < 1989, so Arthur's Magazine was started first.
Action 4: Finish[Arthur's Magazine]

The loop, written out

Stripped of framework vocabulary, an agent is this, and it is worth being able to write it from memory before adopting anything that hides it:

messages = [system, user_question]
for step in range(MAX_STEPS):
    reply = model(messages, tools=TOOLS)
    messages.append(reply)

    if not reply.tool_calls:
        return reply.content              # the model answered

    for call in reply.tool_calls:
        try:
            result = dispatch(call.name, call.arguments)
        except ToolError as e:
            result = f"ERROR: {e}"        # an observation, not an exception
        messages.append(tool_message(call.id, result))

raise StepBudgetExceeded(MAX_STEPS)

The context grows monotonically. Every observation you append is paid for on every subsequent turn, which is why a ten-step agent over verbose tool output costs far more than ten single calls.

Put numbers on that growth, because it surprises people at invoice time. Take a 1,000-token system prompt with tool definitions, an average 600-token observation per step, and roughly 100 tokens of assistant text per turn. Step one sends about 1,000 input tokens; step ten sends about 7,300; and the ten-step run totals around 41,500 input tokens rather than the 10,000 a naive count suggests. Cost grows with the square of the step count, which is why truncating observations is a cost control rather than a tidiness preference.

What native tool calling replaced

Modern APIs moved the action format out of the prose. The model returns a structured tool_calls array with a name and JSON arguments, and you return results in a message with the tool role, keyed to the call id. Tools are declared as JSON Schema, so argument shape is enforced by the provider rather than by your regex.

That removes exactly three problems and no others:

• Parsing the action out of free text, which used to break on any prose flourish.
• Arguments that were syntactically invalid.
• Ambiguity about when the model had finished acting.

The thought step did not disappear; it moved. On a reasoning model it is the model’s own reasoning tokens. On a standard model it is whatever text accompanies the tool call, and it is still worth keeping — a transcript of tool calls with no stated intent is significantly harder to debug.

What did not move is worth listing, because framework marketing implies otherwise: deciding which tools to expose, writing tool descriptions a model can act on, keeping observations small, detecting loops, budgeting the run, and deciding what happens when the model gives up. Those were the hard parts in 2022 and they are the hard parts now. The API removed the parsing, which was never the reason agents were unreliable.

Five failure modes

• The identical-action loop. The model calls search("refund policy") four times because the observation did not contain the answer. Detect repeats by hashing name plus arguments, and on the second repeat inject an observation saying so and listing the tools not yet tried.
• Observation flooding. One tool returns 40,000 tokens of HTML and every later step pays for it. Truncate at the tool boundary with an explicit marker, and summarise long results before appending.
• Hallucinated tool names. The model calls a tool that does not exist. Never raise — return ERROR: unknown tool 'x'. Available: a, b, c as an observation. Models recover from an error they can read.
• Answering without acting. On an easy-looking question the model skips the tools and answers from memory. If grounding is the point, validate that at least one retrieval occurred before accepting a final answer.
• Silent context overflow. The transcript passes the window and the provider returns context_length_exceeded mid-run, usually on step seven of a job that ran fine in testing. Track tokens per step and compact before you hit the wall.

Budgets and stop conditions

A loop with a language model in the condition is a loop that can run forever on your account. Three limits, all of them cheap to add and all of them regretted only when absent: a maximum step count, a total token budget across the run, and a wall-clock deadline. Return the best partial answer when a limit trips, and log which limit it was — that distribution tells you whether the tools are too weak or the task is too vague.

Underneath all of it is one design point: the transcript is your only state, and it is append-only unless you manage it. Compaction — replacing older observations with a summary while keeping the tool calls themselves verbatim — is the standard remedy, and it has a standard hazard, which is that the summary drops the one identifier the model needed on step nine. Keep ids, keep error text, summarise prose.