The Model Won’t Call Your Tool

When a model answers in prose instead of calling the function you registered, the cause is almost never the model. In descending order of frequency it is: the tool call happened and your code discarded it, the tools never left your process, the schema is malformed, or the description does not tell the model when to use it.

Before anything: did it call and you missed it

This is the single most common version of “the model will not call my tool”, and it is not a model problem at all. When a model calls a tool, the assistant message has content of null and the call in a separate field — tool_calls in the OpenAI-shaped APIs, a content block with a tool-use type in others — with finish_reason: "tool_calls" or the provider’s equivalent stop reason.

Code that reads choices[0].message.content and logs it sees an empty string and reports that the model said nothing. Print the whole message object before concluding anything:

import json
print(json.dumps(resp.model_dump()["choices"][0], indent=2))
# look for: finish_reason, message.tool_calls, message.content

The streaming version of this bug is worse because it is subtler. Tool calls arrive as deltas across many chunks: the function name in one, the arguments as a string split arbitrarily across the next twenty, with an index field distinguishing parallel calls. Code that concatenates delta.content and ignores delta.tool_calls silently produces nothing at all. Accumulate per index, and do not attempt to parse the arguments until the stream ends — a partial JSON string is not JSON.

Prove the tools left your process

The second-commonest cause is that the request contained no tools. Log the outgoing body once. Frameworks are the usual culprit: a decorator that failed to register, a tool list built from an empty configuration, a wrapper that drops the parameter when a keyword name does not match, or the older functions parameter still being used against an API that now reads tools.

# Most SDKs expose a hook for the raw request. Failing that, point the
# client at a local echo server and read what actually arrives.
import http.server
class H(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        n = int(self.headers["content-length"])
        print(self.rfile.read(n).decode())
        self.send_response(200); self.end_headers()
http.server.HTTPServer(("127.0.0.1", 8888), H).serve_forever()

# then: base_url="http://127.0.0.1:8888/v1"

Thirty seconds of this answers a question that can otherwise absorb an afternoon of prompt tinkering.

The nine causes, each with a test

1. The call happened and was discarded. Test: print finish_reason. If it is tool_calls, stop reading this page and fix your parser.
2. No tools in the request. Test: the echo server above, or the SDK’s raw-request hook. Look for a non-empty tools array.
3. tool_choice is suppressing it. Values are typically auto, none, required, or an object naming one function. A framework that sets none by default, or leaves it unset against an API whose default is not what you assume, produces exactly this symptom. Test: set tool_choice to the specific function by name. If it now fires, the model can call it and the question is only whether it wants to — which moves you to causes 5 to 8. If it still does not fire, the problem is structural: schema, support, or transport.
4. The model or the endpoint does not support tool calling. Not every model does, small and older ones frequently do not, and some providers only support it on certain endpoints or reject it in combination with other parameters such as a JSON response format. Test: the same request against a model you know supports tools. If that works, it is a capability problem and no prompt will fix it.
5. The schema is invalid or unsupported. The requirements are stricter than JSON Schema in general: a top-level "type": "object", a properties object, a required array, and only the subset of keywords the provider documents. oneOf, allOf, $ref, regex pattern and tuple-typed arrays are commonly unsupported, and strict modes usually require "additionalProperties": false plus every property listed in required. Test: replace your schema with a single optional string property. If the trivial tool fires and yours does not, the schema is the cause — then bisect by adding your properties back one at a time.
6. The description does not say when to use it. See the next section. Test: swap in a description that names the triggering situation explicitly and re-run the same prompt.
7. The system prompt is arguing with you. “Answer concisely”, “respond directly”, “do not use external services” and anything about not making things up all reduce tool use, and the last one is on more system prompts than any other sentence. Test: send the same tools with an empty system prompt. If it fires, bisect your system prompt by halves — this is the fastest way to find the offending clause, and the clause is almost never the one you would have guessed. The general mechanism is covered in the model ignoring part of a long system prompt.
8. Too many tools, or overlapping ones. Selection accuracy degrades as the list grows, and two tools whose descriptions overlap split the model’s choice between them so that neither is reliably chosen. Test: send only the one tool. If it fires alone and not in company, the problem is selection, not calling — too many tools and writing tool descriptions are the follow-ups.
9. A required parameter the model cannot fill. If the schema demands a customer ID and the conversation never mentioned one, a well-behaved model asks a clarifying question rather than inventing a value. That looks identical to a refusal to call. Test: make the parameter optional, or supply it in the prompt, and re-run. This one is often the model being right.

Two further contributors, worth knowing but rarely the whole story: a high temperature makes selection less consistent rather than impossible, so a tool that fires nine times in ten is a salience problem and not a bug; and few-shot examples in the message history showing the assistant answering directly teach the model that answering directly is what happens here. Examples beat instructions, reliably.

The description is the prompt

The schema tells the model how to call. The description tells it when. Most tools that do not fire have a description written for a developer reading an API reference rather than for a model deciding between six options mid-sentence.

# Does not fire
"description": "Gets order data."

# Fires
"description": "Look up the status, contents and delivery date of a
customer order. Use whenever the user refers to an order, a purchase,
a delivery or a return, including vague references such as 'my last
one'. Requires the order ID, or the customer email to search by."

The second version names the trigger conditions, names the vague phrasings that should still count, and says what it needs. Every one of those clauses is doing work at selection time. The same applies to parameter descriptions: a parameter described as "the date" gets ambiguous formats, and one described as "the delivery date in ISO 8601, e.g. 2026-08-04" does not.

A harness that answers this in one run

Rather than testing causes one at a time by hand, run the whole matrix once. Each row isolates one variable, and the pattern of passes tells you where the boundary is.

CASES = [
    ("baseline",        SYSTEM,  TOOLS,        "auto"),
    ("no system",       "",      TOOLS,        "auto"),
    ("forced",          SYSTEM,  TOOLS,        {"type": "function",
                                                "function": {"name": NAME}}),
    ("trivial schema",  SYSTEM,  [TRIVIAL],    "auto"),
    ("one tool only",   SYSTEM,  [TOOLS[0]],   "auto"),
    ("temperature 0",   SYSTEM,  TOOLS,        "auto"),
]

for name, system, tools, choice in CASES:
    r = client.chat.completions.create(
        model=MODEL, tools=tools, tool_choice=choice, temperature=0,
        messages=[{"role": "system", "content": system},
                  {"role": "user", "content": PROMPT}],
    )
    m = r.choices[0].message
    print(f"{name:16} finish={r.choices[0].finish_reason:12} "
          f"calls={len(m.tool_calls or [])}")

Read the output as a set of eliminations. Baseline fails and “no system” passes: the system prompt. Baseline fails and “trivial schema” passes: the schema. Baseline fails and “one tool only” passes: selection under competition. Everything fails including forced: capability or transport, and no amount of prompt work will help.