When Mistral Declines to Call a Tool

You defined a get_weather tool, asked about the weather, and got back a paragraph explaining that you should check a weather service. Nothing failed. The model exercised a choice the API explicitly gives it, and the fix depends on which of four reasons it had.

What the response looks like

There is no error and no special field. The assistant message simply has content and no tool_calls, and the finish reason is the ordinary one:

{
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I don't have access to live weather data. You can check a service like Meteo France for current conditions in Paris.",
        "tool_calls": null
      },
      "finish_reason": "stop"
    }
  ]
}

tool_calls is null or absent, and finish_reason is stop rather than tool_calls. Code that reads message.tool_calls[0] without checking throws a null reference here, which is how most people discover the behaviour exists. Branch on the finish reason.

The content itself is worth reading rather than discarding, because it is usually diagnostic. “I don’t have access to live data” means the model did not believe it had a tool. “What city would you like?” means it did, and is missing a required argument. Those are different bugs.

What tool_choice auto promises

tool_choice defaults to "auto", and the documented meaning of auto is that the model may call a tool or may answer directly — the decision is the model’s. It is not a hint that gets stronger with better prompting; it is a documented branch point. A response with no tool call under auto is inside the contract.

This is the right default for an assistant that mixes conversation with actions. “Hello” should not trigger a database query, and a model that always called something would be unusable. It is the wrong default for a pipeline whose entire purpose is to produce a structured call, and using it there is the actual mistake in a large fraction of these reports.

Why the model chose text

In roughly the order worth checking:

• The tool description is thin. The model sees only the schema. A tool named fetch described as “fetches data” gives it nothing to match a user question against. Descriptions are prompt text and deserve the same care — say what the tool returns, when to use it, and when not to. A single concrete example inside the description is often the whole fix.
• A required argument is missing and unguessable. If city is required and the user said “what’s the weather like?”, a well-behaved model asks rather than hallucinating a city. That is desirable behaviour. Either make the argument optional with a sensible default, or accept the clarifying turn as part of the design.
• The system prompt is fighting the tools. A system message saying “you are a helpful assistant with no access to external systems” — often left over from before tools were added — suppresses calls very effectively. So does a long list of cautious instructions about not making things up.
• The model thinks it already knows. Ask for the capital of France with a lookup_fact tool available and it will answer from parameters, correctly. This becomes a problem only when your tool is authoritative and its knowledge is not — aget_current_price tool competing with a memorised price from training. Say so explicitly in the tool description: “always call this rather than answering from memory; prices change daily”.
• Too many tools. Selection quality degrades as the toolset grows, and a large schema block also consumes context. If you are passing thirty tools, the useful fix is usually routing — narrow the set per request — rather than prompt tuning.

Forcing a call

When a call is not optional, say so in the parameter rather than in the prompt. Mistral’s tool_choice accepts values beyond auto — including none to disable calling entirely, and a mode that requires the model to emit a call — and it also accepts naming a specific function. Compare:

# The model decides. Text is a valid outcome.
client.chat.complete(model=MODEL, messages=msgs, tools=tools,
                     tool_choice="auto")

# A tool call is required. Use this for extraction pipelines.
client.chat.complete(model=MODEL, messages=msgs, tools=tools,
                     tool_choice="any")

# No calls at all, even though tools are defined.
client.chat.complete(model=MODEL, messages=msgs, tools=tools,
                     tool_choice="none")

Forcing has a cost worth naming. A model required to call a tool will call one even when none fits, and it will invent argument values to satisfy a required field. Forcing converts “no call” into “a wrong call”, which is harder to detect. Use it where a call is genuinely always correct — a single extraction function, a structured-output shim — and leave it on auto anywhere a conversational reply is a legitimate answer.

Handling it in the loop

Whatever you set, the loop must handle both branches. The minimal correct shape:

msg = resp.choices[0].message

if resp.choices[0].finish_reason == "tool_calls":
    for call in msg.tool_calls:
        ...                      # run it, append a tool message with call.id
    # then call the model again with the results appended
else:
    return msg.content           # a text answer is a valid terminal state

Two failure modes to avoid on the else branch. Do not retry the same request hoping for a different outcome — sampling variance may eventually produce a call, but you have built a loop whose termination depends on luck, and it will occasionally run until your retry budget is gone. And do not silently return an empty result: if a text answer means the pipeline could not proceed, log the content, because that string is the model telling you which of the five causes above applies.

Where a call is required but forcing is too blunt, there is a middle option that often works better than either: keep tool_choice on auto and make the alternative unattractive. A system message that says what to do when the model cannot proceed — “if you lack an argument, call ask_user with the question rather than answering in prose” — converts the text branch into another tool call, so your loop has one shape instead of two. Giving the model a legitimate escape hatch that is still a tool is usually cheaper than fighting it into calling a tool that does not fit.

One diagnostic that settles the question quickly when you cannot tell whether the schema or the prompt is at fault: send the same request with the user message replaced by an unambiguous, fully-specified instruction — “call get_weather for Paris”. If the model calls the tool, your schema is fine and the problem is that the real user message did not read as a request for that tool, which is a description or a routing problem. If it still answers in prose with an instruction that explicit, the tool definition itself is not reaching the model in the form you think it is: check that tools is actually populated on the request, and that a stray tool_choice: "none" has not been left in a shared request builder.

The mirror-image behaviour — several calls at once rather than none — is covered in parallel tool calls in the Mistral API, and a robust loop has to handle zero, one and many from the same response.