Parallel Tool Calls in Qwen

Qwen will happily emit three tool calls in a single assistant turn. Matching the three results back to the three calls is trivial through an OpenAI-compatible server and genuinely ambiguous if you are parsing the raw chat template, because only one of those two representations has identifiers in it.

What a parallel call looks like

Through any OpenAI-compatible front end — Alibaba Cloud Model Studio, vLLM with a tool-call parser, SGLang, or a gateway — a multi-call turn arrives as an array on the assistant message. Given a request with two tools defined and a prompt like “compare the weather in Hangzhou and Shenzhen”:

{
  "choices": [{
    "finish_reason": "tool_calls",
    "message": {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "id": "call_a3f1c9d2b7e4",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"city\": \"Hangzhou\"}"
          }
        },
        {
          "id": "call_5d8b1e0af632",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"city\": \"Shenzhen\"}"
          }
        }
      ]
    }
  }]
}

Three properties of that response are worth naming. finish_reason is tool_calls, not stop — branch on that field rather than on whether tool_calls is present, because a model can emit text and calls in the same turn. arguments is a string containing JSON, not a JSON object, so it needs a second parse. And the two calls have the same name, which is the case that breaks any dispatcher keyed on function name instead of on call id.

The raw template has no call ids

Underneath, Qwen’s chat template does not produce that JSON. It produces text. Qwen2.5 and Qwen3 use a Hermes-style tool convention inside their ChatML template, and a parallel call is simply two tagged blocks in a row within one assistant turn:

<|im_start|>assistant
<tool_call>
{"name": "get_weather", "arguments": {"city": "Hangzhou"}}
</tool_call>
<tool_call>
{"name": "get_weather", "arguments": {"city": "Shenzhen"}}
</tool_call><|im_end|>

There is no id anywhere in that. The call_a3f1c9d2b7e4 in the previous section was generated by the server’s tool-call parser at the moment it converted this text into OpenAI-shaped JSON — it is a transport-layer identifier, not something the model chose.

The consequence is the load-bearing fact of this page: in the raw format, calls are matched to results by order. Position one of your results corresponds to position one of the calls. If you are driving Qwen through the raw ChatML template — a local transformers loop, a fine-tuning data pipeline, or anything that builds the prompt string itself — you must preserve that order, because nothing else encodes it.

Sending the results back

The two representations diverge again on the return leg, and this is where a hand-rolled loop most often produces a model that repeats itself or hallucinates the answer it was given.

Through the OpenAI-compatible API you append the assistant message verbatim, then one message per call, each with role: "tool" and a tool_call_id matching the id you were sent:

messages = [
    *history,
    assistant_message,                       # exactly as returned
    {"role": "tool", "tool_call_id": "call_a3f1c9d2b7e4",
     "content": '{"temp_c": 24, "cond": "cloudy"}'},
    {"role": "tool", "tool_call_id": "call_5d8b1e0af632",
     "content": '{"temp_c": 29, "cond": "clear"}'},
]

In the raw template the results are not separate messages at all. They are tool_response blocks inside a single user turn, in the same order as the calls:

<|im_start|>user
<tool_response>
{"temp_c": 24, "cond": "cloudy"}
</tool_response>
<tool_response>
{"temp_c": 29, "cond": "clear"}
</tool_response><|im_end|>
<|im_start|>assistant

Two tool_response blocks, one user turn. Splitting them into two user turns is the most common hand-rolled mistake: it produces a conversation shape the model never saw in training, and the usual symptom is the model re-issuing the tool calls it just made.

Where this goes wrong

• Dispatching on function name. Two calls to get_weather in one turn is the common case, not the exotic one. Key your result map on id, or on index when there is no id.
• Dropping a result. Every call needs a response, including the ones that errored. Return the error as the tool content — {"error": "city not found"} — rather than omitting the message. A missing tool_call_id is a validation error on most OpenAI-compatible servers and a silent context corruption on none-of-them-at-all in the raw format.
• Reordering for concurrency. Running the two tools in parallel is exactly what the feature is for, but the results must be reassembled in call order before they go back. An asyncio.gather preserves order; a completion-ordered queue does not.
• Truncation mid-call. If the output ceiling lands in the middle of the second tool_call block you get one valid call and one fragment, and finish_reason will be length rather than tool_calls. Check it before parsing.

What this looks like streaming

Streaming a parallel call is where most implementations break, because the calls arrive interleaved and incomplete. Each delta carries an index identifying which call in the array it belongs to, and arguments arrives as a sequence of string fragments that mean nothing until concatenated:

data: {"choices":[{"delta":{"tool_calls":[{"index":0,
  "id":"call_a3f1c9d2b7e4","function":{"name":"get_weather",
  "arguments":""}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,
  "function":{"arguments":"{\"city\":"}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":1,
  "id":"call_5d8b1e0af632","function":{"name":"get_weather",
  "arguments":""}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,
  "function":{"arguments":" \"Hangzhou\"}"}}]}}]}

Note that index 1 opened before index 0 finished. Accumulate into a dictionary keyed on index, appending argument fragments in arrival order, and only attempt json.loads once the stream has ended. Parsing on each delta to see whether the JSON is complete yet works until an argument value legitimately contains a brace, at which point it does not.

Turning it off

Some workflows want one call at a time — an agent that must observe the result of step one before choosing step two, or a tool with side effects that must not fire twice concurrently. There is no model-level switch for this in the open weights; parallelism is a behaviour of the trained model, not a parameter.

What you have instead is the OpenAI-compatible parallel_tool_calls flag, where the server you are talking to supports it, and the blunter and more reliable option of ignoring every call after the first and returning a single result. The second works everywhere, at the cost of a wasted generation. A system prompt asking for one tool at a time helps and is not a guarantee — treat it as a preference the model usually honours, and make the loop correct for the case where it does not.

Discarding the extra calls needs one piece of care. You cannot simply drop them from the assistant message you append to the history, because the message you send back must be consistent: an assistant turn listing two calls followed by one tool result is a conversation the model has not seen and will react badly to. Either rewrite the assistant message to contain only the call you honoured, or answer every call and discard the results you did not want. Rewriting is cheaper and is what most single-call-at-a-time agents do.

The related lever is tool_choice, which is about whether rather than how many: "auto" lets the model decide, "none" forbids calls entirely, and naming a specific function requires that one. Forcing a named tool is the reliable way to get exactly one call, and it is why the tool-call route to structured output described in Qwen’s JSON output modes works as well as it does.