Skip to content
Multigrid

The Shape of a Parallel Tool Call Response in the OpenAI API

9 min read · updated August 11, 2026

When a model decides it needs three functions to answer, it does not return three responses. It returns one assistant message whose tool_calls is an array of three objects, and the protocol from that point on is unforgiving about what you send back.

The response

Give the model tools for weather, timezone and currency, ask it something that needs all three, and this is the shape that comes back:

{
  "id": "chatcmpl-B1a2b3c4",
  "object": "chat.completion",
  "model": "gpt-4o-2024-08-06",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_9xKf2mQ1",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\":\"Lisbon\",\"units\":\"celsius\"}"
            }
          },
          {
            "id": "call_7pLd4nR8",
            "type": "function",
            "function": {
              "name": "get_local_time",
              "arguments": "{\"city\":\"Lisbon\"}"
            }
          },
          {
            "id": "call_2wTy6vS3",
            "type": "function",
            "function": {
              "name": "convert_currency",
              "arguments": "{\"from\":\"GBP\",\"to\":\"EUR\",\"amount\":50}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

Field by field

  • finish_reason: "tool_calls" — the signal that this turn is a request for tools rather than an answer. Branch on this, not on whether tool_calls is present. The older "function_call" value belongs to the deprecated single-function interface.
  • content: null — usually null on a tool-calling turn, and not always. A model can emit prose and tool calls in the same message, so a client that assumes null and skips the field will silently drop text the user should have seen.
  • tool_calls — an array, always, even for one call. The order is the order the model produced them and carries no dependency information: these are calls the model believes can be made independently. If B needs A’s result, the model is expected to ask for A this turn and B the next.
  • id — an opaque per-call identifier, unique within the request. This is the join key for the whole protocol. Every result you send back must carry the exact id of the call it answers.
  • type: "function" — the discriminator. Function is the value you will see for a function tool; the field exists because other tool types share the envelope.
  • function.name — matches a name from the tools you supplied. Validate it against your registry rather than dispatching on it directly; a name that is not in your map is a bug or an injection, not a function to call.
  • function.arguments — a JSON string, not an object. This is the field that surprises everyone once. It must be parsed. And unless the tool was declared with strict: true, the model is not guaranteed to have produced arguments that satisfy your schema — required fields can be missing, types can be wrong, and hallucinated extra keys are possible. Strict mode is what makes that guarantee, and it has requirements of its own: see strict mode in function calling.

What you must send back

The next request repeats the whole conversation, including the assistant message with its tool_calls intact, followed by one role: "tool" message per call:

"messages": [
  {"role": "user", "content": "What's it like in Lisbon, and what's £50 in euros?"},

  {
    "role": "assistant",
    "content": null,
    "tool_calls": [ /* all three, exactly as received */ ]
  },

  {"role": "tool", "tool_call_id": "call_9xKf2mQ1",
   "content": "{\"temp_c\":19,\"conditions\":\"clear\"}"},
  {"role": "tool", "tool_call_id": "call_7pLd4nR8",
   "content": "{\"local_time\":\"2026-08-11T14:02:00+01:00\"}"},
  {"role": "tool", "tool_call_id": "call_2wTy6vS3",
   "content": "{\"result\":58.10,\"rate\":1.162}"}
]

Four rules govern that block, and three of them are enforced with a 400:

  1. Every call gets a result. Omit one and the request is rejected with a message about an assistant message with tool_calls requiring a response for each call. A failed tool is not an excuse to omit it — send the error as the content.
  2. The ids must match exactly. A tool_call_id that does not correspond to a call in the preceding assistant message is an error. Do not generate your own.
  3. The assistant message must be included. Tool results with no preceding tool call is a malformed conversation.
  4. Content is a string. Serialise your result. There is no structured field, so JSON.stringify it — and keep it small, because every tool result is prompt tokens on this turn and on every subsequent turn of the conversation.

The upside of the array is real: three tools invoked from one round trip is one prefill instead of three, and if your handlers are independent you can run them concurrently. The latency win is roughly the difference between the sum and the maximum of your tool durations, plus two saved round trips.

The failure shape is worth recognising because it does not look like an id problem when it arrives. You wrote the loop against a single tool call, then a model started returning three, and your code took tool_calls[0], executed it, and replied with one tool message. The next request is rejected:

HTTP/1.1 400 Bad Request

{
  "error": {
    "message": "An assistant message with 'tool_calls' must be followed by
                tool messages responding to each 'tool_call_id'. The
                following tool_call_ids did not have response messages:
                call_7pLd4nR8, call_2wTy6vS3",
    "type": "invalid_request_error",
    "param": "messages.[3].role",
    "code": null
  }
}

Note where the error points: at the messages array of the request you just sent, not at anything about tools. And note that it arrives one turn late — the request that produced three calls succeeded fine, and the failure surfaces on the follow-up, which is why the first instinct is usually to look at the wrong request. The rule the message is enforcing is total coverage: every id, every time, including the ones whose handlers threw.

The obvious workaround for that — dropping the failed call from the assistant message so there is nothing to answer — does not work either. Editing the assistant message you received puts a fabricated turn into the history, and the model’s next decision is now conditioned on a record of a request it did not make. Send the failure as content instead: {"error": "timeout after 5s"} is a perfectly good tool result, and models handle it well — a failed tool is information, and the usual response is to retry differently or to tell the user plainly, both of which are better than the alternative.

The second common failure is matching results to calls by function name or by position rather than by id. It works right up until the model asks for the same function twice in one turn — two get_weather calls for two different cities is an entirely ordinary thing for it to do — at which point a name-keyed map silently collapses them and one city gets the other’s temperature. There is no error, the conversation continues, and the answer is confidently wrong. The ids exist precisely because names are not unique within a turn.

The same thing, streamed

Streaming reassembles the array from fragments, and the reassembly is keyed on an index field that only exists in the streamed form:

data: {"choices":[{"delta":{"role":"assistant","tool_calls":[
  {"index":0,"id":"call_9xKf2mQ1","type":"function",
   "function":{"name":"get_weather","arguments":""}}]}}]}

data: {"choices":[{"delta":{"tool_calls":[
  {"index":0,"function":{"arguments":"{\"city\":"}}]}}]}

data: {"choices":[{"delta":{"tool_calls":[
  {"index":0,"function":{"arguments":"\"Lisbon\"}"}}]}}]}

data: {"choices":[{"delta":{"tool_calls":[
  {"index":1,"id":"call_7pLd4nR8","type":"function",
   "function":{"name":"get_local_time","arguments":""}}]}}]}

The id and name arrive once, on the first fragment for that index. Everything after is arguments string fragments to be concatenated in arrival order. Two consequences: you cannot parse arguments until the stream for that index is finished, and you must accumulate into a map keyed by index rather than appending to an array in arrival order, because fragments for index 0 and index 1 can interleave. The general chunk anatomy is in the streaming chunk format.

Turning it off

Set parallel_tool_calls: false and the model returns at most one call per turn. The array is still an array; it has one element. Reasons to do this: your tools have side effects whose ordering matters, your execution layer is not concurrent, or you are debugging an agent loop and want one decision per step to read the trace.

Note that this restricts the model rather than fixing the client — the reply protocol is identical, so code written for the array works either way, and code written for a single call breaks the moment somebody flips the flag back. Write for the array.

Related