Stop Sequences in the xAI API

The stop parameter is four strings and one large caveat: on xAI’s reasoning models it is not supported, which is most of the current lineup.

The parameter

xAI’s API reference documents stop on the chat completions endpoint as an array of up to four sequences that halt generation, and notes that it is unsupported by reasoning models.

{
  "model": "grok-4.3",
  "messages": [
    { "role": "system", "content": "Continue the transcript. Write only the CUSTOMER line." },
    { "role": "user", "content": "AGENT: Good morning, how can I help?\nCUSTOMER:" }
  ],
  "stop": ["\nAGENT:", "\n\n"],
  "max_completion_tokens": 200,
  "reasoning_effort": "none"
}

Four is a small budget and it is a per-request budget, not per sequence-length. A fifth entry is a request-shape error, not a silently-ignored extra, so build the array rather than appending to it from several places in your code.

The matching of these is on the token stream, not on your source text, which is the root of most surprises further down this page. And the parameter is a generation control, not a formatting instruction: nothing about stop tells the model to produce output with that shape. It only cuts.

The sequence is trimmed from the output

When generation halts on a supplied sequence, the sequence does not appear in content. This is the OpenAI convention and it is what the field means: the stop string is a terminator, not part of the answer.

# request stop: ["\nAGENT:"]

"choices": [
  {
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "I was charged twice for the same order last Tuesday."
    },
    "finish_reason": "stop"
  }
]

Two consequences. If you need the terminator — because you are reconstructing a transcript, or because a downstream parser expects the delimiter — you have to append it yourself, and it is easy to forget that the round trip is lossy.

More awkwardly, finish_reason is stop in two different situations: the model reached a natural end, and the model hit your sequence. xAI documents stop as covering a model-defined or a user-supplied stop sequence, so the field alone cannot tell you which happened. The other documented values are length for a token ceiling — see max output tokens — and end_turn or null on non-final streaming chunks. If the distinction matters, infer it from the content: a natural ending is usually a complete sentence, a truncated one usually is not.

Reasoning models do not support it

This is the constraint that makes the page. xAI’s reasoning documentation states that presence_penalty, frequency_penalty and stop cannot be used with reasoning models, and the API reference repeats it on each parameter.

The mechanism is straightforward once you see the ordering. A reasoning model generates an internal trace and then the answer, in one continuous decode. A stop sequence matching against that stream would fire inside the trace — where your delimiter is quite likely to appear, since the trace discusses the format it is about to produce — and terminate the response before any answer existed. Rather than expose that trap, the parameter is refused.

Since grok-4.3 and grok-4.5 both support reasoning, the practical question is which mode you are in. reasoning_effort: “none” is the documented value for no reasoning on grok-4.3, and the non-reasoning snapshot slugs such as grok-4.20-0309-non-reasoning exist as separate models. If your pipeline depends on stop, that dependency is now also a constraint on which model and which effort setting you may use — and that is a good enough reason to redesign around it.

The failure is a request-time rejection rather than a behaviour change, which is the good version of this problem: you find out on the first call, not in production three weeks later. It is worth making sure that is actually true of your code, though. If stop is set once in a shared client wrapper and the model id is configuration, then switching model turns a working route into a 400 without any change to the file that set the parameter. Build the request body per call site, or make the wrapper drop stop when the configured model reasons.

The obvious workaround — let the model run and truncate the string yourself when you see the delimiter — is worth considering honestly, because it is not strictly worse. It costs more: you are billed for every token generated after the point you would have stopped, and on a streaming response you may have already shown some of them. But it is portable across every model and every provider, it survives a switch to a reasoning model, and it cuts on your text rather than on a token boundary, which removes the whitespace and alignment problems below entirely. For short outputs, where the tokens after the delimiter are few, it is usually the better engineering decision.

Where stop sequences fail

• Token boundaries, not character boundaries. Matching happens over tokens. A sequence that does not align with how the tokenizer segments the text can behave unexpectedly at the edges, which is one reason short punctuation-only sequences are less reliable than a distinctive multi-character string.
• Whitespace is significant and invisible. “\nAGENT:”, “AGENT:” and “ AGENT:” are three different sequences. Most “my stop sequence did not work” reports are one of these three being the wrong one.
• It cannot rescue malformed structure. Stopping at the first closing brace of a JSON object does not give you valid JSON; it gives you the object truncated at its first nested close brace.
• It does not shorten the bill in the way people expect. You are charged for the tokens generated up to and including the match. Stopping early saves the tokens after it, not the ones before.
• It is not portable. Four sequences here, a different budget and a different reported reason at Anthropic, and different behaviour again elsewhere. Anything routing across providers should treat the array as per-provider configuration.

What to use instead

Most uses of stop in modern code are a workaround for something that now has a first-class solution.

If you are stopping to cut off a model that keeps writing past the answer, the fix is usually a firmer system message and a tight max_completion_tokens. If you are stopping to extract structure from prose, use schema-constrained output — a constrained decode cannot produce the trailing commentary you were trying to cut off in the first place. If you are stopping to end a turn in a completion-style transcript, consider whether the message array already expresses what the delimiter was standing in for.

The place stop is still exactly right is few-shot completion against a fixed textual format on a non-reasoning model, where the delimiter is genuinely part of the format and the model has been shown it several times. That is a narrower set of cases than the parameter’s prominence suggests.