Qwen's Max Output Tokens: Lower Than Its Context Window

A Qwen model with a 131,072-token context window will refuse to write a 131,072-token answer, and the error will not mention the context window. There are three separate limits in play and only one of them governs how long the reply can be.

Three limits, one name

Alibaba Cloud’s model list publishes these as three columns, and the confusion is entirely a result of all three being called some variation of “max tokens” in conversation:

• Context window. The total sequence the model can hold: everything you send plus everything it generates. It is a budget shared between the two, not a limit on either separately.
• Maximum input. A documented cap on the prompt alone, always a little below the context window. It exists so that the service can guarantee room for a reply — a prompt that filled the window exactly would leave nowhere to answer.
• Maximum output. The generation ceiling, and the value max_tokens is clamped to. This is much lower than the other two, and it is the one that truncates your answer.

The relationship is max_input + max_output ≤ context, but the output ceiling is set far below what that inequality would allow. There is a practical reason: generation is sequential, one forward pass per token, so a very long reply occupies a serving slot for a very long time. The ceiling is a scheduling decision as much as a model one, which is why it moves independently of the architecture.

The hosted ceilings

On Model Studio the pattern across the commercial Qwen endpoints has been an output ceiling in the region of 8,192 tokens, with the reasoning-capable configurations documented separately and higher, against context windows that range from tens of thousands to a million tokens depending on the model. The ratio is the point: a qwen-turbo endpoint with a very large context window has an output ceiling in the same range as one with a much smaller window, because the two numbers are set by different constraints.

One consequence worth internalising: sending max_tokens above the documented ceiling does not raise it. Depending on the endpoint it is either clamped silently or rejected as an invalid parameter, and neither outcome gives you a longer answer.

The reasoning-capable configurations are the case where this gets genuinely confusing, because two ceilings apply to one response. The chain of thought and the visible answer are both generated tokens and both count against the output budget, so an endpoint documented at a higher ceiling for thinking mode is not offering a longer answer — it is offering room for the reasoning that precedes it. A request that returns an empty content with a large token count and finish_reason of length has almost always spent the entire budget reasoning and been cut off before writing anything.

Open weights have no server ceiling

Run Qwen3-8B yourself and there is no service-imposed output limit at all. The constraint becomes pure arithmetic:

available_output = max_model_len - prompt_tokens

# vLLM: max_model_len comes from --max-model-len,
# capped by config.json's max_position_embeddings

Two things replace the hosted ceiling. The first is max_new_tokens in transformers, or max_tokens in an OpenAI-compatible server — the value you choose, which defaults to something small and unhelpful in several stacks. The second is the model card’s own recommendation: the Qwen3 cards suggest a generation budget in the tens of thousands of tokens for thinking mode, because a reasoning pass plus its answer genuinely needs that much room. That recommendation is not a limit; it is the number the team suggests you allow.

The trap in self-hosting is the opposite of the hosted one. Nothing stops you asking for 100,000 output tokens on a model whose --max-model-len is 32,768, and the request will fail at admission with a context-length error that names the total, not the output — which reads as a prompt problem when it is a ceiling problem.

How truncation announces itself

When the ceiling is what stopped the generation, the response says so in one field. Through an OpenAI-compatible endpoint:

"finish_reason": "length"    # hit the output ceiling
"finish_reason": "stop"      # emitted a stop token, complete
"finish_reason": "tool_calls"# stopped to call tools
"finish_reason": "content_filter"  # moderation intervened

Through the native DashScope response shape the same information is in output.finish_reason, with length carrying the same meaning. Checking it is not optional in any code that parses the output, because a truncated reply is usually still syntactically plausible right up to where it stops — truncated JSON, a half-written tool call, a sentence that ends mid-clause. Nothing about the text itself tells you it was cut off.

If finish_reason is stop but the answer looks incomplete, the ceiling is not your problem and you should be looking at stop sequences and the chat template instead — see when Qwen does not stop generating.

Streaming complicates the check without removing it. The field arrives on the final chunk, after the content is already on the user’s screen, so by the time you know the reply was truncated you have already rendered it. Handle it by keeping the last chunk’s finish_reason and reacting after the stream closes — appending a marker, or issuing a continuation — rather than by trying to detect truncation from the text. And note that a stream that ends because the connection dropped carries no finish_reason at all; treat a missing value as a failure rather than as completion, because the two are indistinguishable from the text alone.

Budgeting the window

• Reserve the output before you fill the input. Decide what the reply needs, subtract it from the window, and truncate your retrieved context to what remains. Filling the context first and hoping is how you get a 400 on your longest and most valuable requests.
• Count reasoning tokens as output. With thinking enabled the chain of thought is generated, billed and counted against the same ceiling as the answer. A budget sized for the visible reply will truncate inside the reasoning block. See the enable_thinking parameter.
• Chunk rather than raise. If you need 30,000 tokens of output from an endpoint capped at 8,192, the answer is several requests with an explicit continuation contract, not a larger number. Asking a model to produce something longer than its ceiling produces a confident truncation, not an error.
• Set the ceiling deliberately, low. max_tokens is also a cost control and a latency control. Leaving it at the maximum means one prompt that triggers a loop bills for the maximum.