OpenAI's max_tokens Default and Why Omitting It Truncates Output

Your answer stops mid-sentence. You did not set max_tokens, so you assumed there was no limit. There is always a limit; omitting the parameter chooses a different one rather than removing it, and which one you got depends on the endpoint you called.

The symptom

The response is a normal 200. Nothing throws. The content is real text that simply ends:

"choices": [{
  "message": { "role": "assistant", "content": "To configure the connection pool, set the" },
  "finish_reason": "length"
}],
"usage": { "prompt_tokens": 312, "completion_tokens": 16, "total_tokens": 328 }

finish_reason: "length" is the diagnosis and it is the only thing distinguishing this from a short answer. completion_tokens is the second clue: when it lands on a round number — 16, 256, 1024 — that is a cap being hit, not a model finishing. A model that finished naturally would have no reason to stop on a power of two.

The default, per endpoint

There is no single answer, and this is the whole reason the question is confusing. Three endpoints, three behaviours.

• Legacy /v1/completions: max_tokens defaults to 16. This is the documented default in the reference for the legacy completions endpoint, and it is the source of the classic version of this bug — sixteen tokens is about a dozen words, so the output is obviously and dramatically truncated. If your completion_tokens is exactly 16, this is what happened.
• /v1/chat/completions: optional, no small default. Omit it and generation continues until the model stops on its own or runs out of room. “Out of room” is the next section, and it is where the truncation people report on this endpoint actually comes from. Note also that max_tokens here is deprecated in favour of max_completion_tokens, which is the parameter new code should use.
• /v1/responses: max_output_tokens, optional. Same shape as chat completions — unset means bounded by the model’s own output cap and by what the context leaves.

So the phrase “the max_tokens default” only has a concrete numeric answer on the legacy endpoint. On the modern ones the default is not a number at all; it is a formula, and you compute it.

The invisible ceiling: what the context leaves

Prompt and completion share one budget. The effective maximum output for any request is the smaller of two things:

effective max output  =  min( model's max output tokens,
                             context window − prompt_tokens )

Both terms are per-model and both are published on the model page. Worked on a model with a 128,000-token context and a 16,384-token output cap:

prompt =   2,000 tokens  →  min(16,384, 126,000) = 16,384   the output cap binds
prompt = 120,000 tokens  →  min(16,384,   8,000) =  8,000   the context binds
prompt = 127,000 tokens  →  min(16,384,   1,000) =  1,000   almost nothing left

This produces a failure mode with a very characteristic shape: the feature works fine in development and truncates in production, because production prompts carry a longer conversation history, a bigger retrieved context, or a fatter tool result. Nothing in your code changed. The prompt grew, so the answer’s ceiling fell. A conversation that re-sends its whole transcript walks into this on its own, one turn at a time.

The reverse mistake is also common and produces a hard error rather than a truncation: setting max_tokens to a value that, added to the prompt, exceeds the context window. That is a 400 telling you the messages resulted in more tokens than the model can handle, and it is the API refusing a request it can see cannot fit rather than starting one that would fail part-way.

Reasoning models, where the budget disappears

On o-series models the parameter behaves differently in two ways, and the second one produces the strangest version of this bug.

First, max_tokens is not accepted at all — it returns a 400 with code: "unsupported_parameter" naming it, and you must use max_completion_tokens.

Second, and this is the important one: reasoning tokens count against that budget. The model thinks before it answers, those thinking tokens are billed, and they are not returned to you. So a budget that is too small can be consumed entirely by reasoning, leaving nothing for the visible answer:

"choices": [{
  "message": { "role": "assistant", "content": "" },
  "finish_reason": "length"
}],
"usage": {
  "completion_tokens": 1000,
  "completion_tokens_details": { "reasoning_tokens": 1000 }
}

An empty string, a thousand tokens billed, and finish_reason: "length". Read literally that is exactly correct and it is nothing like what most code expects. The correct response is to raise max_completion_tokens substantially — OpenAI’s reasoning guidance suggests reserving a large allowance on top of the answer you actually want — not to retry the prompt, which will produce the same empty answer at the same cost.

It is worth being precise about what the parameter is and is not. It is a hard ceiling on generation, not a target and not a hint. Setting it to 4,000 does not encourage a longer answer and does not reserve anything; if the model finishes in 200 tokens you are billed for 200. Length is controlled by the prompt, and the cap only decides where an answer that would have continued gets cut off. This matters because the two are routinely confused in the other direction as well — raising max_tokens to make answers more thorough does nothing at all, and lowering it to make them shorter produces truncated answers rather than concise ones.

Fixing it

1. Check finish_reason on every response. Nothing else in this list matters if you are not doing this. A truncated answer is indistinguishable from a complete one by inspection, and the field is the only signal.
2. Set max_completion_tokens explicitly. Pick a number from what the feature needs — an extraction that returns a short object needs 300, not 4,000 — and get a deterministic ceiling rather than an emergent one. An explicit cap is also a cost control: it bounds the worst case of a model that decides to write an essay.
3. Compute the headroom before sending. Estimate the prompt tokens, subtract from the model’s context window, and if the remainder is below what the answer needs, shorten the prompt rather than hoping. Trimming conversation history and filtering tool results are the two levers with the most slack in them.
4. On reasoning models, budget for the invisible tokens. Allow for a substantial reasoning allowance above the answer length, and log usage.completion_tokens_details.reasoning_tokens so you can calibrate it against real traffic instead of guessing twice.
5. If you genuinely need more than the output cap, chunk. No parameter raises it. Split the work — section by section, record by record — and assemble the results yourself. A single request cannot return more than the model’s published maximum output no matter what you set.

Two variations on the symptom are worth separating from the ones above, because their fixes are different. If you are streaming, a cap being hit looks exactly like a stream that stopped — the deltas simply cease and the terminal chunk carries finish_reason: "length". Since streamed responses omit usage unless you ask for it with "stream_options": { "include_usage": true }, you may have neither the count nor the reason unless you deliberately captured both. And if you passed a stop sequence, an answer that ends early with finish_reason: "stop" is not this bug at all: the model produced your delimiter, and the matched text is stripped from content, so a stop sequence that appears naturally in the output truncates it silently and with a completely reassuring finish reason. Rule that out before raising any budget.

What does not help: retrying the identical request, which truncates at the identical place; raising max_tokens above the model’s output cap, which is silently ineffective; or asking the model in the prompt to be complete, which does not change any of the three ceilings above.