GPT-4o-mini's Context Window and Output Cap

The context window and the output cap are two separate documented limits and people routinely conflate them. One bounds everything the model can see; the other bounds only what it can write in a single reply, and it is more than seven times smaller.

The two numbers

As documented by OpenAI on its model reference page for gpt-4o-mini at the time of writing:

• Context window: 128,000 tokens. This is the total budget for the request. System message, every prior turn, tool definitions, tool results, images converted to tokens, and the reply — all of it inside 128,000.
• Maximum output tokens: 16,384. The most the model will produce in one completion, regardless of how much of the window is free.
• Training data cutoff: October 2023. Not a limit, but it belongs next to them when you are deciding whether the model can do a job — see what a knowledge cutoff does and does not mean.

Why they are not independent

The 16,384 is a cap, not an allocation. The two limits compose as a minimum: what you can actually get back is the smaller of the output cap and the space left in the window.

usable_output = min(16_384, 128_000 - prompt_tokens)

With a 5,000-token prompt, you can have the full 16,384. With a 120,000-token prompt, you can have 8,000 — the cap is irrelevant because the window bound is tighter. And with a 128,000-token prompt you get nothing at all; the request fails before generation with a context-length error rather than returning an empty completion.

This is why long-document summarisation with a long output is harder than either half suggests. The document competes with the summary for one budget, and the failure at the boundary is a 400, not a truncation. If you want a guaranteed output length, subtract it from the window before you decide how much document to send, not after.

What a full window costs

OpenAI publishes GPT-4o-mini pricing per million tokens. At the rates documented at the time of writing — $0.15 per million input tokens and $0.60 per million output tokens — the per-1,000-token figures people usually want are:

input   $0.15 / 1,000,000  = $0.00015 per 1K tokens
output  $0.60 / 1,000,000  = $0.00060 per 1K tokens

Now the largest single request the limits permit. Assumptions, stated: a prompt that fills the window right up to the point where the full output cap still fits, and an answer that uses all of it.

prompt  = 128,000 - 16,384 = 111,616 input tokens
output  =                     16,384 output tokens

input   111,616 / 1,000,000 x $0.15 = $0.016742
output   16,384 / 1,000,000 x $0.60 = $0.009830
                                      ---------
total per request                     $0.026573

Under three cents for the largest request the model will accept. That is the number worth carrying around, because it reframes the usual worry: for this model, at these rates, a single maximum-size call is cheap and a hundred thousand of them is $2,657. The cost problem with a long context is never one request, it is the multiplier, and the multiplier is usually a conversation that resends its whole history every turn.

Which is the other half of the arithmetic. A twenty-turn conversation that reaches 100,000 tokens does not cost one prompt; it costs the running sum of every prefix, because each turn resends everything. That growth is quadratic in turn count and is the reason automatic prompt caching exists and is worth structuring your prompt for.

What exceeding it looks like

The two limits fail differently, and telling them apart from the error alone saves a wrong fix. Overflowing the window is a 400 before anything is generated:

{
  "error": {
    "message": "This model's maximum context length is 128000 tokens. However, your messages resulted in 131204 tokens. Please reduce the length of the messages.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}

The message is unusually helpful: it names both the limit and your actual count, so you know exactly how much to shed rather than guessing. Match on code: “context_length_exceeded” and parse nothing out of the prose.

Exceeding the output cap is not an error at all. Ask for more than 16,384 in max_tokens and, depending on the model and the endpoint, you either get a 400 naming the parameter or you get a successful 200 that simply stops at the cap with finish_reason: “length”. The second is the dangerous one, because a truncated answer looks like a complete answer to any code that does not check — check the finish reason on every response. And note that leaving max_tokens unset does not mean unlimited; it means the endpoint’s own default, which is a different number again and is covered in what max_tokens defaults to.

Where the real ceiling is

Three reasons the practical limit sits below 128,000 tokens.

• Time to first token grows with prompt length. Prefill is one pass over the whole prompt. A 110,000-token prompt has a noticeably slower first token than a 5,000-token one, and for anything a human is watching that is the number they experience.
• Retrieval quality across a full window is workload-specific. That a model accepts 128,000 tokens is a statement about what it will process, not a promise about how reliably it will use material buried in the middle. Measure this on your own task rather than assuming either the optimistic or the pessimistic version.
• Your own retry budget. A request near the ceiling has no room for the retry that appends an error message and asks again. Leaving 10% free is the difference between a recoverable failure and a dead end.

Checking these yourself

Do not estimate what the API will tell you for free. Every response carries a usage object, and it is ground truth for the request that just happened:

"usage": {
  "prompt_tokens": 111616,
  "completion_tokens": 16384,
  "total_tokens": 128000,
  "prompt_tokens_details": { "cached_tokens": 0 }
}

Log prompt_tokens and completion_tokens per request from the start. The distribution of prompt sizes across real traffic tells you whether the window is anywhere near binding, and it is usually the answer to “should we move to a bigger model?” — most workloads that worry about context are nowhere near it, and most that exceed it do so through conversation history rather than through any one document.

Two habits make that log worth keeping. Record the p95 of prompt_tokens per route rather than the mean, because a context problem is always a tail problem: the mean stays comfortable while a small fraction of requests sits against the ceiling and fails. And record prompt_tokens_details.cached_tokens beside it, because the same number that tells you how close you are to the window also tells you how much of the prompt you are paying full price for on every call. A long prompt with a high cached ratio is a very different cost situation from a long prompt with none, and the two are indistinguishable if you only log the total.

What the object will not tell you is what an image, a tool definition or a retrieved document each contributed, since everything arrives pre-summed into prompt_tokens. Attributing the total to its parts means sending the parts separately once and subtracting — worth doing once per prompt template, not continuously.