The n Parameter: Multiple Completions in One OpenAI Request

n asks the Chat Completions endpoint for several independent completions of one prompt in a single call. It is one of the few parameters whose cost is exactly and obviously linear, and one of the few whose response shape most client code silently ignores.

What n does

n is an integer, default 1. Set it to 5 and the server runs the sampler five times over the same prompt, producing five independent continuations, and returns all of them in one response. The prompt is processed once — one prefill — and five decode loops run against the resulting key-value cache.

The word to hold on to is independent. Each completion is a fresh walk through the sampler from the same starting distribution; nothing is conditioned on what the other completions produced, and nothing prevents two of them from being identical. That is what makes n useful for self-consistency — the agreement between independent draws is meaningful precisely because they could not see each other — and it is also why n is no help at all when you want variety. Asking for five different phrasings is a prompt instruction, not a sampling parameter.

The completions are independent draws, not ranked variants. Nothing orders them by quality and nothing deduplicates them. At temperature: 0 the sampler takes the argmax every time, so asking for five completions gets you five copies of approximately the same text at five times the output cost, which is the single most common way this parameter is wasted. n is only meaningful with a stochastic sampler.

The response shape

The response object is unchanged except that choices has n entries, each with its own index, its own message and its own finish_reason. This is the shape, with the message bodies shortened:

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "gpt-4o-2024-08-06",
  "choices": [
    { "index": 0,
      "message": { "role": "assistant", "content": "A quiet lamp on..." },
      "finish_reason": "stop" },
    { "index": 1,
      "message": { "role": "assistant", "content": "Rain against the..." },
      "finish_reason": "stop" },
    { "index": 2,
      "message": { "role": "assistant", "content": "The harbour lights..." },
      "finish_reason": "length" }
  ],
  "usage": {
    "prompt_tokens": 1200,
    "completion_tokens": 1180,
    "total_tokens": 2380
  }
}

Two details in there are easy to miss. First, finish_reason is per choice — choice 2 above hit max_tokens while the others ended naturally, which is normal and means you must check the field on every element rather than on the first. Each value has a different recovery. Second, usage is not per choice. There is one usage object for the whole request and completion_tokens is the sum across all n completions, so you cannot attribute cost to an individual choice from the response alone.

Streaming makes the same structure harder to handle. Each chunk carries a choices array too, and the deltas for the different completions are interleaved in arrival order. The index field on each delta is the only thing telling you which completion it belongs to, so a client that appends choices[0].delta.content— which is what almost every streaming example does — will splice fragments of five different answers into one string. With n above 1 you must demultiplex into n buffers keyed on index.

Deriving the cost of raising n

The billing rule is stated plainly in the API reference: you are charged based on the number of generated tokens across all of the choices. So the prompt is billed once and the completions are billed n times. Written out, with P the prompt tokens, C the average completion length, and the two per-token rates:

cost(n)  =  P × rate_in  +  n × C × rate_out

prompt_tokens      = P          (flat in n)
completion_tokens  = n × C      (linear in n)

Put numbers on it. A prompt of 1,200 tokens and completions averaging 400 tokens: at n = 1 you are billed 1,200 in and 400 out; at n = 5 you are billed 1,200 in and 2,000 out. The input side did not move at all. Whether that is cheap or expensive depends entirely on the ratio of the two rates and on the ratio of prompt to completion, and output tokens are the dearer of the two on every OpenAI model published to date.

The saving relative to five separate requests is therefore exactly the four extra copies of the prompt you did not pay for: 4 × 1,200 = 4,800 input tokens. On a long-prompt, short-answer workload — a classification prompt with a large rubric, say — that is most of the bill. On a short-prompt, long-answer workload it is almost nothing, and n is close to a pure multiplier.

Where n is unavailable

n belongs to Chat Completions. It has no equivalent on the Responses API, which returns a single output per call by design — if you need k samples there you make k requests. The legacy /v1/completions endpoint has both n and a separate best_of, where best_of generates candidates server-side, returns only the highest-log-probability one, and bills you for all of them; best_of was never carried over to Chat Completions.

Several newer models restrict the parameter rather than removing it, and a rejected value comes back as a 400 naming n in the param field. Because the set of models that do this changes, the durable advice is to treat n > 1 as a capability to probe rather than assume, especially on reasoning models where multiple samples would multiply the reasoning tokens as well as the visible output.

When n beats parallel requests

The two ways of getting five samples are not equivalent, and the difference is not only cost.

• One prefill versus five. n reuses the prompt’s computed keys and values across all the completions. Five separate requests each pay their own prefill in latency as well as in tokens.
• One failure domain. n gives you five completions or an error. Five requests can return three successes, a rate-limit error and a timeout, which is more code but also more partial availability.
• Latency is the slowest completion. The response does not return until every choice has finished, so a single choice that runs to max_tokens sets the latency for all of them. Parallel requests let you take the first three that land.
• Rate limits count differently. One request with n: 5 is one request against your requests-per-minute limit but roughly five completions’ worth against tokens-per-minute. Which limit you are near decides which shape is safe.

There is one more interaction worth knowing before you reach for it. n composes with the rest of the request in ways that are not always what you expect: with logprobs enabled you get a full logprob structure per choice, which multiplies the response size as well as the token count; with a json_schema response format every choice is independently constrained, so you get n valid but different objects rather than n attempts at one object; and with tools attached, different choices can decide to call different tools, or some to call a tool and others to answer in prose. That last case is genuinely useful for measuring how confident a model is about taking an action — if four of five samples call the tool, that is a signal you can threshold on — and it is genuinely dangerous if your code assumes every choice has the same shape.

The workloads where n earns its place are self-consistency voting, candidate generation for a re-ranker, and anything where you want several drafts of a long prompt cheaply. For everything else, leave it at 1 — the reference’s own advice — and reach for it only when you can say which of the four properties above you are buying.