DeepSeek-V3's Context Window and Output Limit

DeepSeek-V3 is published with a 128K context window. That single number answers less than it looks like it does, because three different limits get quoted under the same name and only one of them is 128K.

The documented window

The DeepSeek-V3 model card on Hugging Face states a maximum context length of 128K tokens, and the DeepSeek-V3 Technical Report, published by DeepSeek-AI in December 2024, describes how that window was produced: the model was pre-trained at a 4K sequence length and then extended in two subsequent stages using YaRN, first to 32K and then to 128K. The report is the primary source for the training procedure, and the model repository is the primary source for the figure shipped with the weights.

That two-stage extension is worth knowing rather than skipping, because it tells you what kind of number 128K is. It is not the length the model was trained at from scratch. It is a length the model was adapted to, by rescaling positional encodings and continuing training on longer sequences. Models extended this way generally behave well across the extended range and still tend to be strongest well inside it. Treat the documented window as the point at which requests are rejected, not as a promise of uniform quality across every position in it.

Where the number is recorded

If you are self-hosting, the authoritative value is not the model card prose. It is max_position_embeddings in the repository’s config.json, because that is the field your serving stack reads. vLLM, SGLang, TGI and Hugging Face transformers all derive the window from the config, and any of them will refuse a request that exceeds it — or, if you have overridden it, will happily accept one and produce degraded output.

python - <<'PY'
import json, urllib.request
url = "https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json"
cfg = json.load(urllib.request.urlopen(url))
print("max_position_embeddings:", cfg["max_position_embeddings"])
print("vocab_size:", cfg["vocab_size"])
print("rope_scaling:", cfg.get("rope_scaling"))
PY

The rope_scaling block is the interesting part of that output. Its presence is the YaRN extension recorded in machine-readable form — a factor, an original context length, and the parameters governing how positions beyond that original length are handled. If you deploy the weights with a stack that ignores rope_scaling, you get a model that accepts long inputs and loses coherence in the extended region, which is a far more confusing failure than a rejected request.

Output is a separate, much smaller limit

The most common misreading of “128K context” is that it licenses a 128K answer. It does not. The context window bounds the total of input plus output; the output has its own ceiling, set by max_tokens, and on the hosted API that ceiling is one to two orders of magnitude smaller than the window. The chat endpoint and the reasoning endpoint carry different defaults and different maxima — the documented values are tabled separately because they change independently of the window.

The practical consequence: a 128K window is a budget for what you put in. You can load an entire codebase or a long transcript and ask a question about it. You cannot ask the model to rewrite that codebase in one response, because the response is capped long before the window is.

What actually fills the window

Four things consume the window, and only the first is obvious.

• Every message in the array, every turn. There is no server-side conversation state. A twenty-turn chat resends all twenty turns, so window pressure grows with the square of conversation length in the worst case, not linearly with the last message.
• Tool definitions. The tools array is serialised into the prompt. A dozen functions with thorough JSON Schema parameter descriptions is a fixed several-thousand-token tax on every request, paid whether or not any tool is called. See the request shape.
• Chat-template scaffolding. The role markers DeepSeek uses are real tokens and they are counted. They are a small overhead per turn, but they are not zero, and they are why a local token count of your raw strings comes in under what the server reports.
• Reasoning tokens, on the reasoning endpoint. The trace is generated, billed and counted against the output budget even though it is not the answer. This is the one that surprises people most — how those tokens appear in usage is its own page.

What happens when you exceed it

Overflow is a request-time rejection, not a truncation. Because the API is OpenAI-compatible, the failure arrives as an HTTP 400 with a body in the familiar error envelope, and the message names both the model’s maximum and what you asked for. It is worth logging that message verbatim rather than a generic “request failed”, because the two numbers in it are exactly what you need to size your truncation strategy.

{
  "error": {
    "message": "This model's maximum context length is <N> tokens. However, you requested <M> tokens (<M-k> in the messages, <k> in the completion). Please reduce the length of the messages or completion.",
    "type": "invalid_request_error",
    "param": null,
    "code": "invalid_request_error"
  }
}

Note that the requested figure includes max_tokens. Reserving a large completion budget can push a request over the limit on its own, which produces the confusing case of a prompt that fits failing anyway. If you are near the edge, lower max_tokens before you start trimming messages.

Sizing a prompt in practice

The window is a budget, and budgets are spent by policy rather than by accident. Four decisions cover almost every case.

• Reserve output first, then fill the rest. Decide the completion budget you need, subtract it and a few hundred tokens of margin from the window, and treat what remains as the space available for messages. Doing it the other way round — filling the context and then discovering there is no room to answer — is what produces the 400 that names a number larger than your prompt.
• Trim the middle, not the ends. When history must be cut, the system instructions and the most recent turns are the parts doing work. Dropping the oldest turns is the standard policy and it has the useful property of preserving the cached prefix, whereas summarising history rewrites the prefix and costs you the cache on every subsequent request.
• Bound retrieved content by tokens, not by document count. “Top five results” is unbounded in tokens if one result happens to be a long file. Count, then cut, then send.
• Measure before the call, not after the failure. A local token count using the model’s own tokenizer is cheap, runs in your process, and lets you degrade deliberately — dropping a document, shortening a completion budget — instead of handing the user an error.

The habit worth building is treating the window as a resource with an owner rather than as a limit you occasionally bump into. A request assembled by four independent pieces of code, none of which knows the budget, will eventually exceed it on real traffic. A single assembly step that knows the number and enforces it will not.