DeepSeek-R1's Distilled Variants: Context Window by Size

There is no single context window for “R1-distill”. The distills are six different base models fine-tuned on the same data, and each keeps the window of the model it started from.

What a distill actually is

The R1-distill releases are not smaller DeepSeek models. DeepSeek took existing open-weight base models from other families, fine-tuned them on reasoning traces generated by DeepSeek-R1, and published the result. The architecture, the tokenizer, the positional encoding scheme and the context length all come from the base. What comes from DeepSeek is the behaviour — the tendency to produce a long reasoning trace before answering.

Every practical property of these models follows from that. They emit the same thinking delimiters because they were trained to. They are governed by their base model’s licence, not by DeepSeek’s alone — which is where the licence question gets complicated. And their context window is whatever their base supported, which is the subject of this page.

The base behind each size

The mapping is published on the DeepSeek-R1 model card, which lists each distill next to the base model it was built from:

R1-Distill-Qwen-1.5B    <- Qwen2.5-Math-1.5B
R1-Distill-Qwen-7B      <- Qwen2.5-Math-7B
R1-Distill-Qwen-14B     <- Qwen2.5-14B
R1-Distill-Qwen-32B     <- Qwen2.5-32B
R1-Distill-Llama-8B     <- Llama-3.1-8B
R1-Distill-Llama-70B    <- Llama-3.3-70B-Instruct

Three families of answer fall out of that list, and they are very different from one another.

The Llama-based distills

Llama 3.1 and Llama 3.3 are documented by Meta with a 128K context window, so these two inherit the most generous windows in the set. They also inherit Meta’s community licence and its acceptable-use policy.

The general-purpose Qwen distills

Qwen2.5’s larger models are documented by Alibaba with long context support, with the caveat that going beyond the natively trained length requires enabling a rope-scaling configuration rather than simply asking for more. Read the Qwen2.5 model card for the current numbers and the required configuration; they are published as two figures, not one.

The maths-specialised Qwen distills

This is the case that catches people. The 1.5B and 7B distills are built on Qwen2.5-Math models, which were released as task-specialised models with a substantially shorter native context than their general-purpose siblings — a few thousand tokens rather than tens of thousands. If you are choosing a small distill for a long-document task, this is the first thing to check and the last thing anyone expects.

The field to read

For any of them, the answer is in the repository rather than in prose. Pull the config directly:

import json, urllib.request

MODELS = [
    "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
    "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B",
    "deepseek-ai/DeepSeek-R1-Distill-Qwen-14B",
    "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
    "deepseek-ai/DeepSeek-R1-Distill-Llama-8B",
    "deepseek-ai/DeepSeek-R1-Distill-Llama-70B",
]

for m in MODELS:
    url = f"https://huggingface.co/{m}/raw/main/config.json"
    cfg = json.load(urllib.request.urlopen(url))
    print(f"{m.split('/')[-1]:34} "
          f"max_position_embeddings={cfg.get('max_position_embeddings')} "
          f"rope_scaling={cfg.get('rope_scaling')}")

That prints the authoritative answer for the revision you would actually download, which no article can do — including this one, which is why no table of numbers appears above. Run it before you size a deployment.

Declared length is not trained length

Here is the subtlety that makes this a real question rather than a lookup. max_position_embeddings is a declaration in a configuration file. It tells your serving stack what to permit. It is not evidence that the model was trained at that length, and the two can differ — a fine-tune inherits weights from its base but its config is written by whoever published the fine-tune.

A model asked to work at positions well beyond anything it saw in training does not error. It degrades: attention over the far region becomes unreliable, instructions early in a long prompt stop being followed, and output quality falls off in a way that looks like a bad model rather than a misconfiguration. This is the failure mode to suspect when a distill performs well on short prompts and poorly on long ones.

What your serving stack does with it

• vLLM and SGLang read max_position_embeddings to size the KV cache at startup and reject requests beyond it. They also accept an override, which is precisely how people end up running past the trained length without realising.
• llama.cpp and Ollama apply their own default context size that is frequently smaller than the model supports, and silently truncate the prompt rather than erroring. A model that “forgets the start of the document” locally is usually this and not the model.
• Rope scaling must be honoured. If the config carries a rope_scaling block and your runtime ignores it, long-context behaviour is wrong in a way that produces plausible-looking bad output rather than an error.
• The reasoning trace lives in the window too. These models think at length before answering, and that trace occupies output budget inside the same window. A 4K-window distill given a 3K prompt has very little room left to reason in, which is a much tighter constraint than it appears.

That last point is the one to carry away when choosing a size. For a reasoning model the usable window is not “window minus prompt”; it is “window minus prompt minus however long the model decides to think”, and the third term is not under your control.

Choosing a size

The window is one input to the choice and rarely the deciding one. Four others matter as much, and they pull in different directions.

• Memory, which follows the parameter count and the precision. This is the hard constraint: a size that does not fit on your hardware is not a candidate whatever else is true of it. Quantisation is the lever that moves the boundary, at some cost in output quality that is worth measuring on your own task rather than assuming.
• Room to think. These models produce long traces, and the trace lives in the same window as everything else. A small distill with a short inherited window can be the wrong tool for a task a larger one handles easily, not because it is less capable but because it runs out of room mid-reasoning.
• The base model’s specialisation. A distill built on a maths-specialised base carries that specialisation. Excellent on the problems it was built for, and a poor choice for general instruction-following at the same parameter count. Read what the base was, not just how big it is.
• The licence. Qwen-based and Llama-based distills sit under different terms with different obligations, and for a distributed product that can decide the question before performance does — the licence page has the detail.

Where a distill fits at all, the reason to prefer one over calling a hosted reasoning model is usually not cost per token — it is that the weights run on infrastructure you control, with no per-request rate limit and no data leaving your network. Those are the properties worth paying the operational overhead for. If none of them applies, a hosted endpoint is almost always the simpler answer.