Ollama's Default Context Length, and Why It Is Not the Number You Read

People arrive at this question because a model with a 128K context window forgot the beginning of a 20,000-token document and said nothing about it. The context that applied was not the model’s. Ollama documents at least three different defaults in three different places, and the one that wins is rarely the largest.

Three defaults, all documented

Each of the figures below is from Ollama’s own documentation, read on 11 August 2026. They do not agree with each other, and that is the actual subject of this page.

• 2048, in the Modelfile reference. The Modelfile reference describes num_ctx as setting the size of the context window used to generate the next token and gives the default as 2048. This is the figure most third-party writing quotes, and it is the value that applies to a request whose context is decided by that parameter path.
• 4,096 / 32,768 / 262,144, chosen by VRAM. The server’s own environment-variable listing describes OLLAMA_CONTEXT_LENGTH as the context length to use unless otherwise specified, with a default of “4k/32k/256k based on VRAM”. Ollama’s context-length page gives the thresholds: under 24 GiB of VRAM, 4k context; 24 to 48 GiB, 32k; 48 GiB and above, 256k.
• Whatever the model image says. A published model can bake PARAMETER num_ctx into its own Modelfile, and many do. On such a model neither of the numbers above is what you get.

One consequence is immediate and unintuitive: on this scheme a machine with a 16 GB card gets a 4k default while a 48 GB card gets 256k, so the same command run on two machines produces different behaviour with no configuration difference between them. If a prompt works on the big workstation and forgets things on the laptop, this is the first thing to check.

Which one actually applies

The ordering, from weakest to strongest, is: the server’s VRAM-derived default, then OLLAMA_CONTEXT_LENGTH if you set it, then any num_ctx baked into the model’s own Modelfile, then num_ctx in the options object of the individual request. The most specific statement wins, which is the sane rule and also the one that surprises people, because it means a model image can override the environment variable you set deliberately on the server.

The per-request form is the one to reach for in application code, because it is the only one that does not depend on how the server was started:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1:8b",
  "messages": [
    {"role": "user", "content": "Summarise the attached transcript."}
  ],
  "options": { "num_ctx": 16384 },
  "stream": false
}'

Two caveats on raising it. Asking for more context than the model’s architecture supports does not give you more — the ceiling is a property of the weights. And asking for more than fits in memory does not fail cleanly; it makes the scheduler move layers to the CPU to free VRAM for the cache, which is a large slowdown rather than an error. The whole request path is covered in the num_ctx parameter.

What happens when you exceed it

Nothing you can see. When the rendered prompt is longer than the context in force, Ollama drops messages from the front of the conversation and proceeds. There is no error, no field in the response saying it happened, and the only record is a debug-level log line.

This is documented as a defect rather than inferred. Ollama issue #14259, “Chat history and embedding truncation happens silently with no user-visible indication”, opened in February 2026 and open at the time of writing, describes the behaviour and locates the logging in server/prompt.go, where it is emitted at debug level. The issue asks for the severity to be raised so that operators can see it at all.

The direction of the drop is what makes it expensive. It is the front of the conversation that goes, and the front of the conversation is where the system prompt and the instructions live. A long-running agent loop under a small context does not gradually lose old chatter; it loses the thing that told it what its job was, and then behaves oddly for reasons that appear to have nothing to do with length. Compare this with a hosted API, which rejects an over-length request outright — noisier, and much easier to debug.

Why the default is small: the arithmetic

The defaults are conservative because context is not free, and the cost is memory that scales linearly with the number of tokens you reserve. Here is the derivation, with every input named so you can redo it for your own model.

The KV cache stores one key and one value vector per token, per layer, per key/value head. For Llama 3.1 8B, the published config.json gives num_hidden_layers 32, num_key_value_heads 8, and hidden_size 4096 against num_attention_heads 32, so the head dimension is 4096 / 32 = 128. At 16-bit cache precision that is 2 bytes per element, and the bytes per token are:

2 (key and value)
  x 32 layers
  x 8 key/value heads
  x 128 head dim
  x 2 bytes                = 131,072 bytes per token
                           = 128 KiB per token

  x   2,048 tokens         = 256 MiB
  x   8,192 tokens         =   1 GiB
  x 131,072 tokens         =  16 GiB

Assumptions, stated: 16-bit cache, no cache quantization, one concurrent sequence, and a model whose grouped-query attention gives it only 8 key/value heads. A model without grouped-query attention pays four times this. Set OLLAMA_NUM_PARALLEL above 1 and you multiply by that too, since each parallel slot needs its own cache.

Now put it next to the weights. That same 8B model at a 4-bit quant is roughly 5 GB of weights. On a 16 GB card, a 4k context costs half a gigabyte of cache and everything fits comfortably; the model’s full 131,072-token window would want 16 GiB of cache alone and cannot fit at any quantization. The tiered default is not timidity. It is the largest number that reliably leaves room for the weights on a card of that size.

The lever this suggests is cache precision rather than context length. OLLAMA_KV_CACHE_TYPE defaults to f16 and accepts quantized types; halving the bytes per element halves every number in the block above. The mechanism behind the cache itself is covered in the KV cache.

Reading the context you actually got

Do not infer it. Ask, while the model is loaded:

curl -s http://localhost:11434/api/ps

The entry for a loaded model carries a context_length field alongside size and size_vram, which is the server telling you what it allocated rather than what you requested. Running ollama ps gives the human-readable version of the same data. If context_length is 4096 when you asked for 32768, something above it in the precedence list won, and the model image is the usual culprit — check with ollama show --modelfile.

For a second signal, compare the prompt_eval_count in a non-streaming response against your own count of the input. If the server evaluated materially fewer tokens than you sent, the difference is what it silently discarded.