What Happens When You Exceed LM Studio's Context Window

The message reads Trying to keep the first 15857 tokens when context the overflows. However, the model is loaded with context length of only 4096 tokens, which is not enough. The typo is LM Studio’s, not yours, and it is a useful fingerprint: this exact string means the prompt did not fit, which is a different failure from the one where the answer stops mid-sentence.

The error, and where it comes from

That string is reported as an HTTP 400 from the local server, so a client library surfaces it as a bad-request error rather than as a generation result. Reports of it on the LM Studio bug tracker go back to the 0.2 line and the wording has survived several releases, misplaced “the” included — see issue 237 in lmstudio-ai/lmstudio-bug-tracker, where it is triggered by a PDF summarisation chain against a model loaded at 4096 tokens.

Read the two numbers rather than the sentence. The first is how many tokens the request wanted to preserve; the second is the context length the model instance was actually loaded with. When the first exceeds the second there is nothing to truncate — every token in the request is one LM Studio has been told to keep — so it refuses instead of quietly producing an answer from a fragment of your prompt. That refusal is the correct behaviour and it is why raising the number in the second position is usually, but not always, the fix.

The three overflow policies

What happens as a conversation grows past the context length is set by one field. LM Studio’s SDK reference documents it as contextOverflowPolicy on the prediction config, with exactly three values — see the LLMPredictionConfigInput reference published by LM Studio:

• stopAtLimit — generation halts when the window is full. Nothing is discarded; the result simply ends, and the prediction reports stopReason of contextLengthReached. This is the value to choose when a truncated answer is worse than a short one, because it is the only one that tells you it happened.
• truncateMiddle — the system prompt and the first user message are preserved and the middle of the conversation is dropped to make room. Generation continues. Instructions given at the top survive; the detail you gave in turn six may not.
• rollingWindow — the oldest messages fall off as new tokens arrive. Generation continues, and the model gradually stops being able to see the beginning of the conversation, including a system prompt if the window rolls far enough.

The three are not degrees of the same behaviour. One returns a signal and two return prose that looks identical to prose produced with the full context. If your application cannot distinguish “the model answered” from “the model answered having forgotten the first half of what you told it”, that is a property of the policy you chose rather than of the model.

Prompt overflow and generation overflow

These are different events and the policy only governs the second. Prompt overflow happens before a single token is generated: the serialised conversation, system prompt and any attached document already exceed the loaded context length. That is the 400 in the lede. Generation overflow happens later, once the prompt fitted but the growing answer pushed the total past the limit, and that is where stopAtLimit versus truncateMiddle versus rollingWindow decides what you get.

The practical consequence is that “set it to rolling window” does not stop the 400. A rolling window still has to keep the tokens the request is anchored to, and if those alone exceed the window there is no window left to roll. A 40-page PDF pasted into a 4096-token instance fails under all three policies.

Context length is decided at load time

The number the error compares against is not the model’s trained maximum. It is the context length of the loaded instance, chosen when the model was loaded and fixed until it is unloaded. From the CLI that is --context-length, documented on LM Studio’s lms load reference, and in the app it is the slider in the load panel.

# load with an explicit window and a name you can address it by
lms load qwen3-8b --context-length 32768 --gpu max --identifier long-ctx

# a request that targets that instance
curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "long-ctx",
    "messages": [{"role": "user", "content": "summarise the attached notes"}],
    "max_tokens": 512
  }'

Two things follow. A model that supports a long context does not have one unless you asked for it at load: many models are loaded at a conservative default, which is why the same file can fail in LM Studio and succeed elsewhere. And raising the number is not free — the KV cache grows linearly with the window, which is the arithmetic worked through on the combined memory budget page. Ask for 128k on a machine that can hold 16k of cache and the load fails, or the offload silently spills to CPU and everything gets slow.

Fixing it properly

1. Read the second number in the error. That is the window you actually have. Unload and reload with --context-length set high enough for the largest prompt you intend to send, plus your max_tokens.
2. If the reload fails or offload drops, you are out of memory rather than out of context. Reduce the window, take a smaller quantisation, or shift layers to CPU — see the GPU offload settings page.
3. Set the overflow policy deliberately. Choose stopAtLimit for anything programmatic so that a truncated context arrives as contextLengthReached rather than as a confident wrong answer.
4. Stop sending the whole document. Chunk it, retrieve against it, or summarise iteratively. A 4096-token model with retrieval beats a 128k-token model fed everything, and it costs a fraction of the cache.