What Happens Past Claude's 200K Context Window

Past the context window, nothing is truncated and nothing is silently dropped. The request is rejected at validation with a 400, and the error body tells you exactly how far over you were.

The error

HTTP/1.1 400 Bad Request

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "prompt is too long: 215426 tokens > 200000 maximum"
  }
}

Three things are worth reading off that response before you start fixing anything. It is a 400, so it is a client error and retrying is pointless — a retry policy that treats all failures as transient will burn its budget here and then surface a misleading timeout. Nothing was generated, so nothing was billed for output. And the message carries two integers: what you sent, and what was allowed.

The wording differs slightly across model families and across the managed cloud platforms that resell Claude, but the shape holds: a 400, an invalid_request_error, and a message about prompt length. Match on the status and the type; if you must match the string, match loosely on “prompt is too long” rather than the whole sentence.

One consequence of failing at validation is that the failure is perfectly reliable, which is more useful than it sounds. Unlike a rate limit or an overload, this error will occur on every attempt with the same body, so you can reproduce it locally with the exact request and iterate on the trimming logic without waiting for conditions to recur. Capture the failing body when you log the error — the request, not just the message — and the fix becomes a test case rather than an investigation.

The number in the error

The count in that message is the real tokenized length of your request as Anthropic computed it — the same figure count_tokens would have returned. It is worth capturing rather than discarding, because it answers the question you are about to ask anyway.

import re

def overflow(exc_message):
    m = re.search(r"prompt is too long: (\d+) tokens > (\d+) maximum",
                  str(exc_message))
    if not m:
        return None
    sent, allowed = int(m.group(1)), int(m.group(2))
    return {"sent": sent, "allowed": allowed, "over_by": sent - allowed}

# {'sent': 215426, 'allowed': 200000, 'over_by': 15426}

Logging over_by turns a class of incident into a distribution you can act on. If your overflows cluster at a few thousand tokens over, the fix is a slightly tighter trimming policy. If they are at 300,000 against a 200,000 window, no amount of trimming will help and the architecture is wrong — that document was never going to fit and should be retrieved from rather than pasted in.

Parse defensively, as above, and treat a failure to match as a null rather than an exception. This is a human-facing string and it is allowed to change.

It is also the cheapest instrumentation available for a problem that is otherwise expensive to measure. Knowing the true token size of the requests your system produces normally requires calling count_tokens on everything; the overflow error hands you the number for exactly the requests you most needed it for, at the moment they failed, at no cost. Capture it before you fix the overflow, because once the trimming works you stop receiving the measurement.

Why a prompt that fits can fail

The context window holds the input and the generated output together. The tokens Claude produces are appended to the same sequence it read, which means your effective input budget is the window minus whatever you reserved with max_tokens:

input_tokens + max_tokens  ≤  context_window

199,000 input + 4,096 max_tokens  = 203,096  → rejected
190,000 input + 4,096 max_tokens  = 194,096  → fine
190,000 input + 32,000 max_tokens = 222,000  → rejected

The third line is the one that produces the confusing bug report. The prompt is identical to the second line’s and well under 200,000 tokens; the only thing that changed was a generous output cap. The error still says the prompt is too long, and the fix is in a different parameter entirely. If you have raised max_tokens recently and started seeing this, that is your cause — the reasoning behind that cap is on the max_tokens page.

Extended thinking makes this sharper still, because thinking tokens are output tokens. A large thinking budget is a large reservation, and it comes out of the same window as your document.

The shared window is also why long conversations fail progressively rather than suddenly. Each exchange appends both the user turn and the model’s answer to a history that is resent in full, so the available headroom shrinks by the size of a whole exchange every time, not by the size of what the user typed. A chat that has been fine for forty turns can fail on the forty-first with no change in behaviour from anybody, which is precisely the report that arrives as “it just broke”.

The failure this is not

Two different problems get reported with the same sentence — “the response got cut off” — and they have nothing in common. Telling them apart is a thirty-second job with the response in front of you and an afternoon without it, so it is worth having the distinction ready:

• Context window exceeded. A 400 before generation. You get an error and no content. Caused by the input being too big.
• Output truncation. A 200 with content, and stop_reason: "max_tokens". The answer stops mid-sentence, often mid-word, and nothing failed. Caused by max_tokens being too small for the answer.

The second is the more dangerous of the two, because it succeeds. A truncated JSON object fails to parse, a truncated summary is simply wrong, and nothing in the transport layer complains. Check stop_reason on every response; it is a one-line guard against a failure mode that otherwise reaches your users. The full set of values is on the stop_reason page.

Getting under the limit

1. Count before you send. One call to /v1/messages/count_tokens with the same body, compared against window - max_tokens - margin. This converts a runtime failure into a branch you control.
2. Trim the middle of a conversation, not the ends. The system prompt and the most recent turns carry the most value. Drop whole exchanges from the oldest end, in pairs so that no assistant turn is left without its user turn, and keep the last few intact.
3. Summarise rather than delete. Once the history no longer fits, replace the dropped turns with a short model-generated summary and keep that in the context permanently. Costs one extra call at the boundary and preserves the thread.
4. Retrieve instead of pasting. If a single document does not fit, the answer is not a larger window — it is chunking, embedding and sending the three chunks that matter. Cheaper per request and usually more accurate, because the model is not asked to find one paragraph in two hundred thousand tokens.
5. Split the work. Map over sections and reduce over the results. Each call is small, they can run concurrently, and the failure mode is one section rather than the whole job.
6. Check whether a longer window is actually available. Anthropic has offered extended context beyond 200K on some models behind an anthropic-beta header, with its own pricing. The header name and eligibility change; read Anthropic’s context windows documentation rather than copying a header string from a blog post.