Command R+'s Context Window and Output Limit

Command R+ is documented with a 128,000-token context window and a 4,096-token maximum output. Those are two different limits with two different failure modes, and only one of them is the one people mean when they say a request was too long.

The documented numbers

Cohere publishes per-model limits on its models documentation. At the time of writing, the Command family is listed as follows:

model                    context length   max output tokens
command-a-03-2025             256,000            8,192
command-r-plus-08-2024        128,000            4,096
command-r-08-2024             128,000            4,096
command-r7b-12-2024           128,000            4,096

Those figures come from Cohere’s models reference and its per-model pages. The number people cite for Command R+ is the 128,000 — it is the one that appears in the marketing copy and in the comparison tables — but the 4,096 is the one that will surprise you in production, because it is small, it is a hard ceiling, and it is not proportional to the context window at all.

Two budgets, not one

The context window is the total sequence the model can attend over: your system instructions, the conversation history, any documents you passed in the documents field, the tool definitions, the tool results, and the tokens the model has generated so far in this turn. Everything competes for the same 128,000.

The output limit is separate and smaller. It caps how many tokens one assistant turn may contain, regardless of how much of the context window is free. With Command R+ you can hand the model 120,000 tokens of documents and still not get more than 4,096 tokens of answer back. This is not a quirk of Cohere’s — most providers set an output ceiling well below their context ceiling, because generation is sequential and an unbounded generation is an unbounded request.

The practical consequence is that Command R+ is shaped for the job it was built for: read a great deal, answer briefly, cite what you used. If your workload is “translate this 200-page document”, the 4,096 is the binding constraint and no amount of context window helps you. You are chunking the output whether you planned to or not.

Reading the limit from the API

Cohere’s models endpoint reports the context length per model, so the live figure is one request away and does not need to be copied into your code as a constant:

curl -s https://api.cohere.com/v1/models/command-r-plus-08-2024 \
  -H "Authorization: Bearer $CO_API_KEY"

{
  "name": "command-r-plus-08-2024",
  "endpoints": ["chat", "summarize"],
  "context_length": 128000,
  "tokenizer_url": "https://storage.googleapis.com/cohere-public/tokenizers/command-r-plus-08-2024.json",
  "is_deprecated": false
}

Two fields there are worth wiring into something that runs on a schedule. context_length is the number this page is about, and is_deprecated is the early warning that the snapshot you pinned is on its way out — see Cohere’s deprecation notices for legacy Command models. Listing /v1/models without a name returns the whole catalogue with the same fields, which is the cheapest possible way to notice that a limit changed under you.

What happens when you exceed it

Overshooting the two limits produces two entirely different results, and confusing them wastes an afternoon.

• Too many input tokens is an error from the API before generation starts. The request is rejected; you are not billed for a completion you did not get. The fix is to send less, and Cohere’s v1 Chat API offers prompt_truncation to do it for you: "AUTO" drops the lowest-relevance documents and the oldest chat history to make the request fit, while "OFF" makes an over-length request an error instead of a silently shortened one.
• Hitting the output limit is not an error at all. The model stops mid-sentence and the response carries a finish reason of MAX_TOKENS rather than COMPLETE. Nothing throws. Nothing logs. If you are not reading finish_reason, a truncated answer looks exactly like a short one, and it will reach your users looking confident and ending halfway through a clause.

The "AUTO" default deserves a moment’s thought rather than acceptance. It means a long-running conversation quietly starts forgetting its own beginning at exactly the point where a user would most expect it to remember, and it means the documents you considered essential can be the ones dropped, because relevance is judged by the model and not by you. On anything where completeness matters more than convenience, setting "OFF" and handling the error yourself gives you a signal instead of a silence.

What fills the window in practice

A 128,000-token window sounds like more room than any application needs, right up until you write down what is actually in a request. On a grounded, tool-using conversation the components are:

preamble / system instruction        200 –  1,500 tokens, every turn
tool definitions (JSON Schema)       150 –    400 tokens per tool, every turn
retrieved documents                2,000 – 60,000 tokens, per turn
chat history                            grows without bound
tool results in this turn             500 –  5,000 tokens
the answer being generated              up to 4,096

Three of those are constant-per-turn costs that people forget to count because they were written once: the preamble, the tool definitions, and any documents you attach unconditionally. Six tools at 300 tokens each is 1,800 tokens on every request in the conversation, whether or not the model uses any of them. That is not a reason to avoid tools, but it is a reason to notice that a tool registry that grows to thirty functions has become a five-figure token cost per turn on its own.

The one that grows is chat history, and the arithmetic is worth doing once. A conversation whose turns average 400 tokens of user text and 600 of answer adds 1,000 tokens per exchange, and every exchange is resent in full on the next request because the API is stateless. Thirty exchanges is 30,000 tokens of history before anything else is in the request, and the fiftieth turn costs roughly fifty times the input tokens of the first for the same question. Retrieval on top of that is what actually consumes a long window.

Two consequences shape how a long-context application is built. First, cost per turn rises linearly with conversation length, so the expensive requests are the late ones and an average that mixes short and long sessions hides it entirely. Second, filling the window is rarely the right goal even when it fits — attention quality is not uniform across 128,000 tokens, and twelve well-chosen documents generally beat a hundred marginal ones. That is exactly what the rerank endpoint exists to do: retrieve broadly, score, and send only the top of the list.

The number differs by host

Command R+ is served on Cohere’s own API, on Amazon Bedrock, on Azure AI Foundry, on Oracle OCI and in private deployments. The weights are the same; the serving limits are not necessarily. A cloud marketplace listing can impose its own maximum on request size or on output tokens, and per-account rate limits vary by host and by tier.

So “Command R+ has a 128k context window” is a statement about the model as Cohere documents it, not a guarantee about the endpoint you happen to be calling. When a request that works against api.cohere.com fails against a marketplace endpoint with the same body, the platform’s own quota page is the place to look before the model page. Cohere’s rate limit documentation covers its own API only.

One more thing the headline number hides: 128,000 tokens is not 128,000 words, and the ratio depends on the tokenizer. Cohere ships a per-model tokenizer that you can run locally to count before you send — the vocabulary is large enough that the words-per-token ratio differs noticeably from other providers’, which matters if you sized your chunks against someone else’s counter.