Gemini 1.5 Pro's Context Window and the Long-Context Caveat

Gemini 1.5 Pro accepts 2,097,152 input tokens—two mebitokens, not a round two million. That figure is real and it is a hard limit enforced by the API. What it does not tell you is how well the model finds one fact among two million tokens of other facts, and Google documents an answer to that too.

The number, and what it counts

The documented input limit for gemini-1.5-pro is 2,097,152 tokens, listed on Google’s model reference alongside a separate output limit of 8,192 tokens. Two limits, not one: the window is the size of what you may send, and the output ceiling is an independent cap on what comes back in a single response. Filling the window does not buy you a longer answer, and the output ceiling is a different page’s problem.

The input count is everything in the request that becomes tokens: the contents array including every previous turn you resend, the system_instruction, tool declarations, and the token cost of any image, audio or video parts. Function declarations in particular are easy to forget—a dozen tools with long descriptions is a non-trivial fixed charge on every call. If you want the true number before you send, the API has an endpoint for it rather than a rule of thumb:

POST https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro:countTokens

{
  "contents": [
    { "role": "user", "parts": [{ "text": "..." }] }
  ],
  "systemInstruction": {
    "parts": [{ "text": "You are a contract analyst." }]
  }
}

The response carries totalTokens, computed by the same tokenizer that will bill you. The countTokens method is the only figure that is correct for your exact payload; character and word heuristics drift badly once media parts are involved.

What two million tokens holds

Token counts are abstract until you convert them into something with a duration or a page count. Two of Google’s documented media rates make that conversion arithmetic rather than guesswork. Google’s token-counting documentation gives audio at 32 tokens per second and video sampled at one frame per second, each frame charged at the per-image rate. Taking those figures as the assumption:

Audio only
  2,097,152 tokens / 32 tokens per second
    = 65,536 seconds
    = 18 hours 12 minutes

Video with its audio track, at 1 fps and 258 tokens per frame
  258 (frame) + 32 (audio) = 290 tokens per second of clip
  2,097,152 / 290
    = 7,231 seconds
    = about 2 hours

Text, at the widely used 4-characters-per-token approximation
  2,097,152 x 4 = about 8.4 million characters
    = roughly 1.4 million English words

The audio and video lines are derived from documented per-second rates, so they are as good as those rates. The text line is an approximation and is labelled as one: real English prose runs close to it, code and non-Latin scripts do not. Gemini’s tokenizer decides, and it disagrees with the four-character rule most on exactly the inputs people care about.

Google’s own caveat

The interesting part of Google’s long-context guide is not the claim that recall is high. It is the qualification attached to it. The standard evaluation for long context is a “needle in a haystack” test: hide one sentence somewhere in a very long document and ask for it back. Gemini 1.5 Pro does extremely well at that. Google’s documentation then notes the harder variant—multiple needles—and that recall falls as the number of facts you need retrieved at once rises.

That distinction is the whole practical caveat, and it is easy to misread the marketing figure as covering both. One fact in two million tokens is close to solved. Twenty facts, scattered, that must all appear in one answer, is a different and considerably less reliable task. So is anything that requires reasoning across distant parts of the context rather than locating one span in it: comparing clause 4 of contract A against clause 19 of contract Q is not retrieval, and its accuracy is not what the needle test measured.

Two behaviours follow that people report as bugs and that are neither bugs nor surprises given the above:

• Something in the middle gets missed. Position within a very long context is not neutral. Material at the start and end of a long input is more reliably used than material buried in the middle, which is why putting the actual question after a long document, rather than before it, is a documented prompting recommendation for Gemini.
• Answers get less specific as the input grows. With two million tokens available, the model has a great deal of plausible material to draw on and no signal about which of it you meant. Long context does not replace the instruction to narrow the task.

The cost and latency the window does not mention

A context limit is a capability statement, not a budget. Filling it is billed at the input rate for every token, on every turn of a conversation, because the API is stateless and each request re-sends everything. A two-million-token prompt asked ten follow-up questions is twenty million input tokens, not two.

Latency scales too, and not linearly—attention work grows with the square of the sequence length, so time-to-first-token on a nearly full window is measured in tens of seconds rather than in the sub-second figures short prompts produce. Google also priced long requests differently across the 1.5 generation, with a higher per-token rate above a 128,000-token threshold. Check the current pricing page before designing around a long window; the threshold structure has changed between model generations.

The limits that bite before the window does

Almost nobody meets the two-million-token ceiling first. Three other limits sit below it, and each one produces a different error that reads like a context problem and is not.

Request body size. Anything sent as inline_data is base64 in a JSON body, and the documented ceiling for a total inline request is 20 MB. Base64 inflates bytes by about a third, so the practical inline limit is under 15 MB of source material—a fraction of what the token window would accept. Larger content has to go through the File API and be referenced by URI, which is a different code path and not a parameter you can raise.

Per-minute token quota. Rate limits on the Gemini API are expressed partly in tokens per minute, not only in requests per minute. A single request that fills a large fraction of the window can exceed a whole minute’s token allowance on its own, which produces an HTTP 429 for a request that is perfectly valid and well inside the context limit. The retry advice for that case is not “retry immediately”—you will consume the next minute too—but back off and, if the pattern is structural, split the work.

Timeouts in everything between you and the model. A request whose prefill takes forty seconds will pass through your language’s default HTTP client timeout, a load balancer, and possibly a serverless function with a hard execution ceiling. Any of the three can cut the connection while the model is still working, and you are billed for the prefill regardless. Long-context calls generally need explicit timeout configuration and, for anything user-facing, streaming so that bytes start arriving before any intermediary loses patience.

The general shape of the advice is that the context window is a capability of the model and the other three are properties of the system around it. A design that fits the first and ignores the others works in a notebook and fails in a deployment.

How to use the top of the range

The window is most valuable when the alternative is building a retrieval pipeline you do not want to maintain, and least valuable when it is used as one. Three things make the difference:

• Cache the stable part. If the same two-million-token corpus is asked many questions, an explicit context cache turns the repeated prefill into a stored artefact charged at a reduced rate. This is the single largest cost lever long context has—see the minimum token floor and TTL.
• Put the question last. Document first, instruction after. This is Google’s documented ordering advice for long inputs and it costs nothing to follow.
• Ask for citations you can verify. If the answer must name the page, section or timestamp it came from, a retrieval failure becomes visible instead of becoming a fluent paragraph. This is worth more at two million tokens than any prompt-engineering flourish.