Implicit and Explicit Caching in the Gemini API

Gemini has two caching mechanisms with the same underlying trick and completely different contracts. One happens without you doing anything and makes no promises; the other is a resource you create, pay storage for, and can rely on.

What both are doing

Prefill is the expensive, parallelisable part of serving a request: reading the prompt and computing the attention keys and values for every token in it. Those key-value tensors depend only on the tokens before them, so if two requests begin with an identical prefix, the tensors for that prefix are identical too.

Caching keeps them instead of recomputing them. The saving is real compute, which is why it shows up as a discounted input rate rather than as a courtesy. Two conditions follow directly from the mechanism and apply to both flavours:

• It has to be a prefix. Shared content in the middle of a prompt caches nothing, because every token after the first difference has different keys and values. Put the stable material — system instruction, documents, tool declarations, few-shot examples — at the front, and the variable part at the end.
• It has to be identical. Byte-for-byte. A timestamp, a session id or a shuffled example ordering at the top of the prompt destroys the hit for everything after it. This is the single most common reason a cache that should work does not.

Implicit caching

Implicit caching is on by default for the Gemini 2.5 models and there is no parameter to enable it. Send a request whose prefix matches a recent one and you may get a hit, billed at a reduced rate on the cached portion. You are not billed for storage.

# Nothing to declare. Structure is the whole API surface.
{
  "systemInstruction": {"parts": [{"text": "<long, stable, identical every time>"}]},
  "contents": [
    {"role": "user", "parts": [
      {"text": "<the 40-page policy document, identical every time>"},
      {"text": "Question: what is the notice period for contractors?"}
    ]}
  ]
}

The important word is may. Google documents implicit caching as a best-effort optimisation, not a guarantee — hits depend on whether a matching prefix is still resident, which depends on traffic and timing you cannot see. Design so a miss costs you money rather than correctness, and never build a latency SLA on it.

There is a minimum prefix length below which nothing is cached at all, and it differs by model. It is covered in the minimum token count for Gemini context caching rather than repeated here, because it is the figure most likely to move.

Explicit caching

Explicit caching makes the cache a resource with a name and a time-to-live. You create it, you are billed for storing it by the hour, and requests that reference it get the discounted rate deterministically. Documented in Google’s context caching guide.

# 1. Create the cache
curl "https://generativelanguage.googleapis.com/v1beta/cachedContents" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "models/gemini-2.5-flash",
    "displayName": "employee-handbook-v7",
    "systemInstruction": {"parts": [{"text": "Answer only from the handbook."}]},
    "contents": [{"role": "user", "parts": [
      {"fileData": {"mimeType": "application/pdf",
                    "fileUri": "https://generativelanguage.googleapis.com/v1beta/files/abc123"}}
    ]}],
    "ttl": "3600s"
  }'

# -> {"name": "cachedContents/9fq2m4xk", "usageMetadata": {"totalTokenCount": 84213}, ...}

# 2. Reference it. contents holds only what is new.
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent" \
  -H "x-goog-api-key: $GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "cachedContent": "cachedContents/9fq2m4xk",
    "contents": [{"role": "user", "parts": [{"text": "What is the notice period for contractors?"}]}]
  }'

Three constraints that are easy to trip over. The cache is bound to one exact model version, and a request naming a different model will not use it. The cached content is immutable — you can update the ttl with a PATCH, but not the contents; to change the document you create a new cache. And the cache expires when the TTL runs out and is then gone, so a long-lived assistant needs either a generous TTL or logic that recreates the cache on a not-found.

That last case is the one to write code for before you meet it. When the TTL elapses, a request naming the cache fails outright rather than falling back to sending the content uncached — the name no longer resolves, so you get an error where you expected an answer. The durable shape is to catch the not-found, recreate the cache from the source material you still hold, and retry once. Storing only the cache name and not what went into it makes that impossible, which is a surprisingly easy state to end up in.

Which one you want

• Implicit when the same prefix is being sent frequently by ordinary traffic — a shared system prompt across many users, a chat where the history grows by appending. You get the discount for free and lose nothing on a miss.
• Explicit when one large body of content is queried repeatedly over a bounded window and you want the discount guaranteed: a long document under interrogation, a video being analysed from several angles, a codebase in a review session.
• Explicit is a bet. You pay storage per hour whether or not you query. It pays off above a break-even query rate you can compute: storage cost per hour versus the per-request saving times the requests you will make in that hour. Below that rate, implicit caching or no caching is cheaper.
• Neither replaces prompt structure. If the stable content is not at the front, both mechanisms are dead and no parameter will revive them.

Verifying you got a hit

One field. usageMetadata.cachedContentTokenCount on the generation response reports how many of the prompt tokens were served from cache:

{
  "usageMetadata": {
    "promptTokenCount": 84265,
    "cachedContentTokenCount": 84213,
    "candidatesTokenCount": 88,
    "totalTokenCount": 84353
  }
}

Zero, or the field being absent, means you paid full rate for the whole prompt. If you expected a hit and got a zero, check the prefix for anything that varies between requests before checking anything else — an injected current date at the top of a system instruction is the classic. Log this field per request; it is the only feedback the API gives you about whether your prompt structure is doing what you think.

Read the number rather than just its truthiness. A value well below promptTokenCount is a partial hit: the prefix matched up to the first difference and everything after it was recomputed. That is normal in a growing chat, where each turn extends the previous prefix, and it is a diagnosis in a supposedly-static prompt — the position of the shortfall tells you roughly where the variation was introduced.

Two things people forget are part of the prefix. Tool declarations count: reordering functionDeclarations between requests, which happens for free if you build them by iterating a dictionary, changes the serialised prefix and kills the hit. So does the system instruction, which is why templating the current timestamp into it is such an effective way to disable caching for an entire application without anyone noticing.

When streaming, the usage figures behave the way they do everywhere else in this API: they are only complete on the final chunk. Code that logs cachedContentTokenCount from the first chunk that carries usageMetadata will report zeroes and send you hunting for a caching problem that does not exist. Size a candidate prefix with countTokens before you decide whether it is worth caching at all.