Context Caching on Disk in the DeepSeek API

DeepSeek’s prompt cache has no parameter, no header and no opt in. It is on, it is keyed on the leading tokens of your request, and the only way to know whether it worked is to read two counters in the response.

What is cached, and when

Prefill — reading the prompt — produces a key/value tensor for every token in the context, and that tensor is what attention consults for the rest of the request. It is a pure function of the token sequence. Send the same leading tokens again and the same tensors would be recomputed identically, which makes them cacheable in the ordinary sense: identical input, identical output, no side effects.

DeepSeek stores that state on disk rather than holding it in GPU memory, which is the design decision the feature is named for. Disk is enormously cheaper per gigabyte than accelerator memory, so the cache can be large and can persist far longer than a GPU-resident cache could. The tradeoff is that a hit costs a read from storage rather than nothing at all, so the saving is large but not total.

The matching is on the prefix, from the very first token, and it is exact. One changed character anywhere in your system prompt invalidates everything from that character onward. A timestamp at the top of a prompt guarantees a permanent miss on every request, and it is the single most common reason a team concludes the cache does not work.

Retention is automatic and unguaranteed. Entries are evicted after a period of disuse, and DeepSeek documents no charge for cache storage and no API for managing it. Correctness never depends on a hit — a miss just costs the normal price.

The two usage fields

Every response carries prompt_cache_hit_tokens and prompt_cache_miss_tokens in usage. They partition prompt_tokens: the two always sum to it, and neither is an addition to it.

"usage": {
  "prompt_tokens": 3200,
  "completion_tokens": 180,
  "total_tokens": 3380,
  "prompt_cache_hit_tokens": 3072,
  "prompt_cache_miss_tokens": 128
}

# 3072 + 128 == 3200  -> the split is a partition, not a bonus

Log both on every request from the first day you use the API. Cache hit rate is not visible anywhere else, and the failure mode of prompt caching is silent: a refactor that moves one line to the top of your prompt takes your hit rate from 95% to zero with no error, no latency alarm and a bill that arrives weeks later. A ratio you already plot is a ratio you notice falling.

The first request with a new prefix is necessarily all miss. Judge the cache on the second and subsequent requests, and expect the first request after a deploy that changed the prompt to be a miss for every instance.

Granularity and why prefixes matter

The cache operates on blocks rather than individual tokens — DeepSeek documents a 64-token granularity — which has two consequences you can act on. Content shorter than one block is not cached at all, so a short prompt gets no benefit however often you repeat it. And the hit count is rounded down to a block boundary, which is why prompt_cache_hit_tokens is typically a clean multiple of 64 and why the tokens between the last cached block and your first changed token are charged as misses even though they were identical.

Blocks explain the shape of the optimisation. Moving a variable element fifty tokens later in the prompt might gain you nothing, because it does not cross a boundary; moving it after a two-thousand-token static document converts two thousand tokens from miss to hit. Think in terms of “how much identical text precedes the first difference”, not in terms of how much text is identical overall.

What it changes about the bill

Input tokens are billed at two rates: one for the cache-hit portion and a higher one for the cache-miss portion. Output is unaffected — nothing about generation is cacheable, because each output token depends on the ones before it in that specific response.

The discount on the hit portion is substantial rather than marginal; the exact multiple is on DeepSeek’s pricing page and it has changed more than once, so it is deliberately not quoted here. What is stable is the structure of the calculation, and you can run it with whatever the current figures are:

cost_input = (hit_tokens  * rate_cache_hit  / 1e6)
           + (miss_tokens * rate_cache_miss / 1e6)
cost_output = completion_tokens * rate_output / 1e6

# a long shared system prompt with a short user turn is the best case:
#   hit_tokens dominates prompt_tokens, and prompt_tokens dominates the request

The workloads that benefit most are the ones that look wasteful without a cache: a large fixed instruction block, a long document answered against many times, a multi-turn conversation whose history is resent in full. The workloads that benefit least are single-shot requests with unique inputs, where there is no prefix to share.

There is a latency effect too. A cache hit skips most of the prefill arithmetic, so time to first token falls for long prompts even though the generation rate afterwards is unchanged. On a long-document workload this is often more noticeable to users than the cost saving is to finance.

Designing prompts to hit it

1. Sort by volatility, most stable first. System instructions, then tool definitions, then retrieved documents, then conversation history, then the current user message. Every request should share the longest possible identical opening with the last one.
2. Remove per-request noise from the top. Timestamps, request IDs, randomised few-shot ordering, a user name interpolated into the greeting. If it must be present, move it to the end of the prompt where it costs one block instead of all of them.
3. Serialise deterministically. Python dictionary iteration order, JSON key order and floating-point formatting all have to be stable, or two logically identical prompts differ byte-wise and miss. json.dumps(obj, sort_keys=True) is the cheapest insurance available here.
4. Append, never rewrite, conversation history. A history that is summarised or re-ordered between turns invalidates from the point of change. Appending preserves the whole prefix.
5. Verify with the counters. Send the same request twice and assert that prompt_cache_hit_tokens is non-zero on the second. Keep that as a test; it will catch the prompt refactor that quietly costs you the cache.

When it does not help

The cache is free and automatic, so there is never a reason to avoid it — but there are workloads where the correct expectation is that it will do nothing, and knowing which is better than being disappointed by a hit rate.

• Short prompts. Below the block size there is nothing to cache, so a high-volume endpoint sending brief classification prompts will show a hit rate near zero however repetitive the traffic is. That is the design, not a fault.
• Genuinely unique inputs. A single-shot summariser over distinct documents shares no prefix beyond the instructions. You will cache the instructions and nothing else, which is worth having and is not the win people expect.
• High-cardinality personalisation at the top. One prompt per user, each with the user’s name and preferences first, gives every user their own cache entry and none of them the benefit of anyone else’s. Moving the personalised block after the shared instructions converts most of it back.
• Output-heavy work. Nothing about generation is cacheable. A workload whose bill is dominated by output — anything on the reasoning path in particular — is barely affected by input caching however well you structure the prompt.
• Long gaps between identical requests. Entries are evicted after disuse. A nightly batch job may find its prefix gone each time, which looks like the cache not working and is the cache working exactly as documented.

One question comes up often enough to answer directly: the cache is keyed on the token sequence, which means an identical prefix is identical regardless of who sent it. If your prompts embed data you treat as confidential, the relevant assurances are in the provider’s terms and security documentation rather than in the caching behaviour, and that is the document to read rather than reasoning about it from first principles.