Redis for Caching, Queues and Vectors

Redis can hold your response cache, your job queue and your vector index, and plenty of teams put all three on one server. The reason not to is specific and mechanical: maxmemory-policy is a server-wide setting, the three workloads need three different values of it, and the one you pick silently breaks the other two.

Three jobs, three failure modes

Start from what each workload does when memory runs out, because that is where the designs diverge.

Two of the three must never evict, and the one that must evict is the one that will consume all available memory if you let it. That tension is the whole of this page.

Job one: the cache

The obvious use is caching model responses and retrieval results. Key design is where this succeeds or fails, and the rule is that every input which could change the answer belongs in the key:

# key = sha256(model | index_version | tenant | k | filters | query)
import hashlib, json

def cache_key(model, index_version, tenant, k, filters, query):
    payload = json.dumps(
        [model, index_version, tenant, k, filters, query],
        sort_keys=True, separators=(",", ":"),
    )
    return "ret:" + hashlib.sha256(payload.encode()).hexdigest()

# SET with an expiry in one command. NX so a concurrent writer does not
# clobber a fresher value.
r.set(cache_key(...), json.dumps(results), ex=3600, nx=True)

index_version in the key is the invalidation strategy, and it is the reason not to reach for SCAN and delete. Bump the version after a re-index and every key derived from it becomes unreachable instantly, with no scan of the keyspace and no risk of a partial delete; LRU reclaims the orphans in its own time. The design is expanded in caching retrieval results.

Set a TTL on every cached key without exception. A key with no expiry in a cache is a memory leak with a slow fuse — it is retained until eviction pressure arrives, and under volatile-lru it is never evicted at all, because that policy only considers keys that have one.

Job two: the queue

Use Streams, not lists. LPUSH and BRPOP make a queue that loses a job whenever a worker dies between popping and finishing, which for a two-minute inference job is not a rare event. Streams with consumer groups give you an acknowledgement and a pending-entries list.

# once, at deploy
XGROUP CREATE jobs:inference workers $ MKSTREAM

# producer
XADD jobs:inference '*' tenant 42 model gpt-4o-mini prompt_id 91827

# worker: claim up to 1 message, block 5 s if empty
XREADGROUP GROUP workers worker-3 COUNT 1 BLOCK 5000 STREAMS jobs:inference '>'

# on success
XACK jobs:inference workers 1717430400000-0

# recover work from a worker that died: reclaim anything pending
# for more than 10 minutes
XAUTOCLAIM jobs:inference workers worker-3 600000 0 COUNT 10

XAUTOCLAIM is the part that makes this a real queue: a message delivered but never acknowledged stays in the pending list, and another worker can take it over after a timeout. Without that step the stream silently accumulates work that nobody will ever do, which looks exactly like a queue that is working until you check XPENDING.

Cap the stream so it cannot grow without bound — XADD jobs:inference MAXLEN ~ 100000 '*' …. The tilde makes the trim approximate and much cheaper, trimming to roughly that length at a convenient point rather than exactly on every write.

Job three: the vector index

Redis’s query engine (the module historically called RediSearch, included in Redis Stack and in Redis 8) indexes vectors held in hashes or JSON documents. Two index types: FLAT, which is exact brute-force, and HNSW, with the same parameters as everywhere else.

FT.CREATE idx:chunks
  ON HASH PREFIX 1 chunk:
  SCHEMA
    tenant    TAG
    doc_id    NUMERIC
    content   TEXT
    embedding VECTOR HNSW 10
      TYPE FLOAT32
      DIM 768
      DISTANCE_METRIC COSINE
      M 16
      EF_CONSTRUCTION 200

HSET chunk:9001 tenant acme doc_id 17 content "..." embedding "<768 float32 bytes>"

FT.SEARCH idx:chunks "@tenant:{acme}=>[KNN 10 @embedding $vec AS score]"
  PARAMS 2 vec "<query bytes>"
  SORTBY score
  DIALECT 2

The number after HNSW is the count of key-value arguments that follow, which is a fixed source of errors — the example above has ten (five pairs), and getting it wrong produces a parse error rather than the index you wanted. DIALECT 2 is required for the KNN query syntax.

The @tenant:{acme}=> prefix is a pre-filter, and this is Redis’s genuine advantage over the Postgres arrangement described in filtering and vector search in one query: the query engine applies the tag filter during traversal rather than after it, so the recall cliff at low selectivity is much shallower. Whether that is worth operating a second datastore for depends entirely on how selective your filters actually are.

Why they cannot share an instance

maxmemory-policy is a server-level configuration. It is not per database number, and SELECT 1 does not isolate you from it — the numbered databases share one keyspace manager, one memory limit and one eviction policy.

CONFIG GET maxmemory-policy
1) "maxmemory-policy"
2) "noeviction"

So consider the two choices on a shared instance. Set allkeys-lru and Redis is free to evict a stream entry or an indexed hash under pressure: a job disappears, or a document vanishes from search results, and nothing anywhere reports an error. Set noeviction and your cache stops accepting writes as soon as the instance reaches maxmemory — every SET returns OOM command not allowed when used memory > 'maxmemory', which at least is loud, but your queue producer gets the same error because it is the same instance.

There is no third option and no per-key opt-out. The conclusion is structural: run the cache on its own instance with allkeys-lru and a memory limit it is expected to reach, and run the queue and the vector index on an instance with noeviction and enough memory that reaching the limit is an alert rather than a Tuesday.

• Cache instance: maxmemory 4gb, maxmemory-policy allkeys-lru, persistence off. Losing it entirely costs a cold period, nothing more.
• Queue and index instance: maxmemory-policy noeviction, AOF persistence on, and an alert at seventy per cent of the limit — because the response to filling it is to add memory or shed load, and both take time you need to have.

Measuring memory before it bites

Both instances need a memory limit, and picking one requires knowing what your data actually costs — which for Redis is never just the bytes you stored. Three commands give you the real picture.

# What one key costs, including internal structure overhead.
MEMORY USAGE chunk:9001
(integer) 3288          # a 768-dim float32 vector is 3072 bytes of payload

# The whole picture, including fragmentation.
INFO memory
used_memory_human:3.41G          # what Redis thinks it is holding
used_memory_rss_human:4.802G     # what the operating system has given it
mem_fragmentation_ratio:1.41     # rss / used_memory
maxmemory_human:6.00G

# For a vector index specifically, the index is separate from the data.
FT.INFO idx:chunks               # look at inverted_sz_mb and vector_index_sz_mb

mem_fragmentation_ratio is the number to watch and the one people are surprised by. A ratio around 1.0 to 1.2 is healthy. Anything above about 1.5 means the allocator is holding memory it is not using, and the operating system sees the larger figure — so an instance configured with maxmemory 6gb can be using nine gigabytes of RSS and be killed by the kernel while Redis believes it has headroom. Fragmentation rises with churn, and a vector index with a mix of insertions and deletions is churn by definition.

Two consequences for sizing. Set maxmemory to roughly sixty per cent of the machine’s RAM rather than ninety, so fragmentation and the copy-on-write memory that a background save needs both have somewhere to live. And alert on RSS, not on used_memory, because RSS is what the kernel is looking at when it decides which process to kill.

For the vector index, the same per-element arithmetic from storing embeddings applies — the vector dominates, the graph is a small fraction of it at high dimensions — with one Redis-specific addition: the document itself is stored in a hash alongside the index entry, so the content field counts twice if you also index it as TEXT. Store the text in Postgres and keep only the id and the vector in Redis where you can; the id-based fetch is what the design in caching retrieval results is built around, and it applies here for the same reason.

What survives a restart

Redis is an in-memory store with optional persistence, and the default configuration is weaker than most people assume. RDB snapshots are point-in-time and lose everything since the last one. AOF appends every write, with appendfsync everysec as the usual setting, which bounds the loss at roughly one second of writes.

For the vector index specifically, weigh persistence against rebuild time. A Redis vector index is derived data — you can rebuild it from your system of record — so the question is whether an FT.CREATE plus a full reload is inside your recovery objective. At a few hundred thousand documents it usually is, and turning persistence off makes the instance materially faster. At tens of millions it is not, and you need the AOF. The same reasoning applied across every store you run is backups and restore for AI data.

Whatever you choose, make the rebuild path something you have actually executed rather than something you believe exists. “We can rebuild the index from Postgres” is a claim with a script behind it or it is not a claim; the version where you find out during an outage that the script assumed an environment variable nobody sets any more is the common one.