Vector Databases Compared: What Actually Differs

Feature grids comparing vector databases are nearly useless, because every engine implements the same handful of published algorithms and the differences that matter are in filtering, updates and operations — none of which fit in a tick-box.

Four index families, and that is all

Product quantisation (Jégou et al., 2011) is not a fifth family — it is a compression scheme layered under IVF or a graph. When a vendor advertises IVF-PQ or HNSW-SQ, that is family plus compression, and the compression is the part with the recall consequence.

Locality-sensitive hashing appears in older comparisons and has been largely displaced for dense embedding search by graph methods. If a comparison you are reading leans on LSH results, check its date.

How to read a recall/QPS curve

The standard artefact in this field is the ann-benchmarks plot (Aumüller, Bernhardsson and Faithfull) — recall on the x-axis, queries per second on a log y-axis, one line per implementation, each point a different parameter setting. It is the most honest comparison methodology available and it is still routinely misread. Four things to hold on to:

• A point is a parameter setting, not a system. Any index can be made fast at low recall or slow at high recall. The only fair comparison is at equal recall — read the plot by drawing a vertical line at the recall you need and comparing where the curves cross it. Comparing headline QPS numbers from two vendors’ marketing pages compares two different points on two different curves.
• The dataset determines the answer. Standard sets like SIFT-1M and GloVe-100-angular have distributions that are not your distribution. Your text embeddings are higher-dimensional and differently clustered, and index rankings do move between datasets.
• Build time is on a different chart. An index that is marginally faster to query and four times slower to build is a bad trade if your corpus turns over weekly.
• Single-threaded and unfiltered. The benchmark usually measures one query at a time with no metadata filter. That is the opposite of production, where you have concurrency and almost always a filter.

Filtering is the real differentiator

Almost every real query is “nearest neighbours where tenant_id = 42 and status = ’published’”. This is where engines genuinely diverge, and the naive approaches both fail:

Post-filtering searches the index for k neighbours and then discards those failing the predicate. If the predicate keeps 1% of rows, asking for 10 and filtering leaves you an expected 0.1 results. The symptom is a search that returns three rows when you asked for ten, or nothing at all for a small tenant — and it gets worse the more selective the filter, which is the opposite of what anyone expects from a database.

Pre-filtering restricts to matching rows first, but an HNSW graph built over all rows is not navigable over an arbitrary subset: the greedy walk gets stranded when its neighbours are all filtered out. Engines solve this in different ways, and this is the question to ask a vendor instead of asking about their feature grid. Qdrant builds additional graph links for indexed payload values so the subgraph stays connected. Weaviate switches strategy based on the predicate’s estimated selectivity. pgvector 0.8 added iterative index scans (hnsw.iterative_scan, set to strict_order or relaxed_order) so a filtered query keeps pulling more candidates from the index until it has enough — before that version, a selective filter simply returned too few rows.

A useful test costs an hour: load a million rows, add a filter that matches 0.1% of them, and ask for the top 10. Count what comes back and time it. That one query separates these systems more sharply than any published benchmark.

The axes nobody benchmarks

• Deletes. HNSW cannot cheaply remove a node from a graph, so engines tombstone and rebuild. Ask what happens to recall and memory when 30% of your corpus has been deleted but not yet compacted, and what the compaction costs.
• Updates. An update is a delete plus an insert, with the same problem. A corpus that churns daily is a different workload from one that is loaded once.
• Memory at build time. Building an HNSW index needs the graph in memory. In pgvector this is governed by maintenance_work_mem, and when it is too small the build spills and slows by an order of magnitude — it emits a notice saying the graph no longer fits, which is your cue to raise the setting rather than wait.
• Transactions and consistency. If the vector and the row it describes live in different systems, they will disagree during failures. Keeping them in one database is a correctness argument, not a convenience one.
• Backup and restore. Restoring a 200 GB index may mean rebuilding it. Time that before you need it.

Choosing

The decision usually collapses to three cases. If your data already lives in Postgres and you are under roughly ten million vectors, pgvector removes an entire distributed system from your architecture and the transactional consistency is worth more than the last few percent of QPS. If you need heavy metadata filtering, high write churn or hundreds of millions of vectors, a dedicated engine earns its operational cost — choose it on its filtering strategy and its delete story. And if you are under about a hundred thousand vectors, you may not need an index at all, which the next page in this cluster works out in numbers.