Choosing and Tuning a pgvector Index

There are two index types in pgvector and four numbers to set. This page derives what each one costs — memory exactly, build time as a scaling law you extrapolate from a small sample — and gives you the SQL that measures the one thing no formula can give you, which is recall on your corpus.

The two index types, and when each loses

HNSW (pgvector 0.5.0 and later) builds a layered proximity graph. Queries walk it from a sparse top layer down to a dense bottom layer. It gives better recall per unit of query time than IVFFlat at essentially every operating point, and it handles inserts gracefully. Its costs are a slow build and an index that is a full copy of your vectors.

IVFFlat partitions the vectors into lists clusters by k-means and searches only the nearest probes of them. It builds far faster and uses far less memory. It has two real problems: it must be built on populated data, because an empty table gives it nothing to cluster; and its recall degrades as the data drifts away from the centroids it learned, so it needs periodic rebuilds. See embedding drift for what causes that drift.

The honest default is HNSW. Choose IVFFlat when the index must be rebuilt frequently and quickly, when memory is the binding constraint, or when you are indexing tens of millions of rows on a machine that cannot hold an HNSW graph. Otherwise the extra build time buys you something.

How much memory the index needs

This is derivable, and the result is more useful than the rule of thumb everyone repeats. An HNSW index in pgvector stores, per row: a copy of the vector, a set of neighbour pointers, and per-tuple page overhead.

The vector. A vector(d) is 4d + 8 bytes — four bytes per float32 component plus an eight-byte header.

The neighbours. In standard HNSW an element gets up to 2m connections on layer 0 and m on each layer above. The level is drawn geometrically so that the probability of reaching level l is m^−l, which makes the expected number of levels above zero the sum of that series: 1/(m−1). So the expected connection slots per element are:

slots(m) = 2m + m/(m−1)

  m = 16   ->  32 + 1.07  =  33.1 slots
  m = 32   ->  64 + 1.03  =  65.0 slots
  m = 64   -> 128 + 1.02  = 129.0 slots

Each slot holds a tuple pointer, six bytes. Add roughly 32 bytes of index-tuple and line-pointer overhead per element. The whole thing:

bytes_per_row(d, m) ≈ (4d + 8)          vector payload
                    + 6 × slots(m)     neighbour pointers
                    + 32               tuple overhead

Assumptions, all checkable: float32 vectors, 6-byte tuple pointers,
standard HNSW level distribution, no page-fill slack counted.

Now work it. At d = 1536, the dimension of several widely used embedding models, and m = 16:

(4 × 1536 + 8) + 6 × 33.1 + 32
= 6152 + 199 + 32
= 6383 bytes per row

× 1,000,000 rows = 6.38 GB of index

And the interesting part — what happens when you quadruple m to 64, which is the sort of thing tuning guides suggest for higher recall:

d = 1536, m = 64:  6152 + 774 + 32 = 6958 B  ->  6.96 GB   (+9%)
d =  384, m = 16:  1544 + 199 + 32 = 1775 B  ->  1.78 GB
d =  384, m = 64:  1544 + 774 + 32 = 2350 B  ->  2.35 GB   (+32%)

So the conventional warning that raising m is expensive in memory is true at 384 dimensions and largely false at 1536, where the vector copy dominates and the whole graph structure is three per cent of the index. If you are on a large-dimension model and short of recall, m is a cheaper lever than the guides imply. If you are on a small-dimension model, it is not.

Check the derivation against reality on your own table — it takes one query, and if it disagrees with the formula by more than about fifteen per cent, trust the query:

SELECT pg_size_pretty(pg_relation_size('chunks_embedding_hnsw')) AS index_size,
       pg_size_pretty(pg_relation_size('chunks'))               AS heap_size,
       (SELECT count(*) FROM chunks)                            AS rows;

The consequence that matters operationally: for the index to be fast it must live in memory. Size shared_buffers and the machine so that the number above, plus your heap’s working set, fits. An HNSW traversal is a chain of random accesses, and random access to disk is the one pattern where the difference between RAM and NVMe is visible in every single query.

Build time as a scaling law

Nobody can tell you how long your build takes, because it depends on your CPU, your memory bandwidth and your parallel worker count. What can be stated is how it scales, and that is enough to plan a migration.

Building the graph inserts N elements. Each insertion does a greedy descent through the upper layers, then a layer-0 search that keeps ef_construction candidates alive and expands up to m neighbours from each, computing a d-dimensional distance every time. So the work per insertion is proportional to ef_construction × m × d, and the number of hops grows with log N:

build_time ∝ N × log N × ef_construction × m × d

Which gives you four practical rules. Doubling ef_construction roughly doubles build time and does not change index size at all. Doubling m roughly doubles build time and changes index size by the amounts derived above. Doubling the dimension roughly doubles build time. And doubling the rows slightly more than doubles it.

To plan a real build, time a sample and extrapolate. Build the index on a 100,000-row copy, then scale:

T(N) ≈ T(n) × (N/n) × (ln N / ln n)

Sample: 100,000 rows built in 95 seconds.
Target: 10,000,000 rows.

T = 95 × (10,000,000 / 100,000) × (ln 10^7 / ln 10^5)
  = 95 × 100 × (16.12 / 11.51)
  = 95 × 100 × 1.40
  = 13,300 s ≈ 3 h 42 min

Valid only if the target build also fits in maintenance_work_mem.
If it spills, this estimate is worthless and the real time is
several times larger.

That caveat is the whole game. Before starting a large build, set maintenance_work_mem above the memory figure you derived in the previous section — with headroom, because the build holds more than the finished index does — and set max_parallel_maintenance_workers to the cores you can spare. Watch it with:

SELECT phase,
       round(100.0 * blocks_done / nullif(blocks_total, 0), 1) AS pct
FROM pg_stat_progress_create_index;

Measuring recall on your own table

Recall cannot be derived. It depends on the intrinsic dimensionality and clustering of your embeddings, which is a property of your corpus and your model, and any recall number quoted for someone else’s data is a number about someone else’s data. So measure it. This runs in one psql session against your live table:

BEGIN;

-- 200 probe vectors. Held-out real queries are better; if you have a
-- query log, embed 200 of those and use them instead of table rows.
CREATE TEMP TABLE probes AS
  SELECT id, embedding FROM chunks ORDER BY random() LIMIT 200;

-- Ground truth: exact top-10, index forcibly disabled.
SET LOCAL enable_indexscan = off;
CREATE TEMP TABLE truth AS
  SELECT p.id AS probe, t.id AS hit
  FROM probes p
  CROSS JOIN LATERAL (
    SELECT c.id FROM chunks c
    WHERE c.id <> p.id
    ORDER BY c.embedding <=> p.embedding
    LIMIT 10
  ) t;

-- Approximate: the index, at the ef_search you plan to run in production.
SET LOCAL enable_indexscan = on;
SET LOCAL enable_seqscan   = off;
SET LOCAL hnsw.ef_search   = 40;
CREATE TEMP TABLE approx AS
  SELECT p.id AS probe, t.id AS hit
  FROM probes p
  CROSS JOIN LATERAL (
    SELECT c.id FROM chunks c
    WHERE c.id <> p.id
    ORDER BY c.embedding <=> p.embedding
    LIMIT 10
  ) t;

SELECT round(100.0 * count(a.hit) / count(*), 2) AS recall_at_10_pct
FROM truth t
LEFT JOIN approx a ON a.probe = t.probe AND a.hit = t.hit;

ROLLBACK;

Run it at hnsw.ef_search of 20, 40, 100, 200 and 400 and you have your own recall-versus-latency curve, on your own data, in about ten minutes. That curve is the only basis on which to choose these parameters.

ef_search: the knob you tune after shipping

m and ef_construction are baked in at build time; changing either means rebuilding the index. hnsw.ef_search is a session GUC, changeable per query, and it is where almost all of your tuning should happen.

SET LOCAL hnsw.ef_search = 100;   -- default 40, maximum 1000

SELECT id, content
FROM chunks
ORDER BY embedding <=> $1
LIMIT 10;

It sets how many candidates the layer-0 search keeps alive. Query cost is roughly linear in it and recall rises with diminishing returns. It must be at least your LIMIT; setting it to 10 for a top-50 query is a silent recall disaster. Use SET LOCAL inside the transaction rather than SET, or the value survives on a pooled connection and applies to the next unrelated request — a trap covered properly in connection pooling for AI workloads.

Sizing IVFFlat, if you use it

IVFFlat has one build parameter and one query parameter, and the project’s own guidance is a good starting point: lists of rows/1000 up to a million rows, and sqrt(rows) above that; probes starting at sqrt(lists).

-- 5,000,000 rows: lists = sqrt(5e6) ≈ 2236
CREATE INDEX chunks_embedding_ivf
  ON chunks USING ivfflat (embedding vector_cosine_ops)
  WITH (lists = 2236);

SET LOCAL ivfflat.probes = 47;   -- sqrt(2236)

The arithmetic to understand: with lists partitions over N rows, each partition holds about N/lists vectors, and searching probes of them scans about probes × N / lists vectors exactly. At the numbers above that is 47 × 5,000,000 / 2236 ≈ 105,000 vectors per query, or two per cent of the table. Raise probes to lists and you have reinvented the sequential scan with extra steps.

Recall failures in IVFFlat have one dominant cause: the true nearest neighbour sits in a cluster you did not probe, and no amount of re-running the query will find it. Rebuild the index after any large load, use the recall harness above unchanged — it does not care which index type you have — and if you cannot reach acceptable recall inside your latency budget, that is the signal to move to HNSW.

Which knob to turn, and in what order

The four parameters are not equally worth your attention, and the derivations above say why. Work down this list and stop as soon as the recall harness reports a number you can live with.

1. hnsw.ef_search, first and usually last. It is free to change, applies per query, and requires no rebuild. Query cost is roughly linear in it, so going from 40 to 100 costs you about two and a half times the traversal work — which on an index that is resident in memory is single-digit milliseconds becoming slightly more single-digit milliseconds. Most teams who think they have an index-tuning problem have an ef_search of 40 and a recall requirement that needs 200.
2. ef_construction, if raising ef_search plateaus. It changes the quality of the graph rather than the effort spent searching it, so it raises the ceiling that ef_search is pushing against. It costs build time linearly and index size not at all, which makes it the cheapest of the two rebuild-requiring parameters. Going from 64 to 200 is a common and defensible move.
3. m, if the graph itself is too sparse. Raising it adds connections, which helps most when your embeddings are high-dimensional and poorly clustered. Use the memory derivation above to decide whether it is affordable: at 1536 dimensions, quadrupling m costs nine per cent of index size; at 384 dimensions it costs a third. That asymmetry is the whole reason to do the arithmetic rather than follow a rule of thumb.
4. The dimension, which is not a pgvector parameter at all. Halving it halves index memory, halves build time and roughly halves query cost, all at once. If your model supports truncation this is by some distance the largest lever on the page, and it belongs in the conversation before any of the three above.

One thing to resist: changing two parameters and rebuilding once. Rebuilds are expensive enough that the temptation to bundle them is strong, and it leaves you unable to attribute the result. Change one, measure with the harness, write the number down. Four measured points are worth more than a configuration somebody arrived at by intuition and cannot now justify.