Skip to content

Vector Quantisation Preview

Quantise your own embeddings to float16, int8, int4 or one bit and measure what it does to recall@k before you commit the index to it.

Recall@10 after quantising
99.5%

Of the 10 true nearest neighbours of each vector, this many are still in the top 10 when the search runs on the quantised copies. Measured on all 120 vectors, exhaustively — no index, no approximation beyond the quantisation itself.

Vectors × dimensions
120 × 64
Storage before, float32
30,720 bytes
Storage after, 8 bits per dimension
8,640 bytes
Compression
3.56×
Mean cosine to the original vector
0.99999
Worst vector's cosine
0.99998
About this measurement:
  • These are 120 pseudo-random vectors from seed 20260803, arranged in 8 clusters. They are NOT embeddings, and the prose below explains why that makes the figure pessimistic. Switch the source to "paste your own" for a measurement that means something about your data.
What this assumes: recall is measured by leave-one-out: every vector in the set is used as a query against all the others, its true top 10 is computed by exact cosine on the float32 values, and the same query is run against the dequantised copies. Scalar quantisation stores a minimum and a scale alongside the codes — 8 bytes per vector for per-vector scaling, 8 bytes in total for global — and that overhead is included in the size above. Binary quantisation is compared by the sign pattern, which orders candidates identically to Hamming distance. There is no index here: this isolates the loss from quantisation alone, so a real system using HNSW or IVF on top will lose a little more. Everything on this page runs in your browser. Nothing you paste is uploaded, logged or sent anywhere.

Quantising embeddings is the cheapest large win in a vector database: float32 to int8 cuts the index to a quarter of its size, and a quarter of the size is a quarter of the memory bandwidth every query has to walk. The reason people hesitate is that the cost is invisible until it is not — nothing errors, results just get slightly worse in a way no dashboard reports. This page makes the cost visible on your own vectors before you commit to it.

Why the demo set is pessimistic, and says so

The vectors this page opens with are pseudo-random numbers around a few cluster centres, not embeddings. That matters in one direction only: real embeddings have strongly correlated dimensions and a value distribution concentrated near zero, which is exactly the shape scalar quantisation handles well, so real data almost always survives quantisation better than this demo does. Treat the demo number as a floor and the number from your pasted vectors as the answer.

Per-vector or global scaling

A scalar quantiser maps a range of floats onto 256 (or 16) integer levels, so it has to know the range. Taking it per vector fits each row tightly and costs 8 bytes a row; taking it once for the whole set costs 8 bytes total but wastes levels on any vector whose values do not span the full range. At 768 dimensions the per-vector overhead is about 1% of the row, which is usually the better trade — but if your vectors are normalised and similarly distributed, global scaling gives up almost nothing. Switch between them above and watch recall rather than guessing.

Binary is not a smaller int8

One bit per dimension is a 32× reduction and it throws away magnitude entirely, keeping only which side of zero each dimension fell. On high-dimensional embeddings that survives surprisingly well, which is why binary indexes exist — but it is a different regime, and the usual way it is deployed is as a first pass that retrieves a few hundred candidates which are then rescored with the full-precision vectors. Recall@10 on the binary codes alone, which is what this page measures, is the number that tells you how wide that first pass needs to be.

Vector Quantisation Preview · Multigrid