Vector Quantisation Preview
Quantise your own embeddings to float16, int8, int4 or one bit and measure what it does to recall@k before you commit the index to it.
Of the 10 true nearest neighbours of each vector, this many are still in the top 10 when the search runs on the quantised copies. Measured on all 120 vectors, exhaustively — no index, no approximation beyond the quantisation itself.
- Vectors × dimensions
- 120 × 64
- Storage before, float32
- 30,720 bytes
- Storage after, 8 bits per dimension
- 8,640 bytes
- Compression
- 3.56×
- Mean cosine to the original vector
- 0.99999
- Worst vector's cosine
- 0.99998
- These are 120 pseudo-random vectors from seed 20260803, arranged in 8 clusters. They are NOT embeddings, and the prose below explains why that makes the figure pessimistic. Switch the source to "paste your own" for a measurement that means something about your data.
Quantising embeddings is the cheapest large win in a vector database: float32 to int8 cuts the index to a quarter of its size, and a quarter of the size is a quarter of the memory bandwidth every query has to walk. The reason people hesitate is that the cost is invisible until it is not — nothing errors, results just get slightly worse in a way no dashboard reports. This page makes the cost visible on your own vectors before you commit to it.
Why the demo set is pessimistic, and says so
The vectors this page opens with are pseudo-random numbers around a few cluster centres, not embeddings. That matters in one direction only: real embeddings have strongly correlated dimensions and a value distribution concentrated near zero, which is exactly the shape scalar quantisation handles well, so real data almost always survives quantisation better than this demo does. Treat the demo number as a floor and the number from your pasted vectors as the answer.
Per-vector or global scaling
A scalar quantiser maps a range of floats onto 256 (or 16) integer levels, so it has to know the range. Taking it per vector fits each row tightly and costs 8 bytes a row; taking it once for the whole set costs 8 bytes total but wastes levels on any vector whose values do not span the full range. At 768 dimensions the per-vector overhead is about 1% of the row, which is usually the better trade — but if your vectors are normalised and similarly distributed, global scaling gives up almost nothing. Switch between them above and watch recall rather than guessing.
Binary is not a smaller int8
One bit per dimension is a 32× reduction and it throws away magnitude entirely, keeping only which side of zero each dimension fell. On high-dimensional embeddings that survives surprisingly well, which is why binary indexes exist — but it is a different regime, and the usual way it is deployed is as a first pass that retrieves a few hundred candidates which are then rescored with the full-precision vectors. Recall@10 on the binary codes alone, which is what this page measures, is the number that tells you how wide that first pass needs to be.