Skip to content
Multigrid

Embedding Projection Visualiser: How Much the Picture Lost

Projects your vectors to two dimensions by PCA and draws lines to each point's true nearest neighbours in the full space, so you can see where the flattening misleads you.

True nearest neighbours the picture also puts nearest
3 of 3

At this k the projection has not reordered anything. It still only carries 69.9% of the variance — raise k, or change the focus point, and it will.

refund a chargefailed payment retinvoice PDFsrotate an API keyOAuth device flowsession expiryadd a database indconnection poolingread replica lagdark mode tokenson-call rotationchangelog policyPC1 — 37.7% of the variancePC2 — 32.2%
● focus point● a true nearest neighbour, dashed line drawn to it○ looks nearest in this picture but is not
Vectors
12
Dimensions in
6
Dimensions out
2
Variance kept by PC1
37.66%
Variance kept by PC2
32.23%
Variance kept in total
69.89%
Variance discarded
30.11%
True top-3 that the picture agrees with
3 of 3
True neighbour the picture pushed furthest away
changelog policy — 3th in the plot
Where these numbers come from: The twelve default vectors were written by hand on six named axes so the picture has something meaningful to project. They are an illustration and they are not any embedding model's output. The projection itself is real: principal component analysis by power iteration, run in your browser, and the explained-variance figures are the genuine share of squared distance the two axes account for.
What this assumes: PCA on mean-centred vectors, top two components by power iteration with deflation, forty iterations from a fixed start vector so the same input always draws the same picture rather than rotating on every keystroke. Explained variance is the share of total squared deviation captured by each component. Neighbours are ranked by cosine similarity in the full space and by Euclidean distance in the plot — with normalisation on, those two orderings would agree exactly if the projection were lossless, which is precisely what the headline number measures. Ragged rows are zero-padded. Nothing here is t-SNE or UMAP: both would look prettier and would be far less honest, because neither preserves distances at all and both invent cluster structure that depends on their own hyperparameters.

Why the picture is not the space

Embedding visualisations are the most-shared artefact in this field and among the least trustworthy. The problem is not that projection is a bad idea; it is that the projection is presented without the number that says how much it dropped. Six dimensions squeezed into two loses whatever the third, fourth, fifth and sixth were carrying, and real embeddings have 768 or 1,536 or 3,072 of them. The variance figures above are the honest accounting, and on real data two components typically keep somewhere between a tenth and a third.

The dashed lines are the point of the page. They connect the focus point to its genuine nearest neighbours, computed in the full space where retrieval actually happens. When a dashed line stretches right across the plot to a point that looks unrelated, the picture has lied to you — and it will keep lying in the same direction, because PCA preserves the largest global structure at the expense of exactly the local structure that nearest- neighbour search depends on. Raise k and watch the agreement fall away.

The practical consequence: do not debug a retrieval system by looking at a scatter plot. If two chunks sit on top of each other here, that is not evidence they are duplicates, and if a chunk sits far from your query it is not evidence it will not be retrieved. Use the plot to see whether your corpus has broad structure at all — clusters that correspond to things you recognise — and use distances in the full space, or an actual retrieval run, for every decision that matters.

The reasoning behind this

Related tools

Embedding Projection Visualiser: How Much the Picture Lost · Multigrid