What Are Embeddings? A Practical Introduction

A user types “how do I stop being billed”. Your help centre contains a page titled “Cancelling your subscription”. Keyword search returns nothing, because the two strings share only the word “I”. That gap is the entire reason embeddings exist, and the fastest way to understand them is to close it.

The search box that fails

Classical search matches tokens. A query and a document are compared by the words they have in common, weighted by how rare those words are — that is BM25, and it is very good at what it does. What it cannot do is notice that billed and subscription are about the same thing, because nothing in the index says so. You can bolt on a synonym list, and people did for twenty years, but a synonym list is a finite hand-written artefact and language is not.

An embedding model replaces the synonym list with a learned function. It takes a piece of text and returns a point in a high-dimensional space, trained so that texts humans consider related land near each other. The help page and the query both become points; the points are close; the search works. No word had to be shared.

What the model actually returns

Concretely, you send text and get back an array of floating-point numbers of a fixed length. That length is the model’s dimension, and it does not vary with the length of your input:

POST /v1/embeddings
{ "model": "text-embedding-3-small", "input": "Cancelling your subscription" }

{
  "data": [{
    "embedding": [-0.0132, 0.0441, -0.0079, ... ],  // 1536 numbers
    "index": 0
  }],
  "usage": { "prompt_tokens": 4, "total_tokens": 4 }
}

Three things in that response are worth pausing on. The array is 1536 long whether the input was four tokens or four thousand — a fixed-size summary is the whole trick, and it is also the whole loss. You are billed on input tokens only; there is no output token count, because the output is not tokens. And there is a hard ceiling on input length: OpenAI documents 8,191 tokens for the text-embedding-3 family, and an over-long input is rejected with an invalid_request_error rather than being silently truncated. That rejection is a favour. A model that quietly dropped the second half of your document would give you an index that is wrong in a way nothing surfaces.

At 1536 dimensions and four bytes per float32, one vector is 6,144 bytes. That is roughly two pages of plain text, to represent a paragraph. Storage of embeddings is almost always larger than storage of the text they came from, which surprises people the first time.

Why nearness means anything

Nothing about a list of numbers is inherently meaningful. The meaning comes from how the model was trained: it was shown pairs of texts known to be related — a question and its answer, a title and its article, a sentence and its translation — and optimised so that the vectors for a true pair score higher against each other than against a few hundred random other texts in the same batch. This is contrastive training, and the objective is literally a ranking objective. The model is not taught what “subscription” means; it is taught to put things that go together nearby.

That has a consequence people skip past: the space is only organised along the axes the training pairs cared about. A model trained on web question/answer pairs arranges text by topic. It has no reason to separate “the deployment succeeded” from “the deployment failed”, because both are about deployments and no training pair ever punished it for confusing them. Negation and polarity are the classic embarrassment of embedding retrieval, and they are not a bug — they are the objective working as specified.

A similarity, computed by hand

Cosine similarity is the angle between two vectors: the dot product divided by the two lengths. In four toy dimensions, take a = [0.2, 0.9, 0.1, 0.3] for the help page, b = [0.3, 0.8, 0.0, 0.4] for the billing query, and c = [0.9, 0.1, 0.4, 0.2] for an unrelated page about API keys.

a . b = 0.2*0.3 + 0.9*0.8 + 0.1*0.0 + 0.3*0.4 = 0.06 + 0.72 + 0 + 0.12 = 0.90
|a|   = sqrt(0.04 + 0.81 + 0.01 + 0.09) = sqrt(0.95) = 0.9747
|b|   = sqrt(0.09 + 0.64 + 0.00 + 0.16) = sqrt(0.89) = 0.9434
cos(a,b) = 0.90 / (0.9747 * 0.9434) = 0.90 / 0.9196 = 0.979

a . c = 0.18 + 0.09 + 0.04 + 0.06 = 0.37
|c|   = sqrt(0.81 + 0.01 + 0.16 + 0.04) = sqrt(1.02) = 1.0100
cos(a,c) = 0.37 / (0.9747 * 1.0100) = 0.37 / 0.9844 = 0.376

0.979 against 0.376. The ranking is the answer; the numbers themselves are not calibrated and mean nothing in isolation. A cosine of 0.82 is not “82% relevant” — some models compress almost everything into 0.7–0.95 and others spread across 0.1–0.9, so a threshold tuned on one model is meaningless on another. Compare scores within one model, never across two.

What embeddings are bad at

• Exact identifiers. Order number INV-2024-88190 is a token sequence the model has no semantics for. It will happily return INV-2024-88191. Anything that must match exactly belongs in a keyword index or a WHERE clause, not in a vector.
• Negation. “deployments that did not fail” and “failed deployments” land close together.
• Rare proper nouns. Your internal codename for a project appeared zero times in training. It is embedded from its subword pieces, which is to say arbitrarily.
• Numeric and temporal comparison. “under €50” is not a region of the space. Filter on a column.
• Long documents. One vector for a 40-page PDF averages away everything specific in it. Chunking is not an implementation detail; it is the main quality lever most teams have.

The four moving parts

Every page in this cluster is about one of those four boxes or about the seams between them. If you read only one more, make it the one on where embeddings lose to BM25, because the systems that work in production are almost never embeddings alone.