What Happens to Your Index When Embedding Dimensions Do Not Match

You pointed the ingest pipeline at a new embedding model and the first insert failed. The error is unambiguous, the cause is a schema decision made months ago, and there are exactly two fixes — plus a third that appears to work and does not.

The error, and what threw it

In Postgres with pgvector, the message is:

ERROR:  expected 1536 dimensions, not 768

Through a client library or an ORM the same failure often arrives wrapped, as a DataException or a driver-level data error carrying that string verbatim, which is why searching for the sentence rather than the exception class finds the answer. Other vector stores phrase it differently — a vector dimension error naming the expected and received sizes, or a rejected upsert naming the collection’s configured size — but every one of them is the same check: the store was told how wide a vector is, and you handed it a different width.

Before fixing it, notice which direction the two numbers point. “Expected 1536, not 768” means your index was built for a 1,536-dimension model and the new one emits 768. That is a loud, immediate, first-row failure and it is the best possible outcome of changing an embedding model, because it stopped you. The genuinely expensive case is a new model that happens to emit the same width, in which case nothing errors and the two spaces silently fail to be comparable while every layer of the stack reports success.

Why the column has a width at all

A vector column is not a variable-length array with a convenient type name. The width is part of the type, because everything downstream depends on it: the on-disk row layout is fixed-size, the distance operators are compiled against a known length, and an approximate index allocates its graph or its centroid lists for a specific dimensionality. In pgvector the declaration looks like this, and the number in the type is the number in the error:

CREATE TABLE chunks (
  id          bigint PRIMARY KEY,
  document_id bigint NOT NULL,
  content     text   NOT NULL,
  embedding   vector(1536) NOT NULL
);

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

The pgvector documentation puts the ceiling for the vector type at 16,000 dimensions, so 768 or 3,072 are nowhere near a type limit; the constraint being enforced is only the one you declared. That also means the error is not telling you the new model is unsupported. It is telling you that this table belongs to the old model, which is exactly the fact the rest of this page turns into a plan.

Fix A: a new column and a new index

The correct fix in almost every case is to stop trying to make the old index accept the new vectors and instead build a second one. A new column on the same table is the smallest version:

ALTER TABLE chunks ADD COLUMN embedding_v2 vector(3072);

-- backfill in batches, resumable, ordered by primary key
UPDATE chunks
   SET embedding_v2 = $1
 WHERE id = $2;

-- once the backfill is complete and verified:
CREATE INDEX CONCURRENTLY chunks_embedding_v2_hnsw
    ON chunks USING hnsw ((embedding_v2::halfvec(3072)) halfvec_cosine_ops);

Three details in there are load-bearing. The old column stays, so reads keep working for the entire backfill and reverting is a change of which column the query builder names. The index is created after the backfill rather than before, because building an approximate index once over a finished table is faster than maintaining it through three million updates. And the index is created concurrently, so the table is not locked against your ingest path while it builds.

A separate table or a separate collection is the larger version of the same move and is usually better once the corpus is big: it keeps the two generations physically apart, makes the disk cost visible as its own object, and makes deletion a drop rather than a rewrite. Either way the read path must select the index by configuration rather than by code, which is the subject of the rollback plan.

Fix B: ask the new model for a narrower vector

Some embedding APIs let you request an output width shorter than the model’s native one. OpenAI’s embeddings endpoint exposes a dimensions parameter for its third-generation embedding models, which OpenAI describes in its announcement of those models as a consequence of training them with Matryoshka representation learning: the most important information is concentrated at the front of the vector, so a prefix of it remains usable. OpenAI’s post introducing the models states their native sizes as 1,536 and 3,072 dimensions.

POST https://api.openai.com/v1/embeddings

{
  "model": "text-embedding-3-large",
  "input": ["...chunk text..."],
  "dimensions": 1536,
  "encoding_format": "float"
}

If the width you request equals the width your column already has, this makes the error go away without a schema change. Be clear about what it does and does not buy you. It saves the schema migration, the second index, and the extra storage. It does not save the re-embedding: you are still calling the model once per chunk over the whole corpus, and the vectors it returns are still in a new space that cannot be compared with the old ones. The saving is in DDL, not in tokens, and if your motivation for switching models was quality, you have also chosen to take a compressed projection of the new model rather than its best output.

The parameter is also model-specific. It applies to families trained for it, and asking for it on a model that was not gets you an error rather than a shorter vector. If the new model has no such parameter, fix A is the only fix.

Why padding and truncating are not fixes

The workaround that appears in every thread about this error is to pad the 768-dimension vector with 768 zeros, or to lop the tail off a 3,072-dimension vector, so it fits the column. The uncomfortable part is that zero-padding is numerically harmless: appending zeros changes neither the dot product between two padded vectors nor either vector’s norm, so cosine similarity among the padded set is exactly what it was among the unpadded set. It genuinely does nothing wrong.

That is precisely why it is dangerous. It removes the error while leaving the actual problem — that the vectors already in the index came from a different model — completely untouched. You now have a column of width 1,536 containing two incompatible populations and no check anywhere that will ever mention it again. The error you silenced was the only thing standing between you and the silent failure described on the previous page.

Truncation is worse, because it is not even numerically neutral. Unless the model was explicitly trained so that prefixes are usable, its dimensions are in no particular order of importance and cutting the last half discards an arbitrary half of the information. Renormalise after any client-side truncation if you do it at all — the truncated vector is no longer unit length, so any code path assuming normalised inputs, including an inner-product index configured as a stand-in for cosine, will quietly mis-rank. See vector normalisation for what depends on that assumption.

The second limit: the index, not the column

There is a second width limit, it is lower than the column limit, and it bites after the migration appears to have succeeded. pgvector’s README gives the vector type a 16,000-dimension ceiling, but its HNSW and IVFFlat indexes support up to 2,000 dimensions for vector and up to 4,000 for the half-precision halfvec type. A 3,072-dimension model therefore stores fine and indexes not at all: the CREATE INDEX fails, and if you skip the index you get a sequential scan whose latency grows linearly with the corpus.

Three ways out, in the order most people should consider them. Request a narrower vector with the dimensions parameter if the model supports it, which keeps you under 2,000 and keeps full precision. Cast to halfvec in the index expression, as in the DDL above, which keeps all 3,072 dimensions at roughly half the storage and half the memory footprint of the graph, at the cost of some precision. Or use quantisation more aggressively, with a binary index for a first-pass shortlist and an exact rescoring step over the full-precision vectors for the top few hundred candidates.

Whichever you pick, decide it before the backfill rather than after. The choice changes the width you write, and discovering it at CREATE INDEX time means re-running the whole backfill you just paid for — which, as the cost derivation shows, is usually more painful in wall-clock hours against a rate limit than it is in money.