Mistral's Embeddings Model: Dimension and Token Limit per Request

Two figures decide how you build on Mistral’s embeddings endpoint: how many dimensions come back, which fixes your vector store schema, and how many tokens go in, which fixes your chunk size. Get the second wrong and you find out at ingestion time, on the one document that matters.

The two numbers

Mistral documents its mistral-embed model with an output dimension of 1,024 and a maximum input length of 8,192 tokens, in the embeddings section of Mistral’s documentation. Both are properties of the model rather than of your account, and neither is configurable on the request — there is no dimensions parameter to shorten the vector the way some other embedding APIs offer.

Of the two, the dimension is the one that is expensive to get wrong, because it is baked into a database column. 1,024 float32 values is 4KB per vector before any index overhead, so a million chunks is about 4GB of raw vectors. If you are choosing an index type or a quantisation scheme, that is the number to size against.

Verifying them from a response

One request tells you both, and it is worth running as a startup assertion rather than trusting a page:

import os
from mistralai import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

resp = client.embeddings.create(
    model="mistral-embed",
    inputs=["a short probe string"],
)

vec = resp.data[0].embedding
print(len(vec))              # the dimension, from the model itself
print(resp.usage.prompt_tokens)
assert len(vec) == EXPECTED_DIM, "embedding dimension changed"

Asserting on the dimension at startup turns a silent corruption into a crash. If a model change ever altered it, the failure without that assertion is vectors of two different lengths in one collection, which most stores reject at write time and some accept and then return nonsense from.

What happens when an input is too long

The API returns an error rather than silently truncating. That is the behaviour you want, and it is worth stating plainly because the alternative — a truncating endpoint — is far more dangerous: you would get a valid-looking vector representing the first 8,192 tokens of a document and nothing about the rest, with no indication anything was dropped. A retrieval system built on that fails in a way that never surfaces as an error, only as answers that mysteriously ignore the second half of every long document.

Because it errors, the failure lands in your ingestion pipeline, which is where you can handle it. Handle it by splitting and re-submitting, not by catching and skipping — a skipped document is a hole in the index that nobody notices until a user asks about it.

Note also that embedding requests are billed on input tokens and there are no output tokens to pay for, which makes them cheap per call but easy to run up in volume: a corpus of a million chunks at several hundred tokens each is a few hundred million tokens in a single ingestion run. Estimate that number before you start the job rather than after, and check the usage field on the first few batches against your estimate.

Sizing chunks against the cap

8,192 tokens is roughly 30,000 characters of English prose, which is a long document section — perhaps ten pages. Almost nobody should be embedding chunks that large, and the cap is rarely the binding constraint on chunk size. Retrieval quality is.

A single vector represents an entire chunk as one point. Embed ten pages and the vector is an average of ten pages of meaning, which is close to nothing in particular and matches everything weakly. Typical working sizes are a few hundred to around a thousand tokens, with some overlap between adjacent chunks so a sentence that straddles a boundary is intact in at least one of them.

• Split on structure first. Headings, paragraphs, list items, code blocks. A chunk that is a coherent unit of meaning retrieves better than a chunk that is 800 tokens ending mid-sentence.
• Then enforce the token cap as a hard backstop. Count tokens rather than characters — the ratio varies with language and with content, and code and non-Latin scripts tokenise far less efficiently than English prose.
• Log the distribution of your chunk sizes once. If the 99th percentile is nowhere near 8,192, the cap is not your problem and you can stop thinking about it. If something is pushing against it, that is usually one pathological input — a minified file, a base64 blob — that should not be in the index at all.

Batching several inputs per request

The inputs field takes an array, and sending many strings in one request is substantially faster than one request per string, because you pay the round trip once. Two limits apply at once and both are easy to overlook: each individual input must be under the per-input token cap, and the request as a whole is subject to a payload size limit and to your rate limits.

def embed_all(client, texts, model="mistral-embed", batch=64):
    out = []
    for i in range(0, len(texts), batch):
        chunk = texts[i:i + batch]
        resp = client.embeddings.create(model=model, inputs=chunk)
        # Order is preserved, but index is returned explicitly — use it.
        for item in sorted(resp.data, key=lambda d: d.index):
            out.append(item.embedding)
    return out

Each returned object carries an index giving its position in the input array. Sorting on it rather than assuming order costs nothing and removes an entire class of catastrophic, silent bug — embeddings attached to the wrong documents produce a search index that is subtly wrong everywhere and looks fine in every unit test.

Start with a batch size in the tens rather than the hundreds. The failure mode of an over-large batch is a rejected request that costs you the whole batch, and re-embedding is charged.

Two operational habits make a large ingestion survivable. First, make the loop resumable: write each batch’s vectors to the store before requesting the next, and key on a stable document identifier so a restart skips what is already there. An ingestion that has to begin again from zero after a rate-limit error at the ninety percent mark is a bill you pay twice. Second, retry with backoff on 429 and on 5xx, but not on 400 — a rejected batch that is too long or malformed will be rejected identically forever, and retrying it burns your rate limit without ever succeeding. Split it instead, or log it and move on with a record of what was skipped.

Finally, the constraint nobody documents because it is not in the API: embeddings from different models are not comparable. A vector produced by one model and a vector produced by another occupy unrelated spaces even when they have the same dimension, so a cosine similarity between them is a number with no meaning. If you ever change embedding models, you re-embed the entire corpus — there is no incremental migration. That is the real reason to check the model id and the dimension at startup and to store the model id alongside the vectors: it makes a half-migrated index detectable instead of silently wrong.