Ollama's Embeddings API, End to End

Embeddings are the local workload that most deserves to be local: small models, no generation loop, and a corpus you may not want to send anywhere. Ollama serves them over the same HTTP surface as everything else, with two endpoints where you might expect one.

Two endpoints, one of them superseded

POST /api/embed is current. POST /api/embeddings still works and is marked in Ollama’s API reference as superseded by the first. They are not interchangeable, and the differences are exactly the ones that break a copy-pasted example:

• The input field is input on the new endpoint and prompt on the old one.
• input accepts a string or an array of strings; prompt takes one string only.
• The response key is embeddings, a list of vectors, on the new endpoint, and embedding, a single vector, on the old one. A client that reads data["embedding"] from /api/embed gets a key error rather than a wrong answer, which is the kinder failure.
• Only the new endpoint carries truncate and dimensions.

Write new code against /api/embed. Migrating old code is mechanical: rename the field, wrap the input in a list, and take embeddings[0] instead of embedding.

Request and response shape

curl http://localhost:11434/api/embed -d '{
  "model": "nomic-embed-text",
  "input": [
    "The kettle is boiling.",
    "Water is being heated in a kettle.",
    "The share price closed down four percent."
  ],
  "truncate": true,
  "keep_alive": "10m"
}'

The response holds embeddings as a list of float arrays in the same order as your input, plus total_duration, load_duration and prompt_eval_count — the last being the token count across the batch, which is the number to watch if you are budgeting a large ingest.

Three optional fields do real work. truncate defaults to true and silently cuts the end of any input longer than the model’s context; setting it to false turns that silence into an error, which is what you want during development. dimensions requests a shorter vector from models trained to support it, which is a storage decision rather than a quality-free one. And keep_alive behaves exactly as it does for generation, defaulting to five minutes — worth setting long for a bulk ingest so the model is not reloaded between batches.

Batching matters more here than it does for generation. An embedding model does one forward pass per input and produces no tokens, so the per-request overhead — HTTP, JSON, scheduling — is a large fraction of the work. Sending a thousand strings as a thousand requests and sending them as a small number of arrays are the same arithmetic on the GPU and very different wall-clock times. The limit on batch size is memory rather than policy: a batch is embedded together, so an enormous array of long documents can exhaust the runner where the same documents in ten batches would not.

Picking a local embedding model

Embedding models are small enough that the usual local constraints barely apply. The weights layer of all-minilm in Ollama’s registry is 45,949,216 bytes — about 44 MiB — and nomic-embed-text is 274,290,656 bytes, about 262 MiB. Both fit in the memory of anything, which means the choice is entirely about output quality and vector width rather than about hardware.

The three things to check before committing a corpus. Vector width, because it fixes your index size and cannot be changed without re-embedding everything. Whether the model is asymmetric — some models expect a prefix distinguishing a query from a document, and using one without the prefix quietly degrades retrieval. And the trained input length, since truncate defaults to on and a model with a short window will discard the tail of every long document without telling you.

Whatever you pick becomes a commitment: re-embedding a corpus is the expensive operation in a retrieval system, and embedding dimensions covers the storage side of the same decision.

A script that builds and compares vectors

1. Pull a model: ollama pull nomic-embed-text.
2. Save this as embed.py. Standard library only — it embeds a batch in one call and ranks the batch against a query by cosine similarity.

   import json, math, urllib.request
   
   HOST = "http://localhost:11434"
   MODEL = "nomic-embed-text"
   
   def embed(texts):
       body = json.dumps({
           "model": MODEL,
           "input": texts,
           "truncate": False,      # error instead of silently cutting
           "keep_alive": "10m",
       }).encode()
       req = urllib.request.Request(
           HOST + "/api/embed",
           data=body,
           headers={"Content-Type": "application/json"},
       )
       return json.loads(urllib.request.urlopen(req).read())["embeddings"]
   
   def cosine(a, b):
       dot = sum(x * y for x, y in zip(a, b))
       na = math.sqrt(sum(x * x for x in a))
       nb = math.sqrt(sum(y * y for y in b))
       return dot / (na * nb)
   
   docs = [
       "The kettle is boiling.",
       "Water is being heated in a kettle.",
       "The share price closed down four percent.",
   ]
   query = "How do I make tea?"
   
   vectors = embed(docs + [query])
   doc_vecs, q = vectors[:-1], vectors[-1]
   
   print("dimensions:", len(q))
   for text, v in sorted(
       zip(docs, doc_vecs), key=lambda p: -cosine(q, p[1])
   ):
       print(round(cosine(q, v), 4), text)

3. Run it: python embed.py. It prints the vector width first — that is the number your vector store needs — and then the documents ranked against the query.
4. Sanity-check the ranking rather than trusting it. The two kettle sentences should score close together and the share-price sentence should score clearly lower. If all three scores sit within a hundredth of each other, the model is not producing usable separation on your text and no amount of index tuning will rescue it. Absolute similarity numbers mean little across models; the ordering and the gaps are what you are checking.
5. For a real ingest, batch by sending a list rather than looping one string at a time, and set a long keep_alive so the model stays resident between batches.

The traps that produce useless vectors

• Assuming normalised output. Not every model returns unit-length vectors, so a raw dot product is not a cosine similarity. Normalise, or compute cosine explicitly as above.
• Mixing models in one index. Vectors from two different embedding models are not comparable even at the same width. One index, one model, one version.
• Leaving truncate at its default during development. Long documents are cut at the tail and the request still succeeds, so the failure looks like poor retrieval rather than like data loss.
• Ignoring the asymmetry. If the model wants a query prefix and a document prefix, embedding both sides identically costs you accuracy that no reranker fully recovers.
• Embedding whole documents. A vector is a fixed budget of meaning; a 40-page document averaged into 768 numbers retrieves nothing well. Chunk first, embed the chunks, and keep the chunk boundaries stable so a re-ingest does not reshuffle the index.