Ollama's REST API, End to End

Ollama’s API is plain HTTP with newline-delimited JSON, served at http://localhost:11434 by default. There is no authentication, no SDK requirement and no websocket. If you can post JSON you have a complete client, and writing one directly is the fastest way to understand what the wrappers are hiding.

The surface you are talking to

The server binds 127.0.0.1:11434 unless OLLAMA_HOST says otherwise, and it rejects cross-origin browser requests from anywhere not listed in OLLAMA_ORIGINS. The endpoints divide into three groups: inference (/api/generate, /api/chat, /api/embed), model management (/api/tags, /api/show, /api/pull, /api/create, /api/copy, /api/delete, /api/push) and introspection (/api/ps, /api/version).

Two conventions apply everywhere. Model names are model:tag with latest assumed, so qwen3 and qwen3:latest are the same model. And all durations in responses are nanoseconds — total_duration, load_duration, prompt_eval_duration, eval_duration — which is worth fixing in your head before you report a two-second request as two billion milliseconds.

The management endpoints are worth knowing even if you never call them, because they explain the CLI. /api/show returns a model’s modelfile, parameters, template, details and capabilities — which is how a client decides whether a model supports tools or vision without trying and failing. /api/pull streams progress objects rather than blocking, so a UI can show a bar. And /api/ps returns each loaded model with size, size_vram and an expires_at timestamp, which together tell you how much memory you are holding and how long you will hold it.

/api/generate

One prompt, one completion, no message history. The only required field is model; prompt is what you want continued. Useful companions: system and template override what the model carries, suffix supplies text that follows the completion for fill-in-the-middle, images takes base64 strings for multimodal models, format takes either "json" or a full JSON schema, raw disables template application entirely, and options carries the sampling and runner parameters — the same names a Modelfile uses.

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "List three uses for a dead badger.",
  "stream": false,
  "keep_alive": "10m",
  "options": {
    "temperature": 0.2,
    "num_ctx": 8192,
    "num_predict": 200,
    "stop": ["\n\n\n"]
  }
}'

The non-streamed response carries the whole completion in response plus the accounting: done, done_reason, prompt_eval_count (tokens in), eval_count (tokens out), and the four durations. done_reason is the field to log — the documented values include stop for a natural or configured stop, load for a request that only warmed the model, and unload for one that only evicted it.

The context field, an array of token ids you could send back to continue a conversation, is marked deprecated in Ollama’s API reference. Do not build on it; that job belongs to /api/chat.

/api/chat

Same server, different contract: you send the whole conversation every time as a messages array of objects with role and content, and the reply comes back as a single message object rather than a bare string. The endpoint applies the model’s chat template for you, which is the reason to prefer it over /api/generate for anything conversational — the special tokens differ per model and getting them wrong degrades output without erroring.

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:8b",
  "stream": false,
  "messages": [
    { "role": "system", "content": "Answer in one sentence." },
    { "role": "user", "content": "Why is the sea salty?" }
  ],
  "options": { "temperature": 0 }
}'

tools takes JSON-schema function definitions and returns message.tool_calls when the model decides to call one; you run the function and append a tool-role message with the result. think switches reasoning on for models that support it and accepts a boolean or one of "low", "medium", "high", "max". There is no server-side conversation state anywhere in this API: every turn resends everything, which is why num_ctx is the real limit on how long a chat can run.

Reading the stream

Streaming is the default. Leave stream unset and the response is a sequence of JSON objects separated by newlines, one per token chunk, each with done: false, until a final object with done: true carrying the counts and durations. It is not server-sent events: there are no data: prefixes and no blank lines, so an SSE parser will not read it.

Parse it by splitting on newlines and decoding each complete line, and buffer partial lines — a chunk boundary can fall inside a JSON object. The final object is where the useful numbers live, and it gives you a throughput figure for free: Ollama’s API reference states that tokens per second is eval_count / eval_duration x 10^9. That is your own machine reporting its own rate, which is the only tokens per second figure anybody should quote about your hardware.

A script that uses both

1. Confirm the server is up and see what is installed: curl -s http://localhost:11434/api/tags. An empty models array means the server is fine and you have pulled nothing.
2. Pull a model if needed — ollama pull qwen3:8b — and check what is resident with curl -s http://localhost:11434/api/ps.
3. Save this as client.py. It uses only the standard library, calls /api/generate without streaming to read the counters, then /api/chat with streaming to print tokens as they arrive.

   import json, urllib.request
   
   HOST = "http://localhost:11434"
   
   def post(path, payload):
       req = urllib.request.Request(
           HOST + path,
           data=json.dumps(payload).encode(),
           headers={"Content-Type": "application/json"},
       )
       return urllib.request.urlopen(req)
   
   # 1. One-shot completion, no streaming.
   res = post("/api/generate", {
       "model": "qwen3:8b",
       "prompt": "Name the three Baltic states.",
       "stream": False,
       "options": {"temperature": 0},
   }).read()
   data = json.loads(res)
   print(data["response"].strip())
   print("done_reason:", data["done_reason"])
   print("in:", data["prompt_eval_count"], "out:", data["eval_count"])
   rate = data["eval_count"] / data["eval_duration"] * 1e9
   print("tok/s on this machine:", round(rate, 1))
   
   # 2. Streaming chat.
   history = [{"role": "user", "content": "Explain a KV cache in two sentences."}]
   stream = post("/api/chat", {
       "model": "qwen3:8b",
       "messages": history,
       "stream": True,
   })
   reply = ""
   for line in stream:                 # urlopen yields complete lines
       line = line.strip()
       if not line:
           continue
       chunk = json.loads(line)
       if not chunk.get("done"):
           piece = chunk["message"]["content"]
           reply += piece
           print(piece, end="", flush=True)
       else:
           print()
           print("total ms:", chunk["total_duration"] // 1_000_000)
   history.append({"role": "assistant", "content": reply})

4. Run it: python client.py. The first run will include a large load_duration if the model was not resident — see keep_alive for why, and run it twice to see the difference.
5. Extend the chat loop by appending each user message and each assistant reply to history and posting the whole array again. That list is the entire conversation state; there is none on the server.

Three failures are worth handling before you ship anything built on this. A model name that is not installed returns an error rather than pulling it, so a client that assumes availability breaks on a fresh machine — call /api/tags first, or /api/pull and wait. A server under load returns HTTP 503 once the queue is full, and the queue depth is OLLAMA_MAX_QUEUE, documented as defaulting to 512; treat 503 as backpressure and retry rather than as an outage. And a streamed response can end early without done: true if the connection drops, so a reader that only commits its buffer on the final object is the one that does not silently truncate answers.

Ollama also serves an OpenAI-compatible surface under /v1, which is useful when a library refuses to speak anything else. It maps onto the same runners, so everything above about loading, context and keep-alive still applies underneath it.