Setting a System Prompt in an Ollama Modelfile

The SYSTEM instruction is one line and almost always works. The interesting part is the small fraction of cases where it appears not to, because in every one of them the string was stored correctly and something downstream discarded it.

Where a system prompt can live

Four places, with a clear precedence, and knowing which one you are using is most of the debugging:

• In the Modelfile, as SYSTEM. Compiled into its own layer, travels with the model, applies to every request anybody makes to it. This is the durable one.
• In the request. For /api/chat, a message with "role": "system"; for /api/generate, the top-level system field, which Ollama’s API reference describes as overriding what is defined in the Modelfile. Per request, wins.
• In the REPL, as /set system "...". Session-scoped, and /save mymodel writes the current session into a new model if you want to keep it.
• Nowhere, which is the default for a model imported from a bare GGUF, and often for library models too.

A fifth option exists and is worth ruling out first: MESSAGE lines in a Modelfile, which prepend example turns rather than a standing instruction. A system message says how to behave; a pair of MESSAGE user and MESSAGE assistant lines shows it. For formatting rules that are easier to demonstrate than to describe — a particular JSON shape, a house style of summary — the examples often hold better than another paragraph of prose, and the two combine freely in one file.

Build a model that carries one

Triple quotes for anything spanning more than one line. Without them, only the first line becomes the system message and the remainder is parsed as instructions Ollama does not recognise.

FROM qwen3:8b

SYSTEM """You are a release-notes editor.

Rewrite each changelog entry as one sentence in the past tense,
starting with a verb, under 120 characters, with no marketing language.
If an entry is already in that form, return it unchanged.
Never invent a version number or a date."""

PARAMETER temperature 0.2
PARAMETER top_p 0.9

1. Save that as Modelfile in an empty directory and change into it.
2. Build it: ollama create relnotes. The -f flag defaults to Modelfile, so it is optional when the name matches.
3. Read the stored layer back, not the file you wrote: ollama show --system relnotes prints the system message exactly as it was compiled, including the line breaks. If this prints nothing, the triple quotes are missing or unbalanced and nothing else is worth trying yet.

Verify it in a fresh session

Verifying in the session that created the model proves nothing — anything you typed is still in the context. The check that means something is a new process with no history:

1. Unload the model so nothing is carried over: ollama stop relnotes.
2. Send a single-shot request from the shell, with no prior turns:

   ollama run relnotes "we have added, at long last, a brand new and totally revolutionary dark mode!!"

   The reply should be one past-tense sentence with the exclamation marks and the marketing gone.
3. Confirm over HTTP too, since that is how an application will reach it, and send no system message of your own:

   curl http://localhost:11434/api/chat -d '{
     "model": "relnotes",
     "stream": false,
     "messages": [
       { "role": "user", "content": "fixed the thing where the sidebar was broken sometimes" }
     ]
   }'

4. Now test the override deliberately: add a system message of your own to the same request and watch the Modelfile’s message lose. That is not a bug, and it is the mechanism behind most reports that a baked-in prompt has stopped working — a framework inserted its own system message on your behalf.

Why it sometimes appears to be ignored

There are four distinct causes and they need different fixes.

The template has no system branch. The system message reaches the template as {{ .System }}, and if the template never references it — common with models imported from a bare GGUF — the string is stored, rendered nowhere, and never seen by the model. ollama show --template relnotes tells you in one command; if there is no .System in the output, fix the template rather than the prompt. See Ollama’s TEMPLATE field.

The caller is overriding it. Most chat UIs, agent frameworks and IDE plugins insert their own system message, which by documented precedence replaces yours. The tell is that the model behaves correctly from ollama run and incorrectly from the application.

It fell out of the context. The system message is tokens like any other. In a long conversation against a small window, truncation can push it out, after which the model is genuinely no longer seeing it. If behaviour degrades gradually over a long session rather than being wrong from the first turn, suspect this and check num_ctx.

Diagnosing which of the four you have takes two commands and no guessing. ollama show --system answers whether the string was stored; ollama show --template answers whether anything renders it. If both are fine and a single-shot ollama run behaves correctly, the problem is on the caller’s side of the HTTP request and not in the model at all.

The model simply did not comply. A system message is a strong prior, not an enforcement mechanism. Small models follow long multi-clause instructions less reliably than large ones, and a contradictory user turn will often win. This is the only one of the four that is a prompt problem, and it is the one people assume first.

Writing one that survives contact

A few properties make the difference between a system prompt that holds and one that decays. Keep it short: every token spends context that a long conversation will eventually want, and a 600-word system prompt is both more expensive and less reliably followed than a 60-word one. State the output format concretely — a sentence count, a character limit, a shape — because a checkable instruction is one the model can satisfy unambiguously. Say what to do rather than only what to avoid, since a negative instruction still puts the forbidden thing in the context. And put the constraint you care about most at the end, where recency works for you.

Then treat the model as a versioned artefact. Rebuilding under the same name replaces the manifest and keeps no history, so keep the Modelfile in version control alongside the code that calls it — the Modelfile is the source, and the built model is the compiled output. Building variants under distinct names (relnotes-terse, relnotes-verbose) costs a manifest each and shares the weights, so having several is nearly free on disk.