The TEMPLATE Field in an Ollama Modelfile

A language model does not take a list of messages. It takes one string of tokens. TEMPLATE is the code that turns the first into the second, and because the model was trained against one particular way of doing that, a template which differs from it degrades output in ways that look like the model being bad rather than the formatting being wrong.

The job TEMPLATE does

When you call /api/chat with a system message and two turns of conversation, something has to decide what text actually reaches the model: which special tokens delimit a turn, where the system message goes, what marks the point at which the assistant should begin speaking. Ollama’s Modelfile reference describes TEMPLATE as the full prompt template passed into the model, which may optionally include a system message, a user’s message and the response from the model. It is written in Go template syntax and delimited with triple quotes when it spans lines.

The reference’s own example makes the shape obvious — this is the ChatML convention, and the literal <|im_start|> strings are tokens the model was trained to treat as structure:

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

Read the end of it carefully, because it is the part people miss. The template stops after the assistant header with nothing following. That trailing fragment is the prompt for generation: the model is being handed a string that ends mid-structure and asked to continue it. Every chat template ends this way, and a template that closes the assistant turn instead of opening it gives the model no reason to say anything.

What the template can see

The reference documents three core variables. {{ .System }} is the system message used to specify custom behaviour; {{ .Prompt }} is the user prompt message; and {{ .Response }} is the response from the model, where the documented behaviour is that text after this variable is omitted when generating.

That last clause is doing more work than it appears to. It is how a single template serves two purposes: rendered for generation it stops at .Response, and rendered while replaying a prior turn it continues past it so the closing token is present in the history. You do not write two templates.

Templates that handle real multi-turn chat and tool calling iterate instead, over a .Messages collection whose entries carry a role and content, with .Tools available where the model was trained to emit tool calls. The Go template rules apply and the one that bites is scope: inside a range, the dot refers to the current message rather than the top-level data, so a reference to the system message from inside the loop has to reach back out with $. Whitespace matters too, since every newline in the template is a real token; the - trim markers exist because of that.

What a wrong template looks like

A template mismatch does not raise an error. Nothing validates that your delimiters are the ones the model saw during training, so the model receives a string in a format it has never been fine-tuned on and does its best. The symptoms are specific enough to diagnose from.

• It does not stop. The model finishes an answer and keeps going — inventing a new user turn, answering itself, running to the token limit. The end-of-turn token in your template is not the one it learned, so nothing it emits terminates the turn. This is the most common single symptom.
• Special tokens appear as text. <|im_end|> or [/INST] showing up in the visible output means the model is producing structure as literal text because the structure it was given was not in the position it expects.
• The system prompt is ignored. If the template renders the system message somewhere the model was not trained to look for instructions — or drops it because a {{ if .System }} guard never fires — the model reads it as ordinary conversation, or not at all.
• Quality collapses on the second turn. A template that is right for a single exchange and wrong for history looks fine until there is history. This is the one that survives testing.
• Tool calls come back as prose. A model trained to emit a structured call renders one only when the tool definitions arrive in the layout it was trained on. A template without a .Tools branch produces a plausible English description of the call instead.

Reading the template a model already has

Almost every template problem is solved by not writing one. Models pulled from the registry arrive with the template their publisher supplied, and the correct move when deriving a custom model is to inherit it rather than re-type it. To see what you have:

# the whole Modelfile the image was built from
ollama show --modelfile llama3.2:3b

# just the template, which is usually what you want to compare
ollama show --template llama3.2:3b

Compare the result against the chat template on the model’s Hugging Face card — that repository’s tokenizer_config.json carries a chat_template field in Jinja, which is the same information in a different template language, and it is the authoritative statement of what the model was trained against. If the two disagree in where the system message lands or which token closes a turn, you have found your bug.

The same check is the first thing to do after importing raw weights, because that path has no publisher to inherit from — see importing a GGUF into Ollama. A GGUF may carry a template in its metadata or may not, and when it does not, Ollama has to guess.

Changing one without breaking it

A Modelfile that names a base image with FROM and does not declare TEMPLATE inherits the base’s template unchanged. That is the safe way to add a system prompt or change sampling parameters:

FROM llama3.2:3b
SYSTEM """You answer in at most three sentences."""
PARAMETER temperature 0.4
PARAMETER num_ctx 8192

Note what is happening there. SYSTEM supplies the value that the existing template will render into its {{ .System }} slot; it does not change the structure. If you declare a TEMPLATE as well, you have replaced the whole rendering with yours and inherited none of the base’s handling of history or tools. The rest of the instructions available in that file, and the build step that turns it into a runnable image, are covered in creating a custom model and Modelfile syntax.

There are real reasons to write one anyway — a fine-tune with a bespoke format, a base model with no template at all, a completion model you want to drive through the chat endpoint. When you do, change one thing at a time and test against a two-turn conversation rather than a single question, because a template broken only in the history branch passes every one-shot test you will think to run.