Gemma Has No System Role: Fixing the Template Error

You moved a working message list to Gemma and got TemplateError: System role not supported. The template is right and your message list is fine; Gemma’s trained format simply has no third role, and the fix is to put the instruction somewhere the model was trained to read.

The error

The exception comes out of Jinja while rendering the chat template bundled with the checkpoint, and the wording varies slightly by transformers version:

jinja2.exceptions.TemplateError: System role not supported

It is raised by this line, which lives inside the chat_template string in tokenizer_config.json for Gemma and Gemma 2 checkpoints:

{% if role == 'system' %}
  {{ raise_exception('System role not supported') }}
{% endif %}

Reproducing it takes three lines, and it is worth reproducing so you can tell this failure apart from a transformers version problem:

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("google/gemma-2-9b-it")
tok.apply_chat_template(
    [{"role": "system", "content": "You are terse."},
     {"role": "user", "content": "Hello"}],
    tokenize=False,
)   # TemplateError: System role not supported

Why the template refuses

Gemma’s instruction tuning used only <start_of_turn>user and <start_of_turn>model. There is no <start_of_turn>system in the training distribution, so there is nothing sensible for the template to render a system turn as. The full format, and what each of those markers does, is set out on the chat-template page.

Google could have made the template drop the message or invent a delimiter. Raising instead is the better choice, and understanding why is the difference between working around this once and working around it correctly. A silently dropped system prompt is the worst possible outcome: your safety instruction, your persona and your output-format rules vanish, and nothing in the response says so. You would only notice through quality that is inexplicably worse than on the model you ported from. An exception, however annoying, tells you on the first run.

This also explains why the workaround is legitimate rather than a hack. Gemma has no separate mechanism for instructions; instruction text and user text were the same channel during tuning. Putting your system prompt in the first user turn is not degrading it to a lower tier, because on Gemma there is no higher tier.

It is worth being clear about what you lose, because the answer is less than people assume and more than nothing. You do not lose the ability to instruct the model: instruction-following on Gemma is trained on user-turn text, so instructions in a user turn are exactly where the model expects them. What you lose is the separation. On a family with a real system channel, later user text cannot overwrite the system instruction by claiming to be one, because the two arrive in structurally different places. Fold them together and that distinction is gone, so a user who writes “ignore the previous instructions” is writing into the same channel your rules occupy. Treat prompt-injection resistance on Gemma as something you build outside the model rather than something the template provides.

The fold

Prepend the system content to the first user message, separated by a blank line, and drop the system message:

def fold_system(messages):
    """Gemma has no system role: merge it into the first user turn."""
    out, pending = [], None
    for m in messages:
        if m["role"] == "system":
            pending = (pending + "\n\n" + m["content"]) if pending else m["content"]
            continue
        if pending and m["role"] == "user":
            m = {**m, "content": pending + "\n\n" + m["content"]}
            pending = None
        out.append(m)
    if pending:                       # system-only conversation
        out.insert(0, {"role": "user", "content": pending})
    return out

Three details in that function are there for a reason. It accumulates multiple system messages rather than keeping the last, because some frameworks emit one per middleware. It copies the message instead of mutating it, because the caller may reuse the list. And it handles the case where a system message arrives with no user turn after it, which otherwise produces an empty prompt.

Render it and check the result before trusting it:

text = tok.apply_chat_template(
    fold_system(messages), tokenize=False, add_generation_prompt=True
)
print(repr(text))

The blank-line separator is doing real work and is not arbitrary. Run the instruction straight into the question with a single newline and the two read as one continuous request, which makes the model more likely to answer the instruction rather than obey it. A blank line is the weakest available signal that these are two distinct pieces of text, and it is the one the model has seen most often in instruction-formatted training data.

Two variants are worth knowing. If the instruction is long and the question is short, some people prefer to label the sections explicitly — a line reading “Instructions:” before the system text and “Question:” before the user text — which costs a handful of tokens and makes the boundary unambiguous. And if the conversation may begin with a model turn, for instance a scripted greeting, fold the system text before that greeting rather than after it, so the instruction still precedes everything the model produced.

Making it survive multi-turn

The fold puts the instruction at the very start of the conversation. On a long exchange, that is thousands of tokens back, and Gemma 2’s alternating local attention means only half its layers can reach that far directly. Persona and format instructions do drift as a conversation extends.

• Re-fold on every request rather than once. Keep the system prompt outside the stored history and apply the fold at send time, so you can change the instruction without rewriting the transcript.
• For strict output formats, restate at the end. Appending the format requirement to the current user turn puts it inside every layer’s reach. This costs tokens on every call and is worth it when the format is load-bearing.
• Do not prefill the model turn to fake a system voice. Writing an opening into <start_of_turn>model to simulate an instruction produces a transcript that does not match the trained format and tends to make the model continue that fake turn rather than obey it.

Gemma 3 accepts the role, with a caveat

Gemma 3’s bundled template does not raise. It accepts a message with role: system and renders it by prepending the content to the first user turn: the same fold, moved inside the template where it belongs. So the error disappears on Gemma 3, and the underlying model behaviour does not change, because the text the model sees is still a user turn.

Keep the distinction in mind when reasoning about behaviour. On Gemma 3 a system message is accepted; it is not privileged. Anything you rely on that assumes a system instruction outranks later user text is an assumption about a different family.

That has a direct consequence for portable code. If you keep your own fold and also pass a system message to a Gemma 3 template, the instruction is folded twice and appears twice in the prompt. Harmless, but it wastes tokens and makes rendered prompts confusing to read in a log. Decide once where the fold happens — in your code for every Gemma generation, or in the template for the ones that support it — and make the other path assert rather than duplicate. Google documents the template alongside the checkpoints, for instance on the gemma-3-27b-it model card.