Llama 3’s System Prompt: How the Reference Template Structures It

On a hosted API the system prompt is a parameter, handled by the provider, positioned by the provider, and given whatever precedence the provider trained in. On Llama it is a header block in a string. Almost everything that surprises people about Llama system prompts follows from that one structural fact.

The block itself

A system message uses the same envelope as every other message, with system as the role name:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a support agent for an airline. Never discuss competitors.
Answer in at most three sentences.<|eot_id|><|start_header_id|>user<|end_header_id|>

Can I change my flight?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

The rules are positional rather than syntactic. The system block, if present, comes first — immediately after <|begin_of_text|> and before any user turn. There is at most one. It is optional; a valid Llama 3 prompt can begin with a user header and the model handles that fine. And, as everywhere in this format, the two newlines after <|end_header_id|> and the absence of any separator before <|start_header_id|> are part of the format, not styling. The full token inventory is in the chat template and special tokens page.

What changed from Llama 2

Llama 2 wrapped the system prompt in tags nested inside the first instruction:

<s>[INST] <<SYS>>
You are a helpful assistant.
<</SYS>>

Can I change my flight? [/INST]

The system prompt was part of the first user turn, which meant that multi-turn rendering had a special case — the tags appear in turn one and nowhere else — and that a great deal of code got it wrong on turn two. Llama 3’s design removes the special case entirely: the system message is a message like any other, with a role, rendered by the same loop. That is the whole of the improvement, and it is why Llama 3 templates are so much shorter than Llama 2 ones.

A second, less obvious change: Llama 2 chat models shipped with a long default system prompt baked into Meta’s reference implementation, which shaped their famously cautious tone. Llama 3 ships no default. If you send no system message, there is no system message — the model is running on its instruct tuning alone.

The two lines you did not write

Render a Llama 3.1 or 3.2 conversation through the tokenizer’s own chat template and you will find a system block you did not ask for:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 11 Aug 2026

You are a support agent for an airline.<|eot_id|>

The chat template in tokenizer_config.json inserts those two lines, and it inserts them even when you supplied no system message at all — in which case the system block exists solely to carry them. The date is rendered from a template variable that defaults to the current date, which means the exact same messages array produces a different prompt tomorrow.

That is worth knowing for three reasons. It quietly costs you tokens on every request. It makes prompt caching miss across a date boundary, because the cached prefix changes at midnight. And it means any test that asserts on an exact rendered prompt string will pass today and fail tomorrow. If you need a stable rendering, pass an explicit date_string to apply_chat_template.

The knowledge-cutoff line is also a claim about the model rather than an instruction to it — the model does not consult it. What it actually knows is a separate question, covered in Llama 3’s knowledge cutoff.

It has no special authority

This is the part that matters for anything user-facing. The system block is tokens in the same sequence as the user’s message. There is no privileged channel, no cryptographic separation, and no mechanism in the architecture that makes a system instruction outrank a conflicting user one. Instruct tuning gives the system role a learned tendency to be followed, and that tendency is real and useful and statistical. It is not enforcement.

Which means a user message that says “ignore your previous instructions” is competing on the same terms as your system prompt, and sometimes it wins. Every model has this property; the reason to say it plainly here is that Llama makes the mechanism visible. You can print the string and see that your rules and the attacker’s text are adjacent characters in one buffer.

The defence is architectural rather than textual. Do not put a secret in the system prompt — everything in the context is potentially recoverable. Do not rely on the system prompt to enforce a policy that matters; enforce it in code around the model, or with a separate classifier such as Llama Guard. And treat any instruction that arrives inside retrieved documents or tool output as hostile by default, because it lands in the same buffer too.

Position beats emphasis

If the system prompt is just tokens near the front of a sequence, the question “why is my instruction being ignored?” has a structural answer as well as a behavioural one. An instruction at the start of a 40,000-token context is competing for attention against everything that arrived after it, and the usual response — restating it more emphatically, in capitals, with the word IMPORTANT — is addressing the wrong variable.

The lever that actually moves is where the instruction sits relative to the thing it governs. A formatting rule stated immediately before the content to be formatted is closer, in the sequence, to the tokens it has to influence than the same rule at the top of a long conversation. This is why a short repeat of the critical constraint in the final user turn is so much more effective than lengthening the system prompt, and it costs a handful of tokens rather than a rewrite.

A related consequence for retrieval-augmented setups: putting retrieved documents between the system prompt and the user question means the instruction and the question are now separated by everything you retrieved. Putting the documents before the system block is worse again. The arrangement that tends to hold up is system block, then documents, then the question with its constraints restated — which keeps the model’s most recent tokens as the ones you most want obeyed.

None of this is Llama-specific, but Llama is where you can inspect it. Print the rendered prompt and read it in order, and it is usually obvious which of your instructions is buried.

Consequences in practice

• Long system prompts are a per-request cost. There is no state between calls, so a 2,000-token system prompt is 2,000 input tokens on every turn of a fifty-turn conversation. Prompt caching is the mitigation, and the injected date line partially defeats it.
• Smaller models follow it less well. Llama 3.2 1B and 3B have noticeably weaker instruction adherence than 8B or 70B, and the usual fix is fewer, shorter, more concrete rules rather than a more emphatic version of the same long one.
• Rendering is per-checkpoint. A fine-tune can ship a modified chat_template that handles the system role differently, or drops it. If a system prompt is being ignored entirely rather than merely outranked, render the prompt and look for it before changing its wording.
• Two system messages are not supported. The format has one slot. Merge them yourself; a template given two may render something the model has never seen.
• Base models do not have a system role. Only the instruct checkpoints were tuned on this format. Sending a system block to a base model produces text continuation of the block itself.