Skip to content
Multigrid

Llama model behaviour

Llama as a weights family rather than an endpoint: tokenizer, chat template, licence terms, context windows and what changes between versions.

Every other model family in this library is a service. You send JSON to a URL, and the shape of the reply is somebody’s contract with you. Llama is not that. Llama is a set of files — weights, a tokenizer, a config, a licence — and the thing that answers your request is whatever runtime you or your provider pointed at those files. That distinction is not pedantry. It is the reason two hosts of “the same” Llama model can disagree about where a message ends, how long an answer may run, and whether a system prompt is honoured at all.

So these pages are written against the artefacts rather than against an API: the special tokens in the tokenizer, the reference prompt format Meta publishes, the fields in config.json, the clauses in the community licence. Where a runtime commonly departs from the reference, that is said explicitly, because the departure is usually what sent you looking. Where a figure is Meta’s, it is attributed and dated; where it is arithmetic, the assumptions are on the page.

Llama 3’s Context Window: From 8K to 128K Across Releases

The documented context length of each Llama 3.x release, where it changed, and why the number your runtime reports may be smaller.

8 min read

Llama 3’s Chat Template and Special Tokens

The reference prompt format Meta publishes, annotated token by token against a fully rendered two-turn conversation.

9 min read

Llama 3’s Tokenizer Vocabulary Size, and What Changed From Llama 2

The documented vocabulary sizes for both generations, what a four-fold vocabulary costs in parameters, and how to reproduce the token-count difference yourself.

8 min read

Llama Guard: Meta’s Separate Moderation Model, Not a Built-In Refusal Layer

Why a downloaded Llama has no moderation endpoint, what Llama Guard is instead, and where it sits in a request.

8 min read

Function Calling With Llama 3: The JSON Convention Meta Documents

How tool calls work on a model with no tool-call API field: the documented prompt convention, the tokens that terminate a call, and what you have to parse.

9 min read

Llama 3’s System Prompt: How the Reference Template Structures It

The system role is a header block inside the prompt string, not an API field — what that changes about position, precedence and the lines the template inserts for you.

8 min read

Does Meta “Deprecate” Llama Models? What That Means for Open Weights

There is no shutdown switch in a file you already have, so deprecation for Llama means something else entirely — and that something else can still break you.

9 min read

Llama 3.2’s Vision Models: Which Sizes Support Image Input

Two of the four Llama 3.2 sizes take images and two do not, and the split is architectural rather than a matter of scale.

8 min read

Llama 3’s Two End Tokens: eot_id and end_of_text

Why Llama 3 has two stop tokens, which model emits which, and the exact generation config that stops on both.

7 min read

Determinism in Self-Hosted Llama: Why the Same Weights Can Still Drift

Temperature 0 removes the sampler as a source of variation and leaves several others, all of them in the arithmetic underneath.

9 min read

Llama’s Output Length: No Hard API Cap, Just max_new_tokens

Open-weight Llama has no provider-imposed output ceiling; the limits are the parameter you set, the context window, and where the model chooses to stop.

8 min read

RoPE Scaling in Llama 3.1: How the Context Window Was Extended Past 8K

The rope_scaling block in Llama 3.1's config, what its four numbers mean, and why factor 8 does not produce a 65K context.

9 min read

Llama 3’s License: What the Terms Actually Permit

The specific obligations in Meta's community licence — attribution, naming, the notice file and the 700-million-user threshold — and what changed at 3.1.

9 min read

Llama 3.1 405B's Context Window: The Documented Number and the Real One

Meta documents 128K tokens for Llama 3.1 405B; the config file says 131,072, and the memory arithmetic decides what you actually get.

8 min read

The Refusal Behaviour Baked Into Llama 3's Instruction Tuning

Why an Instruct checkpoint refuses things the base weights do not, and how that differs from a moderation filter sitting in front of a hosted API.

9 min read

Llama 3.1's Built-In Tools: Brave Search, Wolfram and Code Interpreter

The three tool names Llama 3.1 was tuned to emit, the prompt lines that enable them, and why none of them does anything until you implement it.

9 min read

Why Llama 3's Token Counts Differ From Llama 2's on Identical Text

The same paragraph tokenizes to fewer tokens on Llama 3 than on Llama 2, and the two vocabulary designs explain exactly how much and where.

8 min read

Llama 3's Knowledge Cutoff Date, Model by Model

The cutoff Meta documents for each Llama 3.x and Llama 4 release, why 3 8B and 3 70B differ, and why the prompt template's date line is not evidence.

8 min read

Pinning a Llama Checkpoint by Commit Hash Instead of a Moving Tag

How to load a Hugging Face repository at an exact revision, why main is mutable, and what changes underneath you when it moves.

9 min read

What Happens When You Exceed a Self-Hosted Llama's Context Length

The error vLLM raises, the warning llama.cpp prints, and the silent truncation Ollama performs — three different behaviours from one overflowing prompt.

9 min read

Multi-Turn Tool Results in Llama 3's Chat Template

The ipython role Meta defines for feeding tool output back into the prompt, shown as a complete raw exchange from call to final answer.

9 min read

The Streaming Response Shape a Self-Hosted Llama Server Returns

vLLM, llama.cpp, Ollama and TGI stream the same tokens in four different envelopes; here is each one and what the last event carries.

9 min read

Llama 3.2's 1B and 3B Models: Context Window at the Small End

Meta documents the same 128K window for the 1B and 3B models as for the 405B, and on a small model the cache for it outweighs the weights.

8 min read

Why Llama Sometimes Ignores a Stop Sequence Mid-Generation

Stop tokens and stop strings are two different mechanisms implemented in two different places, and most ignored stop sequences are a mismatch between them.

9 min read

Llama 4's Mixture-of-Experts Routing and What It Changes About Output

Meta's published total and active parameter counts for Scout and Maverick, and what 17B active actually means for a single forward pass.

9 min read

Llama's Context Window Counted in Tokens: Fitting a Document

How to turn a word count into a token budget for a Llama context window, with every term in the arithmetic named and the ratio measured rather than assumed.

8 min read

Other topics