Migrating an Internal Style Guide for Prompts Between Model Families

An internal prompt style guide is a set of conventions somebody arrived at by trial and error against one model family. When the model changes, some of those conventions are still right, some are now actively harmful, and the guide contains no information about which is which. This is how to tell them apart.

Four things change, and only one is the words

“Prompts do not transfer between models” is true and useless. It describes an outcome without naming a cause, so it cannot tell you which of your conventions to keep. The useful version is that a prompt string passes through four independent layers on its way to becoming behaviour, and a migration perturbs all four at once:

• Tokenisation. The model never sees your string. It sees a sequence of token ids produced by a vocabulary that differs between families. Your delimiters, your indentation and your bullet markers all have different token boundaries on the other side.
• The envelope. Your string is wrapped in a chat template and placed in a role. What counts as a role, and how much structural privilege a role carries, is an API design decision that differs between providers.
• Instruction-following priors. Post-training decides how literally the model reads an instruction, how it resolves a conflict between two of them, and how much it infers that you did not say. This is the layer that produces the most surprising regressions, because a model that follows instructions better can produce worse output from a prompt that was tuned to overcome a weaker one.
• Formatting defaults. Absent instruction, the model emits its house style: markdown density, prose versus bullets, maths notation, whether it narrates before calling a tool. You did not ask for any of it and it is not in your prompt.

Every convention in your style guide is a rule that lives on one of those four layers. Sorting them is most of the migration work, and it is work you only have to do once if you record the reason for each rule as you go.

Delimiters are tokens, and tokenisers differ

Most house guides mandate a delimiter convention: XML-ish tags, ### Section markers, triple backticks, or a run of dashes. The convention exists because it worked, and it worked for a reason more specific than “structure helps”. A delimiter is doing two jobs at once. It marks a boundary the model can attend to, and it is itself a sequence of tokens whose cost and salience depend entirely on the vocabulary.

A byte-pair vocabulary merges frequent sequences into single tokens. Whether <instructions> is three tokens or seven is a property of the training corpus that produced the merges, not of your prompt. Swap families and a delimiter that was compact and distinctive can become several common fragments, or the reverse. The practical consequences are that per-call delimiter overhead moves — a twelve-section prompt sent a million times a month is a real line item — and that a convention chosen for visual clarity may have been carrying more attentional weight on the old model than you realised. The library covers the counting mechanism in tokeniser comparison; the migration-specific point is that a delimiter convention is a model-dependent rule and belongs in the part of the guide that expires.

There is a documented preference to work from rather than folklore. Anthropic’s prompting guidance recommends XML tags for prompts that mix instructions, context, examples and variable input, with consistent descriptive tag names and nesting where the content has a natural hierarchy (Anthropic, prompting best practices). That is a recommendation about one family. The correct house rule is not “use XML tags” but “use the delimiter convention the target family’s own documentation recommends, and re-read that documentation on every migration”.

The system prompt is not the same object

Style guides almost always contain a rule about where instructions go: system prompt versus first user turn, top of the prompt versus bottom, before or after the retrieved context. That rule is a claim about the envelope, and the envelope is not portable.

The two dominant shapes differ structurally. Anthropic’s Messages API takes system as a top-level request parameter — it is not a message and has no entry in the messages array. The OpenAI-shaped chat APIs put it in the array as a message with a role. Move a prompt from the first shape to the second and a string that had a distinct structural position becomes an ordinary element of a list that the model reads in sequence; move it the other way and a message that competed with its neighbours acquires a privileged slot. The library has a page on the developer message role and its precedence, which is the detail this rule turns on.

What does not survive translation is the part of your guide that says “put hard constraints in the system prompt so the user cannot override them”. The strength of that guarantee is a post-training property, not an API property, and it is different on the other side. Keep the rule, because it is still the best available place, and stop treating it as a security boundary — that is what injection defences are for, and they need their own re-baselining after a swap.

House style you never asked for

The fourth layer is the one that breaks downstream code. Every model has a default output register, and it appears in every response where your prompt did not explicitly forbid it. Anthropic documents several of its own: current models default to LaTeX for mathematical expressions unless told otherwise, they may skip the verbal summary after a tool call that earlier models produced, and the formatting style of your prompt itself influences the formatting of the response — removing markdown from a prompt reduces markdown in the output.

Read that last one carefully, because it inverts a common style rule. A guide that mandates heavy markdown structure in prompts for the benefit of the humans maintaining them is, on such a model, also instructing the output to be heavily marked up. If a downstream step parses that output, your readability convention is a production dependency.

The same documentation gives the correct form for a formatting rule: state what you want rather than what you do not. “Do not use markdown” is weaker than “write in flowing prose paragraphs”, and an explicit format indicator is stronger still. A house guide should carry the positive form, because the negative form is precisely the sort of rule whose effectiveness varies most between families.

Rewriting the guide, with expiry dates

The rewrite is mechanical once the sort is done. Go through the guide rule by rule and attach two fields to each: which of the four layers it acts on, and whether it exists for the model or for the humans. A rule about naming variables in templates is for the humans and never expires. A rule that says “always restate the constraint at the end of the prompt because the model loses it” is a compensating rule for a weakness in one model, and it is exactly the kind that turns harmful.

Anthropic states this outright for its own migrations: if your prompts previously encouraged the model to be more thorough or to use tools more aggressively, dial that guidance back, because current models are more proactive and may overtrigger on instructions that earlier models needed. That is a documented case of a prompt getting worse results specifically because the new model follows it more faithfully. Every compensating rule in your guide is a candidate.

# Prompt style guide — rule format

## R14. Restate the output contract in the final 200 tokens
Layer:    instruction-following priors
Audience: model
Origin:   compensating for context-position loss on <old family>
Review:   on every model-family change — REMOVE unless re-justified
Evidence: <link to the eval case that failed without it>

## R15. Template variables use SCREAMING_SNAKE and appear in a manifest
Layer:    none (tooling)
Audience: humans
Review:   never expires

The Origin and Evidence fields are the whole point. A rule with a named eval case behind it can be re-run against the new model and kept or dropped on evidence. A rule with no evidence is folklore, and the honest move at a migration is to delete it and see whether anything breaks. The review checklist is where those re-runs get scheduled, and linting rules are where the surviving ones get enforced.