Why the Same Prompt Behaves Differently on Every Model

A prompt is not a specification, it is a configuration fitted to one model’s post-training. Moving it is a port, and ports have a predictable list of things that break.

Why a prompt is model-specific

Four things differ underneath, and everything on this page follows from them. The chat template differs, so your text sits inside different marker tokens. The tokeniser differs, so the same prompt is a different number of tokens and different strings are single tokens. The post-training differs, so verbosity, refusal boundaries and format-following are tuned to different targets. And the API surface differs, so structured output, tool calling and stop sequences are not the same features with the same names.

Nine transfer failures

1 · Fenced JSON

The new model wraps its JSON in a markdown fence. Your parser sees a backtick first and you get json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0). Strip fences before parsing, always, on every model — this one costs nothing to defend against and is the single most common port failure.

2 · Verbosity drift and truncation

The same instruction produces three times the output tokens, the response hits your max_tokens, and you get a syntactically invalid JSON object with finish_reason: "length" — or stop_reason: "max_tokens", depending on the API. The bug report says “invalid JSON”. The cause is length. Always branch on the finish reason before parsing.

3 · The system role is not a role

Some model families have no distinct system turn and the runtime concatenates it into the first user message. Everything you assumed about system instructions outranking user text quietly stops holding, which shows up as an increase in successful prompt injections rather than as an error.

4 · Few-shot format sensitivity

Examples formatted in one family’s convention — XML tags, markdown headings, bare colons — are being read by a model tuned on a different one. The symptom is subtle: the model imitates the formatting of your examples inconsistently while getting the task right, which breaks parsers rather than accuracy.

5 · Structured output is not available

A prompt that leaned on a JSON-schema mode has nothing to lean on. The request either errors on an unsupported parameter or, worse, the parameter is silently ignored and you discover it in the parse rate. Check that the schema was actually enforced rather than accepted.

6 · Tool-call shape

Parallel tool calls, nested schema features, and strict-mode argument validation vary. A loop written against an array of three calls per turn breaks on a model that emits one at a time — not with an exception, but with an agent that takes three times as many steps and hits your step budget.

7 · Reasoning behaviour you did not ask for

Port a latency-sensitive prompt onto a reasoning model and the instruction to answer immediately is ignored, because thinking happens before the instruction is even reachable. Time to first token and cost both jump. The fix is a configuration — an effort or budget setting — not a wording change.

8 · Stop sequences and their limits

Providers cap how many stop sequences you may send and differ on whether the stop string is included in the returned text. A transcript-style prompt that relies on stopping at \nObservation: can quietly run on.

9 · Token budget

Different tokeniser, different count: the same prompt can be noticeably longer or shorter in tokens, and code-heavy or non-English text moves the most. Combined with a smaller window this produces "code": "context_length_exceeded" on inputs that were comfortable before.

The canary set

Before you evaluate quality on a new model, evaluate the contract. Twenty cases, chosen to exercise the mechanics rather than the intelligence, run first:

• Two normal cases, to confirm the happy path parses.
• Two where the correct answer is the not-found literal.
• Two with the longest realistic input, to check the window and truncation.
• Two containing your delimiter in the user text.
• Two containing an injection attempt.
• Two where a tool must be called, and one where none should be.
• Two with non-English or emoji-heavy input, for tokeniser surprises.
• Two where the model should refuse, to locate the new boundary.

Score the canary set on parse rate and contract compliance only. If a candidate fails here, its benchmark scores are irrelevant to you.

The porting checklist

• Re-count the prompt with the target’s tokeniser; re-check the window.
• Confirm how the system turn is represented, and whether it exists at all.
• Confirm structured output support, and verify it was enforced rather than ignored.
• Re-check tool-calling: parallel calls, strict arguments, result message shape.
• Reset sampling parameters explicitly rather than inheriting client defaults.
• Re-measure output length; adjust max_tokens, then branch on finish reason.
• Re-run the injection cases. Refusal and instruction hierarchy do not transfer.
• Translate the delimiter convention if the target documents a different one.
• Only then compare quality, on the same eval set, at the same temperature.

Writing portable prompts up front

Some choices cost nothing today and save the port later. Put the output contract in the final user turn as well as the system prompt, so it survives a model that flattens roles. Validate and retry rather than trusting any one model’s formatting discipline. Prefer delimiters that are unremarkable text in every family. Avoid instructions that depend on a specific model’s verbosity — “three sentences” ports, “be brief” does not. And keep the prompt’s dependence on reasoning explicit, so switching model classes is a config change rather than an archaeology project.