Prompt Portability: What Breaks When You Move a Prompt to Another Model Family

You changed the model and the prompt that worked for a year started producing preambles, ignoring the output format, or refusing things it used to do. The prompt is not broken. It was carrying instructions in a form the old family was post-trained to read, and some of those forms do not mean anything to the new one.

The symptom, precisely

Before changing anything, pin down which of four distinct failures you have, because they have different causes and fixing the wrong one wastes a day:

• Format drift. The content is right, the wrapper is wrong — a “Here is the JSON you asked for:” before the object, a code fence around it, a closing pleasantry.
• Instruction drop. One instruction out of eight is being ignored, consistently, usually one buried mid-prompt.
• Verbosity change. Same content, two or three times the length, which shows up on the bill before it shows up in a bug report.
• Refusal change. Requests that were answered are now declined, or the reverse. This is a post-training difference and no amount of prompt reformatting fixes it; it needs a rewrite of the framing or a different model.

First: the parameters silently changed meaning

Check this before touching the prompt text, because it is the cheapest thing to get wrong and the most likely to be the whole problem.

Temperature is not on the same scale. OpenAI’s Chat Completions API documents temperature as accepting values from 0 to 2. Anthropic’s Messages API documents a range of 0 to 1. A pipeline that ported temperature: 1.0 across did not move a neutral setting; it moved from the middle of one range to the top of the other. If your symptom is “the output got noticeably more erratic”, look here first. Sending a value above the accepted range is a request-validation error rather than a quality problem, which at least fails loudly.

The output cap has different names and different defaults. Anthropic’s Messages API requires max_tokens on every request — it is not optional. OpenAI’s Chat Completions hasmax_tokens as an optional field, and its Responses API and reasoning models use max_completion_tokens instead, which also counts reasoning tokens toward the cap. A port that carries a small value into a reasoning model can consume the whole budget on internal reasoning and return an empty or truncated message with no visible error other than the finish reason.

The stop-sequence field is named differently and behaves the same. OpenAI calls it stop; Anthropic calls it stop_sequences. Both take an array of strings. The subtlety is on the way back out: OpenAI reports finish_reason with the value stop whether the model ended naturally or hit one of your sequences, while Anthropic distinguishes end_turn from stop_sequence in its stop_reason field. Code that branched on the more specific value has nothing to branch on after the port.

Second: the structural conventions

The next tier is about how the prompt marks its own sections. This is where “the prompt was tuned to one family” is literally true: post-training teaches a model which markup carries authority, and different labs made different choices.

Delimiters. Anthropic’s own prompt-engineering documentation recommends XML-style tags — wrapping a document in <document>, instructions in <instructions> — and says Claude models were trained to attend to them. Prompts written for other families more commonly use Markdown headings, all-caps labels, or triple-hash separators. Neither convention is wrong, but a prompt whose section boundaries the model does not treat as boundaries reads as one undifferentiated blob, and the first thing that degrades is the instruction furthest from the end.

Where the system instruction lives. On OpenAI it is a message inside messages with role system (and newer models also accept a developer role, which supersedes it). On Anthropic it is a top-level system parameter and is not a message at all; the messages array is user and assistant turns only. A naive port that pushes a system message into Anthropic’s array either errors or, worse, gets accepted as a user turn where it carries much less weight. This is the most common cause of “it ignores half my rules” after a port.

Instruction position. Long prompts do not weight all positions equally, and the profile differs by family. The practical rule that survives across all of them: put the instruction that must be obeyed immediately before the generation point — last, after the document, not first. Restating the output format in a final short sentence costs a handful of tokens and fixes format drift more often than any other single edit.

Third: few-shot examples are turns, not text

A prompt with worked examples pasted into the system text as Input: ... Output: ... pairs is a prompt whose examples are competing with the real request for the model’s attention. Both APIs support the alternative: encode the examples as actual prior turns in the conversation, alternating user and assistant, before the real user turn. The model then sees the pattern in the same channel it is being asked to continue.

This matters more after a port because families differ in how strongly they imitate in-context formatting from prose versus from turn structure. If your examples are prose and the new model is not copying them, converting them to turns is a mechanical change with no content risk. See the general treatment of few-shot construction for how many to include.

One technique does not port at all and is worth knowing about because people build on it. Anthropic’s Messages API permits the final message in the array to have role assistant, in which case the model continues from that text — prefilling { to force a JSON object, for example. OpenAI’s Chat Completions does not have this; the equivalent guarantee comes from response_format with a JSON schema instead. A prompt that leaned on prefill for format compliance loses that guarantee entirely on the way across and needs the schema-based mechanism substituted, not a reworded instruction.

Working through it in order

1. Fix the parameters: temperature range, the output-cap field name and value, stop-sequence field name. Re-test. This alone resolves a surprising share of ports.
2. Move the system instruction to wherever the new API says it belongs, and confirm it is not sitting in the message array as a user turn.
3. Convert delimiters to the convention the new family’s own documentation recommends. Change nothing else in the same edit.
4. Move the must-obey instruction to the end of the prompt, immediately before generation.
5. Convert prose few-shot examples into conversation turns.
6. Only now rewrite wording. If you rewrite first you will attribute the fix to the wrong change and carry a superstition into the next port.

Do all of this against a fixed set of inputs with recorded expected properties, not against whatever you happen to type. A port is exactly the situation a portability test suite exists for: without one you are comparing the new model to your memory of the old one, and memory is generous about the incumbent.