Keeping Two Prompt Versions in Sync During a Provider Migration

A migration that runs for six weeks means six weeks of prompt edits landing in two places. The copy nobody is looking at drifts, and the drift is discovered on cutover day. The fix is not discipline; it is removing the second copy.

Why two copies diverge within a week

The usual shape of a provider migration is: copy the prompt file, adjust it until the new provider behaves, and keep both until you cut over. That is fine on day one because the two files are identical except for the adjustments. It stops being fine the moment ordinary product work resumes.

Prompt edits during a migration are not rare events. A support ticket reveals the model is mishandling refunds, so somebody adds a clause. A legal review adds a disclaimer. A tool description gets tightened because the model kept calling the wrong tool. Each of these is a two-line change to the file that is live, and every one of them is silently absent from the file that is not. Six weeks in, the two prompts differ by a dozen edits nobody wrote down, and the “equivalent” A/B you run before cutover is comparing the current prompt against a six-week-old one. The new provider looks worse than it is, and you cannot tell how much of that is the model.

The second failure is worse and quieter: somebody does remember, and applies the edit to both. They apply it correctly to the live one and approximately to the other, because the two files have different structure and the paragraph does not sit in the same place. Now the divergence is invisible to a diff, because both files changed.

What the canonical form holds

The pattern is the one every internationalised application already uses: hold the meaning once, render it per target. The canonical form is not a string. It is a structure that holds the parts of a prompt that are genuinely provider-independent:

• The persona and task instructions — the prose that says what the assistant is and what it is for. This is nearly always portable verbatim.
• Constraints and refusal rules — also prose, also portable, and the part most likely to be edited mid-migration because it is where compliance changes land.
• Few-shot examples as structured turns, not as a block of text. A user turn and an assistant turn, each with a role, so a renderer can put them wherever that provider wants them.
• Tool definitions as name, description and a JSON Schema object. The schema itself is portable; the envelope around it is not, and the envelope is the renderer’s problem.
• Sampling intent expressed as a decision rather than a number: deterministic, balanced, creative. The reason is in the next section.

Everything else — where the system text goes, what the tool envelope looks like, what the output-format field is called, what the token cap is called and whether it is required — belongs to a renderer.

The renderer is where portability ends

A renderer takes the canonical structure and produces a request body for exactly one provider. Writing two of them is a morning’s work, and it is worth doing rather than templating because the differences are structural rather than cosmetic. The ones that bite:

System text is not in the same place

OpenAI’s Chat Completions API carries it as a message in the messages array with a role — historically system, and the library already covers the newer developer role. Anthropic’s Messages API takes it as a top-level system parameter that sits outside messages entirely, and the messages array there accepts only user and assistant. OpenAI’s Responses API takes it as instructions. One canonical field, three destinations.

Tool envelopes differ around an identical schema

Chat Completions nests the definition: an array entry of type: "function" with a function object holding name, description and parameters. Anthropic’s Messages API takes name, description and input_schema flat on the tool object. The JSON Schema you put in parameters or input_schema is the same document. Keep it as one object in the canonical form and let each renderer wrap it.

Sampling ranges are not the same range

This is why the canonical form stores an intent rather than a float. OpenAI documents temperature over 0 to 2; Anthropic documents it over 0 to 1. A canonical 0.7 is a mild setting on one and a middling-to-warm one on the other, and copying the number across is the kind of change that shifts behaviour without appearing to. Map balanced to a number in each renderer, once, with a comment saying which documented range it came from.

The token cap is named differently and is not always optional

Anthropic’s Messages API requires max_tokens on every request. Chat Completions treats its cap as optional with a model-dependent default, and OpenAI’s reasoning models reject max_tokens in favour of max_completion_tokens. The Responses API calls it max_output_tokens. A canonical “answer budget” renders to a different key in each place, and one of them fails the request outright if you omit it.

Applying one fix to both sides

With the canonical form in place, the mid-migration refund clause is one edit to one file. What you still need is proof that the edit survived both renderers, because a renderer bug is now a bug in two products at once. That proof is a snapshot test per provider: render the canonical prompt, serialise the request body, compare against a committed file.

# tests/render_snapshot_test.py
from prompts import CANONICAL
from render import render_openai, render_anthropic

def test_openai_snapshot(snapshot):
    body = render_openai(CANONICAL["support_agent"], version=7)
    assert body == snapshot("support_agent.openai.json")

def test_anthropic_snapshot(snapshot):
    body = render_anthropic(CANONICAL["support_agent"], version=7)
    assert body == snapshot("support_agent.anthropic.json")

The snapshots are not there to assert the prompt is good. They are there so that a one-line prose edit produces a two-file diff in the pull request, both diffs contain the same new sentence, and a reviewer can see at a glance that the change reached both providers. When one snapshot changes and the other does not, the renderer swallowed something.

Version the canonical entry, not the rendered output. A prompt version identifier that travels into request metadata and into your logs is what lets you answer “which prompt produced this bad answer” after cutover, when both providers appear in the same log stream. That is the same identifier your logging field migration needs to keep stable across the move.

Building it

1. Take the prompt that is live today and split it by hand into the five canonical parts above. Do not try to make this general yet — do one prompt.
2. Write the renderer for the provider you are currently on. Assert that its output is byte-identical to the request body you send today. Until that assertion passes you have changed production behaviour while claiming not to.
3. Deploy that renderer alone, with no second provider anywhere. This is the risky step and it should be the only thing in its release.
4. Write the second renderer. Run it against the same canonical entry and read the JSON it produces before you send it anywhere. Most mapping mistakes are visible at this point.
5. Commit both snapshots and add the two-file diff expectation to your review checklist for the duration of the migration.
6. Delete the old prompt file. If it still exists, somebody will edit it. This is not paranoia; the file being editable is the entire failure mode this pattern exists to remove.
7. After cutover, delete the renderer you no longer use — or keep it deliberately, as the thing that makes a rollback a config change rather than a revert.