Prompt Registries and Deploying a Prompt Change

A prompt is configuration that behaves like code: it changes behaviour, it needs review, it needs versions and it needs rollback. It is also edited by people who do not deploy, at a cadence code does not have. A registry is what lets both of those be true at once.

Why prompts want a different release cycle

Prompts in the source tree are simple and they work — right up to the point where fixing a two-word instruction means a pull request, a CI run and a deploy, and where the person who noticed the problem cannot do any of those. Then prompts start living in a spreadsheet that someone pastes into the code, and the version history is gone.

The opposite failure is worse. Prompts in a database, edited live, no versioning, no review — a text box in an admin panel that can take down a production feature with a typo and leave no record of what it said before. A registry is the middle: prompts are versioned, reviewed artefacts that ship independently of the binary.

It is worth being clear about what a registry is not, because the category is oversold. It does not make prompts safe to change; only evals do that. It does not replace review. And it adds a dependency to your inference path that has to be designed not to matter, which is most of the work below. Adopt one when at least two of these are true: more than a handful of prompts, more than one person editing them, or a rollback that currently requires a deploy. Below that bar, a constant in the source tree with a hash written to the request log is genuinely enough, and it is the version of this that you should start from.

The data model

Two tables. The first is append-only and immutable; the second is a tiny mutable pointer. That split is the entire design.

-- Immutable. A row is never updated, only superseded.
create table prompt_version (
  prompt_id     text        not null,          -- 'pricing_assistant.system'
  version       text        not null,          -- content hash, e.g. 'v_9f2a1c'
  template      text        not null,          -- with {{placeholders}}
  variables     jsonb       not null,          -- JSON Schema for the inputs
  model_hint    text,                          -- what it was authored against
  params        jsonb       not null,          -- temperature, max_tokens, ...
  notes         text,
  author        text        not null,
  created_at    timestamptz not null default now(),
  primary key (prompt_id, version)
);

-- Mutable, tiny, audited. One row per prompt per environment.
create table prompt_pointer (
  prompt_id      text        not null,
  environment    text        not null,         -- prod | staging | canary
  version        text        not null,
  rollout_bps    integer     not null default 10000,  -- basis points, 0..10000
  previous       text,                         -- for one-keystroke rollback
  updated_by     text        not null,
  updated_at     timestamptz not null default now(),
  primary key (prompt_id, environment),
  foreign key (prompt_id, version) references prompt_version (prompt_id, version)
);

Making version a content hash rather than an incrementing number is worth the small awkwardness. It means identical content cannot get two versions, a version cannot be edited in place without becoming a different version, and the value in your request log is a claim you can verify rather than trust.

The variables schema is the second thing people skip and regret. A prompt is a function with named inputs; declaring them means a template that references {{customer_tier}} after somebody renamed the field fails at publish time rather than producing a prompt with the literal placeholder in it.

Two columns carry more weight than their size suggests. params keeps temperature and the token ceiling on the version rather than in the calling code, because a prompt authored against a 2,000-token ceiling behaves differently under 500 and a rollback that restores the text but not the settings has not restored anything. model_hint records what the prompt was written and evaluated against, which is what lets you answer the question that comes up during every model migration: which prompts have never been evaluated on the model they are now running against?

The rollout_bps column on the pointer is what makes a prompt change a gradual release rather than a switch. Basis points rather than a percentage because 0.5% is a useful first stage for a high-traffic feature and integers are easier to reason about than floats in a configuration table.

Resolving a prompt at call time

The registry must not become a synchronous dependency of your inference path. If the prompt service is down, your feature should not be. Three layers, in order.

import { compiled } from "./prompts.generated";  // baked in at build time

const cache = new Map();          // version -> compiled template
let pointers = compiled.pointers; // refreshed in the background, never inline

// A background task polls the registry every 30s and swaps this map.
// It never throws into the request path; on failure it keeps the old map.
export function resolvePrompt(promptId, ctx) {
  const p = pointers[promptId] ?? compiled.pointers[promptId];

  // Sticky assignment: one tenant sees one prompt version, consistently.
  const bucket = hash32(promptId + ":" + ctx.tenantId) % 10000;
  const version = bucket < p.rolloutBps ? p.version : (p.previous ?? p.version);

  const tpl = cache.get(version) ?? compiled.versions[version];
  if (!tpl) {
    // Unknown version: fall back to what shipped with the binary rather
    // than fetching synchronously. Log it loudly; it means a bad publish.
    metrics.promptFallback.add(1, { prompt_id: promptId });
    return compiled.versions[compiled.pointers[promptId].version];
  }
  return { version, text: tpl.render(ctx.vars), params: tpl.params };
}

Three properties fall out of that. The registry can be entirely unavailable and requests still succeed with the last known good prompt. The assignment is sticky per tenant, so a partial rollout does not flip a single user between two prompts mid-conversation. And the resolved version is returned, so it can be written onto the request row and the span — which is what makes a quality comparison by prompt version possible at all.

Releasing a change

The point of the registry is that publishing is a pointer update, so the release process is a sequence of pointer updates rather than a sequence of deploys:

• Author against a version. A new row in prompt_version. Nothing is live yet.
• Evaluate offline. Run the eval set for that prompt id against the new version and the current one. This is the gate that a text box in an admin panel does not have.
• Point staging at it. One row update.
• Ramp production. Set rollout_bps to 500, then 2500, then 10000, watching the proxy signals between steps. Each stage needs enough traffic to mean something — the sample-size arithmetic is the same as for a model canary.
• Roll back by setting the pointer to previous. No deploy, no build, seconds. This is the capability you are actually buying.

Keep the review. A prompt change that goes live without a second pair of eyes is a production change without code review, and the fact that it is prose does not make it lower risk — prompt edits are one of the most common causes of quality regressions precisely because they feel informal.

A question that comes up immediately: should prompt versions live in git instead? For teams where only engineers edit prompts, yes — a directory of files, a build step that hashes them into the binary, and the pointer as a small config record is simpler than a service and gets you review for free. The registry earns its complexity when the people editing prompts are not the people who deploy, and the honest test is whether that is true for you today rather than whether it might be later.

Either way, the non-negotiable part is the same and it is small: the resolved version identifier must land on the request row. Without it every downstream comparison — cost by prompt version, quality by prompt version, the canary’s two arms — has no join key, and the registry becomes a nicer way to make changes you still cannot measure.

The failure modes, and one fallback that covers them