Centralizing Model Configuration Before a Migration

The point of centralising is not tidiness. It is that a migration becomes a change to data you can review, stage and roll back, instead of a change to code in fourteen files that ships all at once.

Why an env var is not enough

The obvious refactor is to replace every literal with process.env.MODEL or its equivalent. That is better than nothing, and it fails on the first real migration, for three reasons.

One: different call sites need different models. A summariser, a classifier and a code generator do not swap together, so a single variable becomes three variables with ad-hoc names, which is the original problem with extra steps.

Two: a model name is not the only thing that changes when the model changes. The output cap changes, the context window changes, whether the system instruction is a message or a top-level field changes, the legal temperature range changes — Anthropic’s Messages API documents temperature in the range 0 to 1, while OpenAI’s Chat Completions documents 0 to 2, so the same 1.4 that was mildly creative on one is a validation error on the other. A configuration that carries only a string leaves every one of those assumptions hardcoded somewhere else.

Three: an environment variable is a deploy-time value. Migrations are done per tenant, per route or per percentage of traffic, and none of those are expressible as a value you have to restart to change.

Roles, not models

Give each use of a model a stable logical name and let the physical model be an attribute of it. The role is what the application code refers to; the model is what the registry resolves it to.

// roles.ts — the only names application code is allowed to use
export type Role =
  | "chat.default"
  | "chat.long_context"
  | "summarize.batch"
  | "classify.cheap"
  | "extract.structured";

The gain is that the diff of a migration reads as a decision. “summarize.batch moved from provider A to provider B on the 14th” is reviewable; a diff touching fourteen files with a new string in each is not. It also means an experiment gets its own role rather than a stray literal, which is what produced the eleven distinct models the hardcoded-name audit usually turns up.

What the registry record has to carry

The registry maps a role to a record. The fields below are the ones that, in practice, are otherwise scattered as constants and inline branches. Every one of them is a thing that differs between two models you might reasonably swap between.

export type ModelEntry = {
  // identity
  provider: "openai" | "anthropic" | "google" | "self_hosted";
  apiModel: string;          // the literal string the provider expects
  baseUrl?: string;          // absent means the SDK default
  credentialRef: string;     // a key name, never a key

  // budgets — the numbers that were previously bare integers
  contextWindow: number;     // total input+output the model accepts
  defaultMaxOutput: number;  // what you send as the output cap
  hardMaxOutput: number;     // what the provider will refuse above

  // capability flags — the reason a swap can be illegal
  systemPrompt: "message_role" | "top_level_field";
  tools: boolean;
  strictJsonSchema: boolean; // schema-constrained output, not just JSON mode
  streaming: boolean;
  usageInStream: boolean;    // does the stream carry token counts at all
  temperatureRange: [number, number];
  stopSequenceLimit: number | null;

  // economics — placeholders you substitute, not quoted prices
  inputPricePerMTok: number;
  outputPricePerMTok: number;
};

The capability flags are the part teams leave out and then need. They turn “can I route this role to that model?” from a question somebody answers from memory into a predicate the code can evaluate before it sends anything:

function canServe(entry: ModelEntry, need: Requirements): string[] {
  const problems: string[] = [];
  if (need.tools && !entry.tools) problems.push("no tool calling");
  if (need.schema && !entry.strictJsonSchema)
    problems.push("no schema-constrained output; needs validate-and-retry");
  if (need.maxOutput > entry.hardMaxOutput)
    problems.push("output cap too low: " + entry.hardMaxOutput);
  if (need.temperature < entry.temperatureRange[0] ||
      need.temperature > entry.temperatureRange[1])
    problems.push("temperature outside provider range");
  return problems;
}

Run that predicate in a test over every role in the registry. It is the cheapest migration safeguard available: a candidate model that cannot serve a role fails in CI rather than in production at 3am, and the failure message names the specific capability that is missing.

The adapter layer around it

Configuration alone does not make providers interchangeable — the request shapes differ. The adapter is the thin layer that takes a role-shaped request and produces a provider-shaped one. Keep it deliberately small and honest about what it cannot do.

A hand-rolled adapter needs four functions and nothing more: build the request, parse the response, normalise the stream, and classify the error. The first is mostly renaming: a system instruction goes into the messages array with role system for OpenAI-shaped APIs and into the top-level system field for Anthropic’s Messages API; an output cap goes to max_completion_tokens, max_output_tokens or max_tokens depending on which API you are talking to; stop strings go to stop or stop_sequences.

The interesting part is the third function, and the interesting part of that is what does not survive. Streaming shapes are not renamings of each other: one side emits chunk objects with a choices[0].delta containing partial content, the other emits named server-sent events — message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop — where the token counts arrive on different events from the text. Your normalised stream should emit a small set of your own events (text delta, tool-call delta, usage, done) and each adapter should construct them; anything that tries to translate one vendor’s event names into another’s will eventually meet an event with no counterpart. The general treatment is in streaming transport.

The fourth function matters more than its size suggests. Classify every error into your own small enum — rate_limited, overloaded, context_too_long, invalid_request, auth, timeout, unknown — because the retry policy belongs to your application and not to whichever SDK happens to be under it. That is also the only way backoff behaviour stays testable when the provider under the adapter changes.

Migrating call sites without a flag day

1. Create the registry and populate it from the audit inventory, including the wrong and duplicated entries. Do not clean up yet — the first version should reproduce current behaviour exactly.
2. Convert one call site. Assert in a test that the request body the adapter builds is byte-identical to what the old code built, for a fixed input. This is the step that catches an accidentally changed default.
3. Convert the rest, one commit per call site, in order of traffic volume ascending. The lowest-traffic path is the one where a mistake is cheapest to notice.
4. Collapse duplicates only now. With every call site resolved through roles, two roles pointing at the same model are visibly the same decision, and merging them is a one-line change with a reviewable blast radius.
5. Add the resolved role and model to every log line and every metric tag, so that after the cutover you can answer “which requests went where” without guessing. The field-level design is in migrating a structured logging schema.

Making the old way impossible

A registry that can be bypassed is a registry that will be bypassed, usually by a well-meant hotfix at a bad hour. Two guardrails hold it: restrict the provider SDK import to the adapter directory with a lint rule or a CI grep, and make the registry the only code that reads the credential. If application code cannot construct a client, it cannot hardcode a model into one.

The payoff is measured in what a migration becomes. Moving a role to a new provider is a change to one record, deployable behind the same mechanism as any other configuration, revertible in the time it takes to change it back — and reviewable by someone who was not in the migration meeting.