Provider-Agnostic Code: The Interface That Survives a Swap

Almost every team writes a thin wrapper around their model provider in the first week. Almost none of those wrappers survive the first migration, because they abstract the wrong thing: they hide the HTTP call, which was never the hard part.

The wrapper that does not help

The instinctive abstraction mirrors the vendor’s API. A complete(messages, model, temperature, tools) function, one implementation per vendor, a factory that picks one. It compiles, it looks clean, and it fails at the moment of use for a specific reason: the caller still knows it is talking to a chat completion API. Every call site passes a model name, builds a message array, and parses a response. Swapping vendors changes all of that, so the wrapper saved you the HTTP client and nothing else.

The tell is the parameter list. If your interface has temperature on it, it is an API wrapper. If it has documentText on it, it is an abstraction.

Abstract the task, not the API

The boundary worth drawing is at the task. Your application does not want “a completion”; it wants a summary of this document, or a category for this ticket, or an answer grounded in these passages. Each of those is a function from typed input to typed output, and everything model-shaped lives on the far side of it.

// The interface the application sees. No model, no messages, no temperature.
export interface Classifier {
  classify(input: { text: string; categories: string[] }, ctx: Ctx):
    Promise<{ category: string; confidence: number; meta: RunMeta }>;
}

// Everything vendor-shaped lives behind it.
export type RunMeta = {
  route: string;          // which model/provider actually served it
  tokensIn: number;
  tokensOut: number;
  costCents: number;
  finish: "stop" | "length" | "filter" | "tool";
  attempts: number;
};

export type Ctx = { deadline: Deadline; idempotencyKey?: string; signal?: AbortSignal };

Three consequences fall out immediately. The prompt is now an implementation detail of a particular adapter, which is correct, because prompts are model-specific and pretending otherwise is what makes fallback chains fragile. The test double is trivial — a classifier that returns a fixed category needs no HTTP mocking at all. And the retry, deadline and breaker logic has exactly one place to live rather than being sprinkled through call sites.

The RunMeta return is not decoration. If the interface returns only the answer, then cost, route and token counts have to be smuggled out through logging or a side channel, and every consumer that wants to attribute spend reaches around the abstraction. Make the observability part of the contract.

What the interface looks like

For a handful of tasks the interface-per-task approach is clean. Once you have thirty tasks it becomes repetitive, and the useful generalisation is a single executor parameterised by a task definition:

type Task<I, O> = {
  name: string;                       // stable; goes in logs, caches, metrics
  version: number;                    // bump when the prompt changes
  render: (input: I, m: ModelProfile) => Request;   // per-model prompting
  parse: (raw: string) => O;                        // total, throws on garbage
  validate: (out: O) => void;                       // your schema, not theirs
  route: string[];                    // ordered rungs, by profile id
  maxOutputTokens: number;
};

async function run<I, O>(task: Task<I, O>, input: I, ctx: Ctx): Promise<Run<O>> { ... }

The version field earns its place quickly. It goes into the cache key, so editing a prompt cannot serve stale results; into the log line, so a quality regression can be traced to a change; and into the evaluation record, so a stored score is attached to the prompt it was measured against.

Five leaks, and what to do about each

The pattern in that table is worth naming. Seal a leak when the difference is incidental — a name, an encoding, a shape. Expose it when the difference is real, because an abstraction that hides a real difference does not remove it; it just moves the surprise to runtime.

Capabilities as data

The thing that makes routing and fallback tractable is a record per model describing what it can do, kept as data rather than as branches in code:

type ModelProfile = {
  id: string;
  contextTokens: number;
  maxOutputTokens: number;
  supports: { tools: boolean; jsonSchema: boolean; images: boolean; streaming: boolean };
  promptStyle: "system-role" | "prepend-user";
  estimateTokens: (text: string) => number;
};

With profiles, “skip any rung that cannot hold this input” and “skip any rung that cannot do tool calls” are one-line filters rather than special cases, and adding a model is adding a row. Keep the profiles in one file, and keep them honest — a profile that claims a capability the model lacks produces a failure on the fallback path, which is the path you test least.

When not to abstract at all

There is a real cost to this, and two situations where the right answer is to skip it. If you are building something exploratory whose shape is still changing weekly, an abstraction over one implementation is a guess about a future you cannot see, and you will spend more time maintaining the boundary than a migration would ever cost.

And if a feature genuinely depends on one model’s distinctive behaviour — a capability nothing else offers — do not pretend otherwise. Write it directly against that model, name the file after the model, and accept that this feature has a hard dependency. A false abstraction over a real dependency is worse than an honest direct call, because it tells the next reader that a swap is possible.