A Typed Client for Any Model Provider
13 min read · updated August 4, 2026
The value of typing a model client is not autocomplete. It is that a discriminated union makes the compiler enumerate the cases a provider can send you — text, tool call, refusal, filtered, truncated, error — and refuse to build when you have forgotten one.
Why the SDK types are not enough
Provider SDKs are typed, often well. The problem is that their types describe that provider’s wire format, so the moment your application handles two providers, or one gateway in front of several, your domain logic is written against whichever one you started with. Every addition after that is a translation layer bolted on from outside.
The second problem is subtler. Wire types are permissive by necessity — nearly every field is optional, because one object shape carries a streaming delta, a final message and a tool call. So choices[0].message.content types as string | null | undefined, and every call site handles three cases where the domain has one. That permissiveness is correct for a wire type and wrong for a domain type.
The fix is a small internal model of what your application deals in, with adapters at the edge. Perhaps two hundred lines, and it is the difference between adding a provider in an afternoon and adding one in a sprint.
Messages as a discriminated union
// types.ts
/* A message is one of four things, and the discriminant is `role`.
Writing it this way rather than as one optional-heavy object means the
compiler knows a tool message has a tool_call_id and a user message does
not, instead of both having "maybe". */
export type SystemMessage = { role: "system"; content: string };
export type UserMessage = {
role: "user";
content: string | ContentPart[]; // multimodal is an array, text is not
};
export type AssistantMessage = {
role: "assistant";
content: string | null; // null when the turn is only tool calls
toolCalls?: ToolCall[];
};
export type ToolMessage = {
role: "tool";
toolCallId: string; // required here, absent elsewhere
content: string;
};
export type Message = SystemMessage | UserMessage | AssistantMessage | ToolMessage;
export type ContentPart =
| { type: "text"; text: string }
| { type: "image"; url: string; detail?: "low" | "high" | "auto" };
export type ToolCall = {
id: string;
name: string;
/* Deliberately `unknown`, not `any` and not a typed shape. The provider
sends a JSON string the model produced; nothing has validated it against
your tool's schema yet. Forcing the caller to parse before use is the
entire point, and it is where a Zod schema goes. */
arguments: unknown;
};The arguments: unknown prevents a whole class of production incident. A typed shape there would be a lie — the model generated that JSON and nothing has checked it — and any would let a caller read args.userId and pass it straight to a database query. unknown forces a parse, and the Zod schema you already have is the natural thing to parse it with. The security argument is in secure tool calls.
Errors as values, not exceptions
A thrown error carries no type information. catch (err) gives you unknown, and every caller reconstructs the same instanceof-and-string-match logic to decide whether to retry, fall back or surface the failure. Return a union instead, and the decision becomes a switch the compiler checks.
export type Failure =
| { kind: "auth"; status: 401 | 403; message: string }
| { kind: "rate_limit"; retryAfterMs: number | null }
| { kind: "insufficient_credit"; message: string }
| { kind: "context_length"; limit: number | null; sent: number | null }
| { kind: "content_filter"; stage: "input" | "output" }
| { kind: "timeout"; afterMs: number }
| { kind: "network"; cause: string }
| { kind: "provider"; status: number; body: string }
| { kind: "aborted" };
export type Completion = {
text: string;
toolCalls: ToolCall[];
finish: "stop" | "length" | "tool_calls" | "content_filter";
usage: { promptTokens: number; completionTokens: number } | null;
model: string; // what actually served it, after any fallback
};
export type Result =
| { ok: true; completion: Completion }
| { ok: false; failure: Failure };That union is where the retry policy lives, once rather than at every call site. rate_limit and network are retryable; timeout is retryable once with a longer budget; auth, insufficient_credit and context_length are not retryable at all, and retrying them just spends what is left. Splitting rate_limit from insufficient_credit matters specifically because providers send both as HTTP 429 — the status alone cannot tell you which, and one of them will never succeed on retry.
Exhaustiveness that fails the build
// exhaustive.ts
export function assertNever(x: never): never {
throw new Error("unhandled variant: " + JSON.stringify(x));
}
// ui.ts
import type { Failure } from "./types";
import { assertNever } from "./exhaustive";
export function messageFor(failure: Failure): string {
switch (failure.kind) {
case "auth":
return "The API key was rejected. Check the key and its permissions.";
case "rate_limit":
return failure.retryAfterMs
? "Rate limited. Try again in " + Math.ceil(failure.retryAfterMs / 1000) + "s."
: "Rate limited. Try again shortly.";
case "insufficient_credit":
return "The account is out of credit.";
case "context_length":
return failure.limit
? "Too long: " + failure.sent + " tokens sent, limit " + failure.limit + "."
: "The conversation is too long for this model.";
case "content_filter":
return failure.stage === "input"
? "That request was blocked before it reached the model."
: "The answer was blocked by a safety filter.";
case "timeout":
return "No response after " + Math.round(failure.afterMs / 1000) + "s.";
case "network":
return "Could not reach the provider.";
case "provider":
return "The provider returned an error (" + failure.status + ").";
case "aborted":
return "Cancelled.";
default:
return assertNever(failure); // <- compile error if a variant is added
}
}Add a tenth variant to Failure and this file stops compiling, with an error pointing at the exact line. That is a different quality of guarantee from a test: a test catches it only if somebody wrote the test, and the compiler catches it in every switch in the codebase without anybody remembering. This is the strongest single reason to model errors as a union rather than as strings.
The client
// client.ts
import type { Message, Result, Failure } from "./types";
export type ClientOptions = {
baseUrl?: string;
apiKey: string;
defaultModel: string;
timeoutMs?: number;
};
export type CompleteOptions = {
model?: string;
maxTokens?: number;
temperature?: number;
signal?: AbortSignal;
};
export function createClient(opts: ClientOptions) {
const baseUrl = opts.baseUrl ?? "https://api.multigrid.ai/v1";
const timeoutMs = opts.timeoutMs ?? 60_000;
async function complete(
messages: Message[],
o: CompleteOptions = {},
): Promise<Result> {
const signal = o.signal
? AbortSignal.any([o.signal, AbortSignal.timeout(timeoutMs)])
: AbortSignal.timeout(timeoutMs);
let res: Response;
try {
res = await fetch(baseUrl + "/chat/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: "Bearer " + opts.apiKey,
},
body: JSON.stringify({
model: o.model ?? opts.defaultModel,
messages: messages.map(toWire),
max_tokens: o.maxTokens ?? 1024,
temperature: o.temperature,
}),
signal,
});
} catch (err) {
return { ok: false, failure: classifyThrown(err, timeoutMs) };
}
if (!res.ok) return { ok: false, failure: await classifyResponse(res) };
const data = await res.json();
const choice = data.choices?.[0];
return {
ok: true,
completion: {
text: choice?.message?.content ?? "",
toolCalls: (choice?.message?.tool_calls ?? []).map((t: any) => ({
id: t.id,
name: t.function?.name ?? "",
arguments: safeJson(t.function?.arguments),
})),
finish: choice?.finish_reason ?? "stop",
usage: data.usage
? {
promptTokens: data.usage.prompt_tokens,
completionTokens: data.usage.completion_tokens,
}
: null,
model: data.model ?? o.model ?? opts.defaultModel,
},
};
}
return { complete };
}
function safeJson(raw: unknown): unknown {
if (typeof raw !== "string") return raw;
try {
return JSON.parse(raw);
} catch {
return { __unparsed: raw }; // never throw on the model's own output
}
}
function classifyThrown(err: unknown, timeoutMs: number): Failure {
const e = err as { name?: string; message?: string };
if (e?.name === "AbortError") return { kind: "aborted" };
if (e?.name === "TimeoutError") return { kind: "timeout", afterMs: timeoutMs };
return { kind: "network", cause: e?.message ?? String(err) };
}
async function classifyResponse(res: Response): Promise<Failure> {
const body = await res.text().catch(() => "");
const lower = body.toLowerCase();
if (res.status === 401 || res.status === 403) {
return { kind: "auth", status: res.status, message: body.slice(0, 300) };
}
if (res.status === 429) {
// Two different conditions wear this status. Read the body, not the code.
if (lower.includes("credit") || lower.includes("balance") || lower.includes("quota")) {
return { kind: "insufficient_credit", message: body.slice(0, 300) };
}
const header = res.headers.get("retry-after");
return { kind: "rate_limit", retryAfterMs: header ? Number(header) * 1000 : null };
}
if (lower.includes("context length") || lower.includes("maximum context")) {
return { kind: "context_length", limit: null, sent: null };
}
if (lower.includes("content_filter") || lower.includes("content policy")) {
return { kind: "content_filter", stage: "input" };
}
return { kind: "provider", status: res.status, body: body.slice(0, 500) };
}
function toWire(m: Message): Record<string, unknown> {
switch (m.role) {
case "tool":
return { role: "tool", tool_call_id: m.toolCallId, content: m.content };
case "assistant":
return {
role: "assistant",
content: m.content,
...(m.toolCalls?.length
? {
tool_calls: m.toolCalls.map((t) => ({
id: t.id,
type: "function",
function: { name: t.name, arguments: JSON.stringify(t.arguments) },
})),
}
: {}),
};
default:
return { role: m.role, content: m.content };
}
}String-matching the error body in classifyResponse is not elegant, and it is honest: providers do not agree on error codes, so something has to do this translation. Doing it in one function that returns a typed value is strictly better than doing it implicitly at twelve call sites. Normalising API errors is the general version of the problem.
Where the abstraction has to leak
A common interface over several providers is worth building and it is not free. Four things do not survive the abstraction, and pretending otherwise is how these clients become worse than the SDKs they replaced.
- Sampling parameters.
temperatureandtop_pare near-universal. Frequency and presence penalties, repetition penalties, min-p, logit bias and seeds are not, and their ranges differ where they exist. Expose the common two as first-class options, and everything else through a passthrough that names the provider. - Reasoning controls. Reasoning models take effort or budget parameters with different names and different semantics per provider, and their tokens are billed differently. Do not flatten these into one option; the meanings genuinely differ.
- Structured output. Schema-constrained decoding exists under different field names, supporting different JSON Schema subsets. Feature-detect and degrade to prompt-plus-validate rather than assuming.
- Caching. Some providers cache prefixes automatically, some require explicit markers in the request. That distinction is invisible in a unified type and very visible in the bill — see how prompt caching differs by provider.
The escape hatch that keeps this honest is a single providerOptions?: Record<string, unknown> merged into the request body by the adapter. It is untyped on purpose: it marks the boundary where you have left the common model, and marking it is better than either pretending the differences do not exist or growing the shared type until it is the union of every provider’s quirks.