Testing a Repository Pattern Wrapped Around an LLM Call

A repository hides a store behind domain language, and every testing guide on the pattern assumes the store is a database. Three properties of a model break those assumptions, and each one changes what the repository must return.

Three ways it is not a database

A read costs money and the amount varies. With a database, cost is an operational concern that never appears in a return type. With a model, the same logical query costs a different amount every time, depending on how long the answer was. If the repository swallows that, nothing above it can attribute spend to a feature, and the only place left to measure is a provider dashboard that knows nothing about your tenants.

A read can half-succeed. A truncated answer is not an error — you get a 200, a body, and a finish_reason of length. The same is true of a content filter stopping generation partway. A database either returns rows or raises.

Two identical reads return different bytes. Which means the classic repository test — write, read back, compare for equality — has no analogue at all. You cannot assert the value. You can only assert its shape, its invariants and its provenance.

And a read takes seconds, not milliseconds. Callers written against a database repository assume a lookup is cheap enough to do inside a loop, inside a request handler, or while holding a lock. None of those is safe at second-scale latency. Keep the interface asynchronous everywhere and never add a synchronous convenience wrapper, because the first thing somebody will do with one is call it inside a map over a thousand rows.

What crosses the repository boundary

Those three facts dictate the return type. A repository that returns a bare domain object is throwing away the two things its callers need to behave correctly.

// src/domain/ports.ts
export type Extraction = {
  invoiceNumber: string;
  totalCents: number;
  currency: string;
};

export type Read<T> = {
  value: T;
  complete: boolean;          // false when generation stopped early
  cost: { inputTokens: number; outputTokens: number; model: string };
};

export interface InvoiceExtractor {
  extract(documentText: string): Promise<Read<Extraction>>;
}

complete is a boolean rather than the raw provider string on purpose: the values differ between providers, and translating them is the adapter’s job, not the caller’s. cost keeps the token counts and the model id rather than a currency amount, because prices change and a stored number would be wrong six months later while token counts stay true forever.

One thing deliberately does not cross the boundary: the prompt. If a caller can pass prompt text through the repository, it is not a repository at all, it is the client under a new name, and every call site becomes a place where prompt changes have to be found and reviewed. The interface takes domain inputs — a document, an order, a ticket — and the prompt is an implementation detail of the adapter, tested where it is assembled rather than where it is used.

Testing the adapter: mapping and taxonomy

The adapter is the only implementation that talks to a provider, so it is the only one that needs an intercepted transport. Its tests are about two things and nothing else: does it map a real response body onto the domain type, and does it map each failure onto the right category.

import { describe, expect, it } from "vitest";
import { http, HttpResponse } from "msw";
import { server } from "../test/setup";
import { OpenAiInvoiceExtractor } from "../src/adapters/invoice";
import { TransientLlmError, MalformedOutputError } from "../src/domain/errors";

const repo = new OpenAiInvoiceExtractor("sk-test", { maxRetries: 0 });

const reply = (content: string, finish = "stop") =>
  HttpResponse.json({
    model: "gpt-4o-mini",
    choices: [{ index: 0, finish_reason: finish, message: { role: "assistant", content } }],
    usage: { prompt_tokens: 812, completion_tokens: 24, total_tokens: 836 },
  });

describe("OpenAiInvoiceExtractor", () => {
  it("maps a well-formed reply onto the domain type and keeps the cost", async () => {
    server.use(
      http.post("https://api.openai.com/v1/chat/completions", () =>
        reply('{"invoiceNumber":"INV-7781","totalCents":124900,"currency":"EUR"}'),
      ),
    );

    const read = await repo.extract("…");

    expect(read.value).toEqual({ invoiceNumber: "INV-7781", totalCents: 124900, currency: "EUR" });
    expect(read.complete).toBe(true);
    expect(read.cost).toEqual({ inputTokens: 812, outputTokens: 24, model: "gpt-4o-mini" });
  });

  it("reports incomplete rather than throwing when generation was cut short", async () => {
    server.use(
      http.post("https://api.openai.com/v1/chat/completions", () =>
        reply('{"invoiceNumber":"INV-7781","totalCents":1249', "length"),
      ),
    );
    await expect(repo.extract("…")).rejects.toBeInstanceOf(MalformedOutputError);
  });

  it("classifies a 503 as transient", async () => {
    server.use(
      http.post("https://api.openai.com/v1/chat/completions", () =>
        HttpResponse.json({ error: { message: "upstream unavailable" } }, { status: 503 }),
      ),
    );
    await expect(repo.extract("…")).rejects.toBeInstanceOf(TransientLlmError);
  });
});

The middle test is the one worth arguing about, and the assertion encodes a real decision: when the JSON is truncated there is no domain value to return, so complete: false is not expressive enough and the adapter throws instead. For a prose summary the opposite choice is right — a shortened summary is still usable, so return it with complete: false and let the caller decide. Whichever you pick, the test is where the decision is written down.

Notice what these tests do not claim. None of them asserts that the model was right about the invoice, because that is a question about a third party’s accuracy on a distribution of documents and it needs a labelled dataset and a score, not an assertion. The adapter tests assert only that a given response body maps to a given domain value, which is entirely your code and entirely deterministic. Keeping those two concerns in separate suites is what lets you run this one on every commit.

The in-memory repository for everyone else

Every test above the adapter uses an in-memory implementation. This is the part the database literature gets right and it transfers cleanly: one fake implementation, shared, that satisfies the same interface.

// test/doubles/in-memory-extractor.ts
import type { Extraction, InvoiceExtractor, Read } from "../../src/domain/ports";

export class InMemoryExtractor implements InvoiceExtractor {
  readonly calls: string[] = [];
  private queue: Array<Read<Extraction> | Error> = [];

  push(next: Read<Extraction> | Error) { this.queue.push(next); }

  async extract(documentText: string): Promise<Read<Extraction>> {
    this.calls.push(documentText);
    const next = this.queue.shift();
    if (!next) throw new Error("InMemoryExtractor: no queued result for this call");
    if (next instanceof Error) throw next;
    return next;
  }
}

Throwing on an empty queue rather than returning a default is deliberate. A default result means a test that makes an unexpected extra call still passes, and an unexpected extra call to a paid API is exactly the bug you want the suite to catch. The implements clause is what stops the double drifting: change the interface and the compiler fails here, not six months later in production.

Give the queued results realistic token counts rather than zeros. A test that asserts cost aggregation against a double which always returns zero passes for every implementation, including one that throws the numbers away entirely — the assertion looks meaningful and constrains nothing. The same applies to complete: if every canned result is complete, no caller’s handling of a truncated read is ever exercised, so keep at least one incomplete result in the fixtures and one test that consumes it.

Partial failure is the interesting case

A repository over a batch is where this pattern earns its keep. Ask for twenty invoices and three of them come back truncated, one hits a content filter, and one 429s halfway through. A database repository would be in a transaction; here there is none, and seventeen of the calls have already been paid for.

So the batch method returns per-item outcomes rather than throwing, and the test asserts the split:

it("returns per-item outcomes and never discards work already paid for", async () => {
  const repo = new InMemoryExtractor();
  repo.push(ok("INV-1"));
  repo.push(new MalformedOutputError("truncated"));
  repo.push(ok("INV-3"));

  const result = await extractAll(repo, ["a", "b", "c"]);

  expect(result.succeeded.map((r) => r.value.invoiceNumber)).toEqual(["INV-1", "INV-3"]);
  expect(result.failed).toHaveLength(1);
  expect(result.failed[0].input).toBe("b");
  expect(result.totalInputTokens).toBe(1624);   // both successes, not all three
});

That last assertion is the one people forget. If a partial batch reports the cost of everything or of nothing, your spend attribution is wrong in a way no dashboard will reveal. Retrying the failed item then adds its own question — whether the retry is safe to repeat — which is its own page. Assert that the retry re-sends only the failed item, too: a batch method that retries the whole list on any failure pays for seventeen successful extractions a second time, and the bug is invisible in any test whose fixtures all succeed.