Testing Node.js Code That Calls an LLM

The interesting question in a Node service that calls a model is not how to mock it. It is which of your three layers actually needs a fake at all — because two of them do not, and testing those two first removes most of what people reach for a mocking library to do.

Three layers, one of which needs a fake

Split the code before you write a single test. Almost every LLM feature in a Node codebase is three things stacked:

• Assembly. Turning application state into a request body — the system prompt, the message array, the tool schemas, the token budget you truncated to. Pure functions of their inputs.
• Transport. One HTTP call, plus everything you wrapped around it: retries, timeouts, an idempotency key, the header you attach for cost attribution.
• Interpretation. Turning a completion into something your application can use — parsing JSON out of it, validating it, dispatching a tool call, mapping a refusal onto an error.

Assembly and interpretation are ordinary deterministic code and need no network and no fake. They are also where most bugs live, which is why testing the prompt builder and testing the output parser are separate pages rather than sections here. Only transport needs something standing in for the provider, and it needs a stand-in that behaves like an HTTP server, because that is what your code is talking to.

Intercept HTTP, not the SDK

The tempting move is vi.mock("openai"). It works and it is a trap, because the thing you have now frozen is the shape of somebody else’s client object. When the SDK renames a field, moves a method or changes what it throws, your mock keeps returning the old shape and your tests stay green while production breaks. That is the classic failure of mocking a dependency you do not own.

Intercepting HTTP keeps the real SDK in the test. Its request construction, its error classes, its retry behaviour and its streaming parser all execute; only the socket is replaced. Mock Service Worker does this in Node by patching the request modules, so it catches global fetch and anything built on http.request alike. Its Node entry point is documented by the msw project at mswjs.io/docs/api/setup-server. The handler patterns themselves — matching a chat completions URL, shaping a reply, covering the streaming endpoint — are set out in mocking the OpenAI API with msw, so this page keeps to where the boundary goes and what to assert once it is there.

// test/setup.ts
import { setupServer } from "msw/node";
import { afterAll, afterEach, beforeAll } from "vitest";

export const server = setupServer();

beforeAll(() => server.listen({ onUnhandledRequest: "error" }));
afterEach(() => server.resetHandlers());
afterAll(() => server.close());

Two limits are worth knowing before you commit to this. Interception happens at the request layer, so anything a client does below it — a raw socket, a gRPC transport, a native binding — passes straight through untouched, and a small number of provider SDKs use one of those for at least one endpoint. And interception is per-process: if your runner executes test files in parallel worker threads, each worker needs its own server instance, which is exactly what a setup file registered per worker gives you. Handlers added by one test cannot leak into another as long as resetHandlers runs after each, which is the only reason that line is in the boilerplate.

Register that file with setupFiles in your vitest config. The server starts with no handlers on purpose: every test declares the responses it needs and nothing is inherited from the test before it.

What to assert on an outgoing request

Here is the part that decides whether the suite is worth anything. You cannot assert on the reply, because you wrote it. What you can assert on is the request, which your code built and which is entirely deterministic. Capture it in the resolver:

import { http, HttpResponse } from "msw";
import { expect, it } from "vitest";
import { server } from "./setup";
import { summarise } from "../src/summarise";

it("sends the document under a user turn and pins the model", async () => {
  let body: any;

  server.use(
    http.post("https://api.openai.com/v1/chat/completions", async ({ request }) => {
      body = await request.json();
      return HttpResponse.json({
        object: "chat.completion",
        model: "gpt-4o-mini",
        choices: [
          {
            index: 0,
            finish_reason: "stop",
            message: { role: "assistant", content: "A short summary." },
          },
        ],
        usage: { prompt_tokens: 41, completion_tokens: 4, total_tokens: 45 },
      });
    }),
  );

  await summarise("Quarterly revenue rose.");

  expect(body.model).toBe("gpt-4o-mini");
  expect(body.messages).toHaveLength(2);
  expect(body.messages[0].role).toBe("system");
  expect(body.messages[1].content).toContain("Quarterly revenue rose.");
  expect(body.temperature).toBe(0);
  expect(body.max_tokens).toBeLessThanOrEqual(256);
});

Every one of those assertions is about a decision your code made. That the model is pinned rather than defaulted. That the document went into a user turn and not into the system prompt, where a later instruction could not override it. That the token ceiling you thought you set is actually on the wire. Those break for real reasons and never break for a reason you cannot act on.

The response object above is not decoration either. Give it a real finish_reason, real usage numbers and the real nesting, because your interpretation layer reads those fields, and a stub that omits usage is how a cost-tracking bug ships with a green suite.

Making a real network call fail the run

onUnhandledRequest: "error" in the snippet above is the single most valuable line in the setup. Without it, a code path you forgot to cover reaches out to the real provider, and depending on whether a key happens to be in the environment it either costs money quietly or fails with an authentication error that reads like a config problem rather than a missing handler.

Belt and braces: unset the credential in the test environment as well, so a request that escapes interception cannot succeed even by accident. In vitest, env in the config or a line in your setup file does it.

process.env.OPENAI_API_KEY = "sk-test-not-a-real-key";
process.env.OPENAI_BASE_URL = "https://api.openai.com/v1";

Pin the base URL rather than leaving it unset, too. A colleague with a proxy configured in their shell will otherwise send requests somewhere your handler pattern does not match, and msw will correctly report an unhandled request for a URL nobody else in the team sees.

A suite you can run now

1. Install the two dev dependencies: npm i -D vitest msw. msw v2 requires a Node version with global fetch, so Node 18 or later.
2. Create test/setup.ts exactly as above and pointtest.setupFiles at it in vitest.config.ts.
3. Move the code that builds your request body into an exported function that takes its inputs as arguments and returns the body. Write assertions against its return value directly — no server, no handler, microseconds per case.
4. Write the transport test above for one happy path, asserting only on the captured request and on what your function returns.
5. Add one failure case with a status you actually handle: a handler returning HttpResponse.json({ error: { message: "Rate limit reached" } }, { status: 429 }) and an assertion that your code surfaces a retryable error rather than a raw SDK exception.
6. Run with the key unset locally to prove the guard works. If anything still reaches the network, msw names the offending URL and you have found a code path with no test.

From there the growth is by fixture rather than by scaffolding: each new provider behaviour you have seen in production becomes another handler, and the surrounding suite does not change.