Stubbing an LLM API With WireMock in Java

Every other library in this cluster works by patching something inside the process. WireMock does the opposite: it starts a real HTTP server on a real port and you point your client at it. That is heavier, and on the JVM it is very often the only option that works.

Why a server instead of a patched transport

In Node or Python you can reach into the HTTP layer from the test process. On the JVM there is no equivalent global hook that works across OkHttp, Apache HttpClient, the JDK’s own HttpClient and whatever your provider SDK chose — and the SDKs generally do not expose the client for you to replace.

So instead of intercepting the call, you move the destination. Every OpenAI-compatible client takes a base URL, because that is how people point them at Azure, a proxy or a self-hosted server. Point it at http://localhost:port where WireMock is listening and the request travels the full stack — serialisation, headers, connection pooling, timeouts, TLS if you enable it — and arrives somewhere you control.

That fidelity is the reason to like this approach rather than the reason to tolerate it. A test that patches the transport cannot catch a connection-pool exhaustion bug or a timeout that was configured on the wrong client. A test against a real socket can.

The stub

import static com.github.tomakehurst.wiremock.client.WireMock.*;

@WireMockTest
class SummariserTest {

    @Test
    void parsesTheFirstChoice(WireMockRuntimeInfo wm) {
        stubFor(post(urlEqualTo("/v1/chat/completions"))
            .willReturn(okJson("""
                {
                  "id": "chatcmpl-abc",
                  "model": "gpt-4o-mini",
                  "choices": [{
                    "index": 0,
                    "finish_reason": "stop",
                    "message": {"role": "assistant", "content": "Three bullets."}
                  }],
                  "usage": {"prompt_tokens": 214, "completion_tokens": 9}
                }
                """)));

        Summariser summariser = new Summariser(wm.getHttpBaseUrl(), "test-key");

        assertThat(summariser.summarise("some input")).isEqualTo("Three bullets.");
    }
}

The base URL is injected rather than fixed because @WireMockTest binds a random free port by default. Take it from WireMockRuntimeInfo and pass it in. Hard-coding a port is the fastest way to a suite that fails on a developer machine running something else, and to two test classes that cannot run at the same time.

Note that the constructor takes a base URL at all. If yours does not, that is the change to make first, and it is worth making regardless: it is the same parameter you need to point at a staging gateway, a self-hosted model, or a proxy.

Matching on the part of the body you mean

WireMock’s equalToJson compares whole documents, and a chat request is the wrong shape for that: it grows a field every time somebody adds a parameter, and the test that breaks is not the one about that parameter. Use matchingJsonPath to assert on individual paths and let the rest vary.

stubFor(post(urlEqualTo("/v1/chat/completions"))
    .withHeader("Authorization", matching("Bearer .+"))
    .withRequestBody(matchingJsonPath("$.model", equalTo("gpt-4o-mini")))
    .withRequestBody(matchingJsonPath("$.messages[0].role", equalTo("system")))
    .withRequestBody(matchingJsonPath("$.tools[?(@.function.name == 'lookup_order')]"))
    .willReturn(okJson(toolCallResponse)));

The header matcher is deliberately a pattern rather than a literal. You want to know that a key was attached, not to pin the key into the test file — the second is how secrets end up in a repository.

One WireMock-specific trap: if the request does not match any stub, you get a 404 with a near-miss report in the server log, and your client will surface that as a strange provider error rather than as a test failure. When a test fails with a message from deep inside the SDK, read WireMock’s near-miss output first. It tells you exactly which matcher missed and by how much.

Keep stub definitions out of the test methods once you have more than a handful. A small builder that takes a model id and an assistant message and returns a stub keeps each test to the one line that is actually about the behaviour under test, and it gives you a single place to fix when a provider adds a field. The alternative — a full JSON body pasted into every test method — is how a class reaches a thousand lines of which twenty are assertions.

Latency, chunking and failures

This is where running a real server pays off, because you can be unpleasant to your client in ways a patched transport cannot.

• Fixed delay. withFixedDelay(millis) on the response makes the server slow. Set it above your read timeout and assert that your code raises a timeout, and above your circuit breaker threshold and assert that it opens. A long completion is slow legitimately, so a client configured with no read timeout at all is a common and invisible bug until the day a provider hangs. See testing an LLM call timeout.
• Chunked dribble. withChunkedDribbleDelay(numberOfChunks, totalDuration) splits the response body into chunks spread over a period. For an SSE body this is the closest thing to a real stream any of these tools offer, and it will split a data: line mid-way, which is exactly the case that breaks naive parsers.
• Faults. WireMock can return an empty response, a malformed chunk or a connection reset. Providers do all three under load, and the code path that handles them is otherwise untested.
• Scenarios. A stateful scenario lets the first call return 429 and the second 200, which is how you test backoff without sleeping through a real one.

Verifying the request

Stub matching and verification are separate, and you want both. A stub that matched proves a request of that shape arrived; it does not prove how many arrived, and a duplicated completion is a bug that shows up on the invoice rather than in the output.

verify(exactly(1), postRequestedFor(urlEqualTo("/v1/chat/completions"))
    .withRequestBody(matchingJsonPath("$.stream", equalTo("false"))));

Assert the count explicitly. And where your application supports more than one provider, resist writing a second copy of the whole test class: keep one set of assertions and vary the base URL and the stubbed response body, so adding a provider costs a fixture rather than a file.

Verification also gives you a way to assert a negative, which stubs cannot. verify(exactly(0), postRequestedFor(...)) proves that a cached result, a short-circuit for an empty input, or a guard that refuses to call the model on unvalidated user text actually prevented the request. Those are the paths where a regression costs money rather than correctness, and they are invisible to any assertion phrased in terms of what the code returned.