What “OpenAI-Compatible” Actually Means for an Endpoint

“OpenAI-compatible” is not a certification and there is no conformance suite behind it. It is a claim that you can point an OpenAI client library at a different base_url and it will work. That claim is testable, and it is worth knowing exactly what it has to be true of.

What the claim is actually about

The reason the phrase exists at all is that OpenAI’s client libraries take a base URL as configuration. Set it to your own server, keep the same code, and the SDK will issue the same HTTP requests to your host. Everything the phrase means follows from that: an OpenAI-compatible endpoint is one that satisfies the expectations those libraries have when they parse a response.

Note what this makes the reference implementation. It is not a specification document — the compatibility target is the wire format that the SDKs accept, which is documented but not versioned as a spec that third parties can conform to. In practice the surface that matters is the smaller one that widely used clients depend on, and that is what a server should be tested against.

It is also worth being clear that the phrase now attaches to two different OpenAI APIs. The overwhelming majority of servers claiming compatibility implement /v1/chat/completions, the older Chat Completions shape. Support for the newer Responses API, with its typed streaming events and its input rather than messagesfield, is a separate and much rarer claim. If a server says “OpenAI-compatible” without qualification, assume Chat Completions.

The surface a client depends on

Concretely, a server has to get these right before any of the interesting behaviour matters.

• Path and method. POST to a path ending /chat/completions, where the configured base URL supplies the prefix. Clients append the path to the base URL as given, so a server whose base URL already ends in /v1 and a client configured with the same suffix produce /v1/v1/chat/completions and a 404. This is the single most common first failure.
• Auth. An Authorization: Bearer <key> header. A server that requires no key still has to tolerate the header being sent, because the SDK will send one; several clients also refuse to start without a non-empty key, which is why placeholder values are conventional.
• Request fields. At minimum model and messages, where each message is an object with role and content, and roles include at least system, user and assistant. Unknown fields in the body should be ignored rather than rejected — clients send fields their users set, and a server that 400s on an unrecognised key is unusable behind a shared abstraction.
• Response envelope. A JSON object with id, object set to the literal chat.completion, created, model, and a choices array whose entries have index, message (with role and content) and finish_reason. Typed SDKs validate this envelope; a missing object field or a created sent as a string rather than a number fails at deserialisation, before any of your model output is reached.
• A usage object. usage with prompt_tokens, completion_tokens and total_tokens. Some servers omit it. Nothing crashes, but every cost and quota feature downstream goes blank — see the usage object mapping page for why an absent count is worse than a wrong one.
• A model listing. GET /v1/models returning an object with object: "list" and a data array. Tools that populate a model dropdown call this, and a server without it appears to have no models even though inference works.

The streaming contract

Streaming is where most compatibility claims are thinnest, because it has more implicit requirements than the request shape does. With stream: true the server must respond with Content-Type: text/event-stream and emit server-sent events whose data: payload is a chunk object.

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","created":1750000000,
       "model":"local-model","choices":[{"index":0,
       "delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","created":1750000000,
       "model":"local-model","choices":[{"index":0,
       "delta":{"content":"Hel"},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","created":1750000000,
       "model":"local-model","choices":[{"index":0,
       "delta":{},"finish_reason":"stop"}]}

data: [DONE]

The requirements hidden in that are: object is chat.completion.chunk, not chat.completion; the content is under delta, not message; finish_reason is present and null on every chunk until the last one, where it is set; the id is stable across all chunks of one response; and the stream ends with the literal line data: [DONE], which is the one payload that is deliberately not JSON. A server that closes the connection instead leaves clients unable to distinguish completion from a dropped connection.

Two further details separate a working stream from a good one. Chunks must be flushed as they are produced rather than buffered, or the stream is a slow non-streaming response with extra steps — and any reverse proxy in front of the server must be configured not to buffer them either, which is a configuration failure that looks exactly like an implementation failure. And if the server supports stream_options: { include_usage: true }, it must emit the extra usage-bearing chunk with an empty choices array before the sentinel. If it does not support the option, ignoring it is acceptable; silently sending nothing while accepting the flag is what leaves callers with no token counts and no error. The chunk shape itself is covered in the streaming chunk format page.

Three levels of the same claim

When you read the phrase on a project’s README, it is usually one of three quite different assertions, and distinguishing them tells you what you will spend your week on.

• Transport-compatible. The path, the auth header and the response envelope are right, so a curl works and a simple client works. This is the level most self-hosted inference servers reach. Streaming may or may not be correct.
• SDK-compatible. The official client libraries work unmodified, including streaming, which means the envelope validates under typed deserialisation and the sentinel is present. This is the level the phrase implies and a meaningful fraction of servers do not reach.
• Semantically compatible. The parameters do what they do on OpenAI: temperature has the same range, stop sequences are honoured and excluded from the output, response_format constrains generation rather than being advisory, tool_choice is enforced. Almost nothing reaches this level completely, and the gaps are the subject of where OpenAI-compatible endpoints break.

What the claim never covers

Even a server at the third level is not a drop-in for a model, and conflating the two is where teams lose time. Compatibility is about the interface; it says nothing about tokenisation, so token counts and therefore costs and context limits differ. It says nothing about which models are available or what the model strings are, so the identifier in your request has to change. It says nothing about rate-limit headers, request ids, organisation headers or the shape of the error bodies, which are covered in the error shape mapping page.

And it says nothing about behaviour. Two servers can be perfectly compatible at the wire level and return answers different enough to fail your evaluations, because compatibility is a property of the envelope and quality is a property of the model inside it. Treat the two as separate migrations with separate tests, and run the interface test first: there is no point comparing outputs from an endpoint that is silently ignoring your stop parameter.