Running LM Studio's Local Server in OpenAI-Compatible Mode

LM Studio’s server exists so that code written against a hosted API can be pointed at your own machine by changing one string. It mostly delivers that, and the interesting part of this page is the handful of places where it does not.

What the server is

It is an HTTP server, part of the same process that runs the chat UI, that exposes the loaded model over familiar routes. Per LM Studio’s OpenAI compatibility documentation, the base URL is http://localhost:1234/v1 and the supported endpoints are:

• GET /v1/models — what is available locally. The id values it returns are the model keys you pass in requests, and they are LM Studio’s own identifiers, not hosted model names.
• POST /v1/chat/completions — the one you want, with messages, temperature and streaming.
• POST /v1/completions — raw text continuation, no chat template applied. Useful for base models and fill-in tasks, wrong for anything instruction-tuned.
• POST /v1/embeddings — requires an embedding model loaded, which is a different model from your chat one.
• POST /v1/responses — the newer request shape, present alongside the older ones.

There is no authentication. Any client library that insists on an API key will accept an arbitrary string, because nothing checks it. That also means binding this server to anything other than loopback is a decision about network exposure, not a convenience.

Starting it

Two routes, and the CLI is the one worth learning because it survives the UI being rearranged. lms ships with the app. Per the documented flags, lms server start takes:

• --port — the port to run on. If not provided, it uses the last used port, which is a small trap: the port is remembered state rather than a constant, so a script that assumes 1234 can be wrong on a machine where somebody changed it once.
• --cors — enable CORS support for web application development. Documented as disabled when not set, so a browser calling this directly needs it.
• --bind — the network address to bind to, with 127.0.0.1 the default and 0.0.0.0 to listen on all IPv4 interfaces.

Making the first call

1. Download a model in the app, or from the CLI with lms get — see loading a model straight from Hugging Face.
2. Start the server and confirm it is up:

   lms server start --port 1234
   lms server status

3. Ask it what it has, and take the model key from the response rather than guessing it:

   curl -s http://localhost:1234/v1/models

4. Make a chat request against that key:

   curl http://localhost:1234/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{
       "model": "MODEL_KEY_FROM_STEP_3",
       "messages": [
         {"role": "system", "content": "Answer in one sentence."},
         {"role": "user", "content": "Why is output slower than input?"}
       ],
       "temperature": 0.2
     }'

5. Point an existing client at it by changing only the base URL. In the OpenAI Python client that is OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio"); the key is required by the library and ignored by the server.
6. Watch what the server is doing while you test, which is the part people skip: lms log stream prints the prompts going in and the text coming out, including the fully rendered prompt after the chat template has been applied.

That last step is worth keeping open the first time you connect a real application. Most “the local model is much worse” reports resolve into the prompt not looking the way the author assumed, and the log stream is where you see that directly rather than inferring it.

Just-in-time loading

A request naming a model that is not currently in memory triggers a load rather than an error. That is convenient and it distorts your first latency measurement badly: the first call after a cold start includes reading several gigabytes from disk and allocating the cache, which can be tens of seconds on a large model, and every subsequent call does not.

Two consequences for anything automated. A timeout tuned to warm latency will fire on the first request of the day, so either raise it or warm the model deliberately with lms load before traffic arrives. And an idle model may be unloaded again to reclaim memory, so “it was fast a minute ago” is not evidence that it is loaded now — lms ps is.

Where the compatibility stops

The wire format matches. The behaviour behind it does not always, and these are the differences that turn into bugs.

• Model identifiers are local. Nothing here is called gpt-4o-mini. Any code with a hardcoded hosted model name has to be parameterised before it can be pointed anywhere else, which is usually the first change a “just change the base URL” migration actually requires.
• Context length is a load-time property. On a hosted API the window is a fact about the model. Here it was fixed when the model was loaded, and an over-long prompt meets whatever LM Studio’s configured overflow behaviour is rather than a clean rejection — see LM Studio’s context overflow behaviour.
• Sampling defaults come from the app. Fields you omit are not filled in with a hosted provider’s defaults but with the loaded model’s configuration, which a preset may have set. An identical request can behave differently on two machines for that reason alone; see presets and system prompts.
• There is a richer native API alongside it. LM Studio’s own REST surface offers things the OpenAI shape has no room for — its documentation lists stateful chats, model load streaming events, prompt processing streaming events, and specifying context length in the request. If you want a progress bar during a cold load, that is where it comes from.