What Is a Large Language Model? A Working Definition

Almost every definition of a large language model describes its training. That is history. If you are calling one through an API, the useful definition is about what it does when your request arrives.

The definition

A large language model is a function. It takes a sequence of tokens and returns a probability distribution over what the next token might be. That is the whole of it. Everything else — chat, reasoning, tool calling, JSON output — is built by calling that function repeatedly and doing something with the answers.

“Large” refers to the parameter count, which is the size of the learned function. “Language model” is the older term for exactly this next-token job, and it predates the current generation by decades. The models changed; the interface did not.

It returns a distribution, not an answer

This is the part most explanations skip, and it is the part that explains the behaviour people find surprising. The model does not choose a word. It produces a score for every token in its vocabulary — tens of thousands of numbers — and something outside the model picks one. That picker is the sampler, and it is controlled by parameters you set: temperature, top-p, top-k.

So “the model gave me a different answer the second time” is not the model being unreliable. It is the sampler drawing from a distribution, which is what you asked it to do. Set temperature to 0 and you get the highest-scoring token every time — near-deterministic output, from the same model, with nothing about it changed.

Four consequences you can feel

• It cannot know what it did not compute. The distribution is over the next token given everything in the context. Nothing outside the context exists to it — not a file you did not include, not a conversation from yesterday unless you resent it.
• Confidence and correctness are different numbers.The scores say how likely a token is to follow, given the training data. A fluent, confident, wrong answer is not a malfunction; it is a high-probability continuation that happens not to be true.
• Length costs time linearly. One call produces one token. A 500-token answer is 500 sequential passes, which is why output is slower than input and why streaming exists.
• The prompt is the entire program. There is no state between calls. Every request re-reads everything, which is why context length is a hard limit and why prompt caching is worth money.

What it is not

It is not a database — it has no lookup, only a distribution shaped by what it saw. It is not a reasoner in the way a solver is, though reasoning models spend extra tokens working through a problem before answering, which measurably helps on some tasks. And it is not an agent: a model that calls tools is a loop somebody else wrote, calling the same next-token function with the tool results appended.

Hold on to the function definition and most of the confusing behaviour stops being confusing. The rest of this cluster works outward from it — why generation is sequential and why a 400B model can cost like a 40B one.