LLM fundamentals & architecture
What a language model actually does when it answers you, and which parts of the architecture you can feel in latency, quality and cost.
Most explanations of language models describe how they are trained. That is the wrong half for anyone who is going to use one. Training happened once, somewhere else, and you cannot change it. What you can change is the prompt, the parameters, the model and the route — and to reason about those you need a picture of what happens between your request arriving and the first token coming back.
These pages build that picture from the inference side. Where an architectural detail has no consequence you can observe, it is left out.
What Is a Large Language Model? A Working Definition
A definition built from what a model does at inference time, rather than from how it was trained.
6 min read
Autoregressive Generation: Why LLMs Write One Token at a Time
Why output is generated sequentially, and how that single constraint shapes latency, streaming and what you pay.
7 min read
Mixture of Experts: Why a 400B Model Can Cost Like a 40B One
How sparse activation separates a model's total size from the work each token costs, and what that means for price and memory.
7 min read