Skip to content
Multigrid

LLM fundamentals & architecture

What a language model actually does when it answers you, and which parts of the architecture you can feel in latency, quality and cost.

Most explanations of language models describe how they are trained. That is the wrong half for anyone who is going to use one. Training happened once, somewhere else, and you cannot change it. What you can change is the prompt, the parameters, the model and the route — and to reason about those you need a picture of what happens between your request arriving and the first token coming back.

These pages build that picture from the inference side. Where an architectural detail has no consequence you can observe, it is left out.

What Is a Large Language Model? A Working Definition

A definition built from what a model does at inference time, rather than from how it was trained.

6 min read

Autoregressive Generation: Why LLMs Write One Token at a Time

Why output is generated sequentially, and how that single constraint shapes latency, streaming and what you pay.

7 min read

Mixture of Experts: Why a 400B Model Can Cost Like a 40B One

How sparse activation separates a model's total size from the work each token costs, and what that means for price and memory.

7 min read

Attention Explained Without the Math

What attention actually computes, described as a lookup you already understand, with the linear algebra kept as a footnote.

6 min read

How a Transformer Actually Works, One Layer at a Time

One token followed from the tokenizer to the logits, with the real tensor shapes of a published 7B model at every step.

6 min read

Self-Attention vs Cross-Attention: What Each Is For

The same operation with a different source for keys and values, and why that one difference decides which architectures use which.

5 min read

Why Almost Every Modern LLM Is Decoder-Only

Three architectures competed for the same job. The one that won had denser training signal, one uniform interface and a cache that survives between turns.

6 min read

Model Parameters: What 7B, 70B and 400B Actually Buy You

What a parameter count does and does not predict, with the memory arithmetic it does predict worked out exactly.

6 min read

Positional Encoding: How a Model Knows Word Order

Why attention is blind to order by construction, and how sinusoidal, learned, RoPE and ALiBi each fix it with different consequences.

7 min read

Layer Normalisation, Residuals and Why Deep Nets Train at All

The two pieces that make a 100-layer stack trainable, and the pre-norm versus post-norm decision that used to decide whether training survived.

6 min read

The Softmax Bottleneck and What It Limits

A rank argument about the final layer: why the set of distributions a model can express is bounded by its hidden width, and what that does and does not stop.

6 min read

Temperature, Top-p and Top-k: A Practical Sampling Guide

What each sampling parameter does to the distribution, why stacking them hides which one is binding, and a script that builds the comparison table on your own prompt.

7 min read

Greedy Decoding vs Beam Search vs Sampling

Three different objectives dressed as three algorithms, why the one that maximises probability produces the worst prose, and which tasks each suits.

6 min read

What Logprobs Are and What You Can Actually Do With Them

The numbers behind the token the model chose, four things they are genuinely good for, and the ways they mislead.

6 min read

Base Models vs Instruct Models vs Chat Models

Three distinct artefacts that share a name, how each behaves when you prompt it, and the chat template that decides whether you get sense or noise.

6 min read

Scaling Laws: What Chinchilla Said and What Changed Since

The compute-optimal result, the arithmetic that quietly replaced it once models were served rather than published, and what the laws never covered.

7 min read

Emergent Abilities: Real Phenomenon or Measurement Artefact?

The claim that capabilities appear suddenly with scale, the metric-choice critique that undercut it, and what survives either way.

6 min read

Why Models Have a Knowledge Cutoff (And What Leaks Past It)

A cutoff is a property of a data pipeline, not a wall in the model — four mechanisms make it fuzzy, and a probe you can run on facts only you can date.

6 min read

Model Weights, Checkpoints and What “Open” Really Means

What is actually inside a released checkpoint, which formats carry which risks, and the licence clauses that decide whether you can ship it.

6 min read

Encoder Models Are Not Dead: Where BERT-Style Still Wins

Two structural differences — one forward pass instead of a decode loop, and token offsets instead of a string — that make a 110M encoder the right tool for several common jobs.

6 min read

Other topics