Supervised, Unsupervised and Reinforcement Learning

Textbooks introduce the learning paradigms in the order they were invented. That order is useless when you are standing in front of a real problem, because the paradigm is not something you choose — it is decided for you by the data you can get hold of.

The question that decides it

Before anything else: for how many examples do you know the right answer, and what did knowing it cost? Every paradigm below is an answer to a different version of that question, and the cost part is not a footnote. Labels are the expensive input in most projects, more expensive than compute and much more expensive than engineering time.

Supervised: you have pairs

You have inputs x and, for each one, the correct output y. The model is fitted to reproduce y from x, and the whole apparatus of loss functions, held-out splits and precision and recall exists to serve this case.

It is the paradigm with the best tooling and the worst economics. If a radiologist takes four minutes per image, ten thousand labelled images is roughly 670 hours of specialist time before you have trained anything. That number, not the model architecture, is what usually decides whether a supervised project happens.

Unsupervised: you only have inputs

No labels at all. What you can still do is describe the structure of the data: group it (k-means, hierarchical clustering), compress it into fewer dimensions (PCA, autoencoders), estimate where it is dense and therefore what counts as an outlier, or find items that co-occur.

The catch is that there is no ground truth, so there is no honest accuracy number. “Are these the right clusters?” has no answer inside the data; it has an answer only once a human looks at them, or once the clusters are used for something downstream that does have a metric. Treat every unsupervised result as a hypothesis, not a finding.

Self-supervised: the label is inside the input

This is the one that changed everything, and it is not a compromise between the two above — it is supervised learning where the labels were free because they were already in the data.

Take a document and hide the next word. The input is everything before; the label is the word you hid. You did not pay an annotator, and you did not have to decide what the task was. Now count what one document yields: a 1,000-token article is not one training example, it is roughly 1,000 of them, one per position, because every prefix is an input and every following token is its label. A corpus of a hundred billion tokens is a hundred billion supervised examples that cost nothing to label.

That is the entire economic engine behind large language models, and the same trick appears elsewhere: masked patches in images, contrastive pairs built by augmenting the same photo twice, and the training of embedding models. The loss used is ordinary cross-entropy. Nothing about the mathematics is new; the labelling bill is what changed.

Reinforcement: you only have a score

No correct output exists, or nobody can state one. What you have is an environment, a sequence of actions, and eventually a number saying how well that went. The learning problem is credit assignment: the reward arrives at move 60 and you need to know which of the earlier moves deserved it.

Reinforcement learning is the most data-hungry paradigm by a wide margin, which is why its famous successes are in simulators — games, robotics in simulation — where episodes are cheap and repeatable. Its appearance in language models is narrower than the headlines suggest: RLHF uses a reward model trained on human preference comparisons, and DPO removes the reinforcement step entirely by optimising the preference data directly. The reward in both cases is a learned stand-in for a human judgement, not a score the world handed back.

Credit assignment is worth seeing concretely, because it is the whole difficulty. A recommendation engine shows a user twelve items over a week and the user subscribes on Friday. Which impression earned it? Supervised learning cannot be asked this question — there is no label per impression — and the reinforcement formulation answers it by propagating the eventual reward backwards through the sequence with a discount factor. That machinery is powerful and it is also why RL is data-hungry: the signal per decision is a fraction of one number, rather than one label per example. Before reaching for it, check whether logged decisions and outcomes can be reframed as a supervised problem. They usually can, and the supervised version is an order of magnitude less work.

Choosing, in one table

That last row is the one worth internalising, because it is the modern default. Very few teams now start from raw data with a randomly initialised model. They start from someone else’s self-supervised pretraining and spend their small label budget on the last few centimetres.