Fine-Tuning for Format vs Fine-Tuning for Knowledge
5 min read · updated August 3, 2026
Two teams run the same procedure with the same tooling on the same base model. One reports that fine-tuning transformed their product; the other that it did nothing. Usually neither team is wrong, and the difference is which of these two things they were trying to teach.
The asymmetry
Fine-tuning is very good at making a model do a thing it can already do — reliably, by default, without being asked. It is poor at making a model know a thing it did not know. The first is selection among existing behaviours; the second is writing new information into weights, and gradient descent on a small dataset is a bad instrument for writing.
Once you hold that distinction, the confusing reports resolve. Format tuning succeeds because the base model already knows how to emit JSON, already knows how to be terse, already knows six registers of English. You are raising the prior on one of them, not teaching it.
Why format is easy
Concretely: a base model given a schema and asked for conforming JSON will produce it most of the time, and fail in a predictable handful of ways — a markdown fence around the object, a trailing explanation, a field omitted when its value is unknown, an enum answered in prose. Every one of those failures is the model choosing a plausible continuation that happens to violate your contract.
Fine-tuning on a few hundred examples moves probability mass away from those continuations. It does not teach the model what JSON is; it makes the conforming continuation the overwhelmingly likely one. That is exactly what a small number of gradient steps on a consistent target distribution does well, and it is why the same intervention works for tone, length, structure, refusal style and label vocabulary.
The second reason it is easy: format signal is present in every example. A thousand training examples give a thousand demonstrations of the format. Contrast that with a fact, which appears once.
Why knowledge is hard
A fact that appears once in your dataset contributes to one example’s gradient, competing against the stylistic regularities present in all of them. To fit it, you have to train long enough or hard enough that the model memorises the example — and by then you have overfitted everything else.
The failure is worse than mere ineffectiveness, and this is the part worth carrying away. Training on statements the model cannot ground in anything teaches it a meta-lesson: that producing confident specific assertions is the desired behaviour. It generalises that lesson to questions where it has no information at all. So the model does not just fail to learn your facts; it gets more willing to invent facts in general.
The evidence
- Zhou et al., 2023 — LIMA (arXiv 2305.11206). Their Superficial Alignment Hypothesis states that a model’s knowledge and capabilities are learned almost entirely during pretraining, and alignment tuning mainly teaches it which subdistribution of formats to use when interacting with users. They support it by instruction-tuning a strong base on only 1,000 curated examples. This is the format side of the asymmetry, stated as a hypothesis by people who tested it.
- Gekhman et al., 2024 (arXiv 2405.05904). Studied what happens when fine-tuning examples contain knowledge the model did not already have. Reported that such examples are fitted slowly relative to ones consistent with existing knowledge, and that as they are fitted the model’s tendency to hallucinate increases. This is the mechanism in the section above, measured.
- Ovadia et al., 2023 (arXiv 2312.05934). Compared fine-tuning against retrieval for knowledge injection and found retrieval ahead on the knowledge-intensive tasks tested.
- Gudibande et al., 2023 (arXiv 2305.15717). Trained models to imitate a much stronger proprietary model and found the students closed the gap on style and human preference while gaining far less on factuality and reasoning. Different setting, same asymmetry: imitation transfers form more readily than substance.
A test that tells them apart
Twenty minutes, before you build anything. Take twenty failing examples. For each, hand-write the ideal output. Then put the ideal output next to the input and ask what the model would have needed.
| What the ideal output contains | Description |
|---|---|
| Only rearranged input | Format problem. The information was all present; the model shaped it wrongly. Fine-tuning will work, and a few hundred examples is likely enough. |
| A judgement the model got wrong | Behaviour problem. Which fields matter, when to refuse, how much detail. Fine-tuning works, and you need more examples because you are teaching a decision boundary. |
| A fact that appears nowhere in the input | Knowledge problem. Retrieval, a tool call, or a bigger context. Fine-tuning is the wrong instrument and will make hallucination worse. |
| A multi-step derivation the model cannot do | Capability problem. A stronger model, or decomposition into steps. A low-rank adapter does not add reasoning capacity the base lacks. |
The distribution of those twenty across the four rows is your plan. If most land in the third row, no amount of dataset work will help and the fortnight is better spent on retrieval.
What to do with this
- Fine-tune for form, retrieve for facts, and let each do its job. The combination is not a compromise, it is the correct architecture.
- Treat a long style section in your system prompt as a backlog item. It is a fine-tuning dataset specification that you are currently paying for on every request.
- Measure format compliance separately from quality. Valid-output rate is a hard number and it is the one format tuning moves. Bundling it into a subjective quality score hides the win.
- If you fine-tune on any factual content, add a hallucination check to the evaluation suite, on questions outside the training domain. That is where the damage shows, and a task-only evaluation will never see it.