Synthetic Data: What It's Good For and What It Isn't

“Synthetic data” names three techniques with almost nothing in common except that a computer produced the rows. They fail differently, they carry different privacy properties, and the advice that applies to one is frequently wrong about the others.

Three unrelated things, one word

Before any argument about whether synthetic data works, establish which kind is on the table. The distinction is not academic — it decides whether the output can contain a real person’s record, whether it can be more diverse than its input, and whether the failure mode is statistical or editorial.

Most of this cluster is about the third family, because that is what people mean in 2026 when they say the phrase. But the first family is the one that quietly does most of the useful work in a real codebase — see test fixtures — and the second is the one with a regulator attached, covered in privacy-safe synthetic data.

The axis that sorts every use case

There is one question that predicts, better than anything else, whether synthetic data will work for a given job: how closely does the generated joint distribution have to match the real one?

Not the marginals — anyone can match marginals. The joint: the way fields co-vary, the shape of the tail, the frequency of the awkward combinations. A generator is a compression of whatever it learned, and compression loses the rare cases first. So a job that only needs plausible-looking rows is nearly free, a job that needs the tail is hard, and a job whose entire answer depends on the tail is one synthetic data cannot do.

That single question also explains why the same technique gets glowing and damning write-ups in the same month. Both are true. They are different jobs.

The use cases, sorted

From least to most distribution-sensitive. The further down you go, the more the generated data has to earn its place with verification rather than volume.

• Fixtures, demos and load tests. Distribution barely matters; only shape, volume and referential integrity do. Rule-based generation is correct here and a language model is overkill.
• Adversarial and robustness probes. You want the unusual inputs, deliberately. Being unrepresentative is the point, so the usual objection does not apply at all. See generating the inputs that break your feature.
• Format and style tuning. Teaching a model to answer in a house structure, a schema or a tone. The content of the examples matters much less than their form, and form is exactly what generation is good at.
• Instruction data for capabilities the teacher already has. Distillation, essentially. Bounded above by the teacher, which is a real ceiling but a known one.
• Filling a known gap in a real corpus. You have real data, you know which slice is thin, and you generate to that slice. Works because the real data anchors everything else.
• Eval sets. Now the distribution matters a great deal, because an eval measures whatever distribution it is drawn from. A generated eval measures how well the model handles the generator’s idea of the problem. Usable as a smoke test, not as the thing you make release decisions on.
• Replacing a real training set entirely. The regime the collapse literature is about. Sometimes fine, sometimes not, and the conditions are specific.
• Statistical inference about real people. Estimating a rate, a correlation or a treatment effect from synthetic rows. The answer you get is a property of the generator, not of the world. This is the one to refuse.

What generation cannot do

One principle covers most of the disappointments: sampling from a model cannot add information the model does not have. Ten thousand generated rows from a generator fitted to two hundred real ones contain the information in two hundred real ones, arranged more smoothly. The confidence intervals you compute on the ten thousand are wrong, and they are wrong in the flattering direction.

Three consequences worth holding on to:

• Volume is not evidence. A hundred thousand generated examples is a hundred thousand draws from one prior. It looks like a large dataset to every tool that counts rows.
• The tail goes first. Rare combinations are rare in the sample too, and after a filtering pass that rewards plausibility they are rarer still. Whatever you were hoping to learn about the unusual cases is the part most reliably lost.
• Errors are systematic, not random. A generator does not make independent mistakes; it makes the same mistake across thousands of rows because they all came from one set of weights. Averaging does not help, and neither does more data.

Three questions before you generate

Can the answer be verified without a human? If a unit test, a schema, a solver, a compiler or a database can say whether a generated example is correct, generation is on very solid ground: the filter is external to the generator, so the pipeline is not marking its own homework. If verification requires judgement, you are relying on a model to grade a model and you inherit every bias that comes with that.

Does anything depend on the tail? If the value of the dataset is in the rare cases — fraud, faults, edge-case handling — then generation from a fitted model is the wrong instrument, because the tail is exactly what fitting smoothed away. Generate the rare cases from rules or from domain knowledge instead, where you control the frequency directly.

What real data stays in the mix? Almost every safe recipe in this cluster has real data somewhere in it — real prompts, real seeds, real held-out evaluation, or real records the generator was fitted to. A pipeline with no real data at any stage has no anchor to reality and no way of noticing it has drifted.