Generating Training Data With an LLM: The Pipeline

The difference between a generation pipeline that produces a usable dataset and one that produces ten thousand paraphrases of the same example is not the prompt. It is whether there is a constraint at every stage that comes from outside the generator.

Five stages, five gates

A pipeline that works has the same shape whether you are building instruction data, extraction examples or classification labels. Each stage has one job and one gate; a stage without a gate is where the corpus quietly goes wrong.

Stage 1: seeds, and why they decide everything

Self-Instruct (Wang et al., 2022, arXiv 2212.10560) is the paper that made this shape standard. It starts from 175 human-written seed tasks, samples a few of them into the prompt as examples, asks the model for new instructions, then generates responses to those instructions and filters new instructions against the existing pool by ROUGE-L overlap, keeping only those below a similarity threshold. Stanford’s Alpaca built its 52,000-example set with that pipeline.

The seeds matter more than the generator does, for a reason that is easy to state and easy to forget: everything downstream is a perturbation of them. A seed set drawn from real production prompts produces a dataset whose input distribution resembles production. A seed set somebody wrote in an afternoon produces a dataset that resembles that afternoon. If you have traffic, use it — production logs are the best seed corpus available, and they are free.

Stage 2 and 3: expansion and generation

Expansion is where diversity is won or lost. The instinct is to raise temperature; the effective move is to raise the entropy of the conditioning. Give the generator a cross-product to walk: eight domains times five personas times three difficulty levels is 120 distinct prompt contexts, and each one produces a different kind of example rather than a different wording of the same one.

Evol-Instruct, from the WizardLM paper (Xu et al., 2023, arXiv 2304.12244), is the systematic version of the difficulty axis. Rather than asking for hard examples, it applies named rewriting operators to an existing instruction — add constraints, deepen, concretise, increase the reasoning steps, complicate the input — and also an in-breadth operator that mutates an instruction into a new one on a related topic. Failed evolutions are detected and discarded. The output is a difficulty gradient rather than a flat pile, which is what you want if the dataset is meant to teach anything beyond format.

At generation time, produce more than one candidate. Generating three and keeping the best is a materially different pipeline from generating one and keeping it if it looks acceptable, because the first has a selection signal and the second only has a rejection threshold.

Stage 4: the filter stack

Order these cheapest-first. Every row a cheap filter kills is a row an expensive filter does not have to look at.

• Structural checks. Parses as JSON, matches the schema, non-empty, within length bounds, no refusal boilerplate, no leaked prompt scaffolding. Free, and catches more than people expect.
• Hard verification, where it exists. Run the test, execute the code, check the arithmetic, validate against the database, confirm the extracted value appears verbatim in the source document. This is the strongest filter in the stack because it does not involve the generator’s opinion at all. Where a hard verifier exists, build the whole pipeline around it.
• Deduplication. Exact hash, then near-duplicate detection by MinHash or by embedding similarity. Generation produces near-duplicates at a rate that will surprise you, and duplicates do specific damage beyond wasting rows.
• Model scoring, last. A rubric-scored pass with a different model than the generator, used to rank rather than to judge absolutely. Cheap per row and the weakest link in the stack — it shares failure modes with the generator, so treat its score as a sorting key and not as truth. The judging page covers what it can and cannot be trusted with.

The pipeline, end to end

Concrete enough to adapt. The structure is the deliverable: attributes enumerated, k candidates, a hard verifier before any model scoring, and a stratified final sample rather than a top-k one.

import hashlib, itertools, random
from dataclasses import dataclass

DOMAINS   = ["billing", "shipping", "returns", "account", "api"]
PERSONAS  = ["terse expert", "confused first-timer", "angry", "formal"]
LEVELS    = ["single-step", "needs a lookup", "needs a policy exception"]

@dataclass
class Candidate:
    prompt: str
    response: str
    attrs: tuple           # (domain, persona, level) — kept, not discarded

def expand(seeds):
    """Cross-product, not temperature. Every combination is visited."""
    for seed, (d, p, lv) in itertools.product(
        seeds, itertools.product(DOMAINS, PERSONAS, LEVELS)
    ):
        yield seed, (d, p, lv)

def generate(seed, attrs, k=3):
    d, p, lv = attrs
    prompt = (
        f"Seed situation: {seed}\n"
        f"Rewrite it as a {p} customer asking about {d}. "
        f"Difficulty: {lv}. Output the customer message only."
    )
    return [call_model(prompt, temperature=1.0) for _ in range(k)]

# ---- filters, cheapest first -------------------------------------------

def structural_ok(c: Candidate) -> bool:
    if not (20 <= len(c.response) <= 2000):      return False
    if c.response.lower().startswith("sure,"):   return False   # assistant-ese
    if "As an AI" in c.response:                 return False
    return True

def verified(c: Candidate) -> bool:
    """Replace with a REAL verifier: a test run, a schema check, a solver.
    If you have one, it belongs here and it outranks every filter below."""
    return schema.validate(c.response)

_seen = set()
def not_duplicate(c: Candidate) -> bool:
    h = hashlib.sha256(c.response.strip().lower().encode()).hexdigest()
    if h in _seen: return False
    _seen.add(h)
    return minhash_novel(c.response, threshold=0.8)   # near-dup, not just exact

# ---- assembly -----------------------------------------------------------

def build(seeds, target_per_bucket=40):
    survivors = {}
    for seed, attrs in expand(seeds):
        for r in generate(seed, attrs):
            c = Candidate(seed, r, attrs)
            if not structural_ok(c):  continue
            if not verified(c):       continue
            if not not_duplicate(c):  continue
            survivors.setdefault(attrs, []).append(c)

    # Stratified sample: every bucket contributes, none dominates.
    out = []
    for attrs, rows in survivors.items():
        rows.sort(key=rubric_score, reverse=True)      # score used to RANK
        out.extend(rows[:target_per_bucket])
    random.shuffle(out)
    return out

The detail worth copying is that attrs is carried on every row and used at the end. Keeping the generation attributes turns the final sample from a top-k list into a design, and it gives you the histogram you need in order to notice that four of your sixty buckets produced nothing.

What a run costs per surviving example

Generating a dataset is a large batch of API calls, so the cost is fully calculable in advance. Every number below is a labelled assumption — substitute your own; the structure is what transfers.

• Assume 800 input tokens per generation call (600 shared prefix + 200 per-item).
• Assume 400 output tokens per candidate.
• Assume an input price of $0.50 per million and an output price of $1.50 per million.
• Assume k = 3 candidates per prompt.
• Assume a combined filter yield of 35% of candidates surviving.
• Assume a scoring pass over each survivor: 1,200 input tokens, 30 output tokens.

Generation, per candidate: 800 × $0.0000005 = $0.00040 of input, plus 400 × $0.0000015 = $0.00060 of output, so $0.00100 per candidate. At a 35% yield that is $0.00100 ÷ 0.35 = $0.00286 of generation per surviving example.

Scoring runs on survivors only: 1,200 × $0.0000005 + 30 × $0.0000015 = $0.000645 each. Total per surviving example: $0.0035. A 20,000-example dataset therefore costs about $70 in tokens, and a 200,000-example one about $700.

Two things fall out of that arithmetic immediately. First, the filter yield is in the denominator, so a pipeline that keeps 10% costs three and a half times one that keeps 35% for the same output — which is an argument for cheap filters early, not for a looser filter. Second, the token cost is almost never the binding constraint at these volumes; the review time for the sample you spot-check by hand is. Budget the human hours, not the dollars.

Two structural levers change the numbers rather than the arithmetic. The 600-token shared prefix is identical across every call in the run, which is exactly the shape prompt caching is for; and a generation run is latency-insensitive by definition, which is what batch pricing tiers exist to reward.