Enums vs Free Text: Constraining the Answer Space

An enum in a prompt is a request. An enum in a strictly enforced schema is a change to the vocabulary. The second one eliminates a whole class of bug for free — and then hands you a subtler one, which is the model picking the wrong member of a set it was never able to leave.

Not a hint: a change to what is reachable

Ask for one of paid, unpaid, partially_paid in the prompt and you will spend the next year adding cases to a normaliser: "Paid", "PAID", "paid.", "Paid in full", "The invoice is paid". Put the same three in an enum under a strict schema and the sampler mask makes every one of those unreachable. There is nothing to normalise, because there is nothing else the tokens can spell.

That is the entire mechanical benefit and it is worth stating plainly, because it is often oversold as an accuracy benefit. Closing the set removes surface variation completely. It does not make the model better at deciding, and if the right answer is not in the set, the constraint guarantees a wrong answer rather than an odd one.

The second-order benefit is bigger and less discussed: a closed set makes your output evaluable. Free text can only be assessed by a human or another model. Five labels give you a confusion matrix, per-class precision and recall, and the ability to see that one specific pair of labels accounts for most of your errors — which is usually a labelling problem you can fix in an afternoon.

Six ways to design the labels badly

• Opaque names. TYPE_A, status_3, CAT_07. The model reads label names as text; that is its only information about what they mean. Names carried over from a legacy database throw away the one signal you were given for free. Use refund_request, and map to CAT_07 in code.
• Overlapping meanings. billing_issue and payment_problem in one set split probability mass between two spellings of one concept. Whichever wins, your accuracy on both drops and the confusion matrix shows a single hot off-diagonal cell. Merge them or write descriptions that draw a hard line.
• Different levels of abstraction. A set containing technical, billing and password_reset_link_expired asks the model to choose between a category and an instance. Keep one taxonomy level per field; use a second field for the sub-type.
• Too many at once. Long enums are legal — OpenAI documents a cap on total enum values across a schema and a stricter character budget once a single enum passes a couple of hundred members — but a flat 200-way choice is a hard task for the model as well as for the constraint compiler. Split into a coarse field and a fine field conditioned on it, or do two calls.
• Tokenisation-hostile labels. Members that differ only in a rare suffix or that tokenise into unusual fragments give the decoder little to work with. Ordinary lowercase words separated by underscores are safe.
• No definitions. The enum members are the answer set; the field description is where you say what each one means. One line per label, especially for the two that get confused.

The escape hatch is not optional

Under enforcement the model must emit a member of the set. If your document is a supplier statement and your labels only cover customer invoices, it will pick the closest one and your validator will accept it, because it is valid. There is no error to catch anywhere.

Always include a named out — other, not_stated, unclear — with a description that makes using it legitimate rather than a failure. Then monitor its rate. A rising other rate is the cheapest drift detector you will ever build: it tells you the world changed before any accuracy metric does, and it needs no labels.

Consider two outs rather than one where the distinction matters: not_stated (the document is silent) and unclear (the document says something, and it does not fit). Those need different responses from you, and one bucket cannot tell you which you have.

When free text is still right

Enums are wrong when the set is genuinely open — a product name, a person, a quoted span — and wrong when you do not yet know the set. Do not invent one prematurely; a discovery pass with free text over a few hundred documents, clustered by hand, is how you find out what the labels should be. Freezing the taxonomy before you have looked at the data is a much more expensive mistake than a week of normalising strings.

The middle position is worth knowing: keep the field free text, and add a separate closed field for the part you do know. “Category is one of five; subcategory_free is whatever the document says.” You get the evaluable field and the discovery signal from one call.

Normalising free text, if you must

When you cannot constrain — an endpoint without schema support, a legacy pipeline — the normaliser should be strict and should count what it could not handle, because the count is the argument for fixing it:

from collections import Counter

CANON = {
    "paid": "paid", "paid in full": "paid", "settled": "paid",
    "unpaid": "unpaid", "outstanding": "unpaid", "due": "unpaid",
    "partially paid": "partially_paid", "part paid": "partially_paid",
}
unmatched = Counter()

def normalise(raw: str) -> str | None:
    key = " ".join(raw.strip().lower().rstrip(".").split())
    hit = CANON.get(key)
    if hit is None:
        unmatched[key] += 1      # this counter is the whole point
    return hit

Resist the urge to add fuzzy matching. A near-miss that silently maps "not paid" to paid is worse than an unmatched value you can see in the counter, and edit-distance matching on short labels does exactly that.