Punctuation, Casing and a Readable Transcript

Recognisers that emit unpunctuated lower-case text leave four separate jobs undone: sentence boundaries, casing, written forms for numbers and entities, and structure. The last of them is easy to do with a language model and easy to do dangerously, which is what the guard in this page is for.

What raw output is missing

Raw:
  yeah so i spoke to sarah at acme on the fourteenth of march
  and she said the invoice for twelve thousand four hundred
  pounds hadnt come through can you check that its reference
  i n v dash two two nine one

Formatted:
  Yeah, so I spoke to Sarah at Acme on 14 March, and she said
  the invoice for GBP 12,400 hadn't come through. Can you check
  that? Its reference is INV-2291.

Four distinct transformations happened there:

  1. sentence segmentation   where do full stops and question
                             marks go
  2. truecasing              sentence-initial capitals, proper
                             nouns, the pronoun I
  3. inverse text            "twelve thousand four hundred pounds"
     normalisation           -> "GBP 12,400"; "i n v dash two two
                             nine one" -> "INV-2291"
  4. structure               paragraphs, speaker labels, removal
                             of disfluencies where appropriate

Note that step 3 changed the word count. That is the property that makes verification necessary and it is also why formatting must happen after alignment, with the aligned token stream kept as the source of truth.

Also worth saying plainly: many recognisers now emit punctuation and casing natively, and if yours does, most of this page is unnecessary. Check before building. What almost none of them do well is step 3 in your domain’s conventions, and none of them do step 4.

Three ways to restore it

A hybrid is usually right for a long archive: a classifier for bulk punctuation and casing, because it is cheap and cannot rewrite, and a language model only for the passages a human will actually read.

Inverse text normalisation

ITN is the conversion from spoken form to written form — the inverse of what a text-to-speech front end does. It is where transcripts most visibly fail, because “twenty twenty six” rendered as words in a financial document reads as an error even when it is a perfect transcription.

The classic implementation is a weighted finite-state transducer: deterministic, auditable, fast, and correct for the cases you encoded. Rule-based ITN grammars ship with several open toolkits. The trade-off is exactly what you would expect — rules never hallucinate and never generalise.

• Do it per domain, not in general. “Twenty twenty” is a year in a news transcript and a score in a sports one. There is no universally correct rendering, so encode your domain’s.
• Keep it reversible. Store the spoken form alongside the written form for anything you convert. Search should match both: somebody looking for “twelve thousand four hundred” and somebody looking for “12,400” want the same result.
• Handle spelled-out sequences explicitly. “i n v dash two two nine one” is a reference number being dictated, and it is extremely common in support calls. A rule that collapses runs of single letters is worth more than any model here.
• Currencies, dates and phone numbers deserve unit tests. They are the fields people act on, and a wrong one is worse than an unformatted one.

The guard that makes an LLM safe here

The failure mode is specific and it is not loud. Ask a language model to punctuate a transcript and it will occasionally also tidy the grammar, drop a false start, merge two hesitant sentences, or replace a word it judged to be a recognition error. Every one of those changes what the record says a person said, and none of them looks wrong on the page.

The fix is mechanical: normalise both sides down to the things formatting is allowed to change, and reject the output if anything else moved.

# verify.py -- formatting must not change the words.
# Uses score() from the word error rate page.

import re
from wer import score

WORD_LEVEL_ALLOWED = {
    # Written forms your ITN is permitted to produce, mapped back
    # to the spoken form for comparison. Extend for your domain.
    "gbp": "pounds", "usd": "dollars", "eur": "euros",
    "%": "per cent", "&": "and",
}

def canonical(text: str) -> list:
    """Reduce to the sequence formatting may NOT change."""
    text = text.lower()
    for written, spoken in WORD_LEVEL_ALLOWED.items():
        text = text.replace(written, spoken)
    # digits -> a marker: ITN is allowed to turn words into digits,
    # so both sides collapse to the same placeholder.
    text = re.sub(r"\d[\d,.]*", " NUM ", text)
    text = re.sub(r"\b(?:zero|one|two|three|four|five|six|seven|eight|"
                  r"nine|ten|eleven|twelve|twenty|thirty|forty|fifty|"
                  r"hundred|thousand|million)\b", " NUM ", text)
    text = re.sub(r"[^a-z\s]", " ", text)          # punctuation: allowed
    text = re.sub(r"\bnum(?:\s+num)+\b", "num", text)  # collapse runs
    return text.split()

def formatting_is_faithful(raw: str, formatted: str, *, tolerance=0.0):
    """Returns (ok, report). tolerance is a WER allowance; 0.0 means
    the canonical word sequences must match exactly."""
    a, b = canonical(raw), canonical(formatted)
    r = score(a, b)
    ok = r.wer <= tolerance
    return ok, {
        "wer": r.wer,
        "substitutions": r.substitutions,
        "deletions": r.deletions,     # the model dropped words
        "insertions": r.insertions,   # the model added words
    }

if __name__ == "__main__":
    raw = ("yeah so i spoke to sarah at acme on the fourteenth of march "
           "and she said the invoice for twelve thousand four hundred "
           "pounds hadnt come through")
    good = ("Yeah, so I spoke to Sarah at Acme on 14 March, and she said "
            "the invoice for GBP 12,400 hadn't come through.")
    bad  = ("I spoke to Sarah at Acme in March. She said the invoice "
            "had not arrived.")

    print(formatting_is_faithful(raw, good))
    print(formatting_is_faithful(raw, bad))

Contractions need one more rule than the code above carries — the formatter turning hadnt into hadn’t is a punctuation change, and expanding it to had not is a word change. Reuse the contraction map from the WER page so both sides canonicalise identically, and decide deliberately which direction you are normalising.

Then wire it in as a hard gate. On failure, fall back to the classifier output or to the raw text — never to the model’s version with a warning logged, because nobody reads that log and the record is the product.

Paragraphs, speakers and readability

Punctuation makes sentences. Structure makes a document, and it comes from signals the text alone does not carry:

1. Break paragraphs on pauses. You have word timestamps; a gap above roughly a second within one speaker is a natural paragraph boundary and needs no model at all.
2. Always break on a speaker change, and label it. This is the single largest readability improvement available for a multi-speaker transcript, and it comes free from diarisation.
3. Decide about disfluencies explicitly, once. Removing “um”, false starts and repeated words makes a readable document and makes a bad record. A meeting summary should remove them; a legal, clinical or research transcript must not. If you might ever need both, keep the verbatim version and generate the clean one, never the reverse.
4. Cap sentence length rather than trusting the model. Punctuation restoration on conversational speech tends to produce sentences that run on, because the speech did. A hard rule that splits at a conjunction after N words reads better than a longer prompt.
5. Keep the timing on every unit you emit. A paragraph with a start time is clickable; one without is text.

Evaluating the result

Word error rate cannot score this, because the words are unchanged by construction. Score each transformation on its own terms:

• Punctuation: per-class precision and recall. Full stops, commas and question marks separately, against a hand-marked reference. Commas are the hardest and the least consequential; full stops are the ones that decide whether the document reads correctly. An aggregate F1 over all marks hides exactly the distinction you need.
• Casing: accuracy on tokens that should be capitalised, reported separately from overall accuracy. Most tokens are lower case, so overall accuracy is above 90% for a system that capitalises nothing.
• ITN: exact match per entity class. Dates, currencies, phone numbers, reference codes. These are the fields people act on, so score them as a list rather than as a rate.
• Fidelity: the pass rate of the guard above, tracked over time. A rising failure rate after a model change is the earliest signal you will get that formatting has started editing.