Podcast Production With AI

Every downstream artefact — show notes, chapters, clips, the written version, the search index — is a transformation of the transcript. So transcript accuracy is not one task among several. It is the input to all of them, and an error there is an error everywhere.

Transcription first, everything downstream

The correct order of work is transcript, correction, then everything else, and the correction step is the one people skip because the raw output looks fine when you read a paragraph of it.

1. Record with separate tracks per speaker if you possibly can.
2. Transcribe with speaker labels and word-level timings. Word-level rather than segment-level, because chapters and clip extraction both need to find a moment rather than a paragraph.
3. Correct the transcript against a prepared vocabulary list. Ten minutes, described below.
4. Generate show notes, chapters and clip candidates from the corrected transcript, each with the timestamp it came from.
5. Check every extracted quotation against the audio, not the text.

Separate tracks matter more than any software choice. Overlapping speech on one track is where error rates rise most sharply, and no amount of post-processing recovers two people talking at once as well as two microphones do.

What a 5% error rate costs, in numbers

Word error rate is the standard measure: substitutions plus insertions plus deletions, divided by the number of words in the reference. A 5 per cent WER sounds excellent. Here is what it means on one episode.

Assumptions (substitute your own):
  episode length          45 minutes
  conversational speech   150 words per minute
  word error rate         5%

  total words   = 45 × 150            = 6,750
  errors        = 6,750 × 0.05        = 338
  per minute    = 338 / 45            ≈ 7.5
  interval      = 60 / 7.5            = one error every 8 seconds

Now the part that matters. Errors are not uniform. They fall on rare
tokens — proper nouns, product names, technical terms, numbers — because
those are precisely the words a model has the least evidence for.

  suppose names and jargon are 4% of words        = 270 words
  and they attract 50% of all errors              = 169 errors
  error rate ON THOSE WORDS  = 169 / 270          ≈ 63%

Every other word is then wrong about 2.6% of the time.

The clustering assumption is illustrative rather than measured, and you can check it on your own episode in ten minutes by listing the names and counting how many came through correctly. But the direction is not in doubt and it follows from the mechanism: a rare word has less acoustic and linguistic evidence behind it, so it is the one that gets replaced by a common word that sounds similar.

The consequence is that headline WER is close to useless as a quality bar for you. The errors are concentrated in exactly the words your audience uses to search, the words a guest will notice, and the words a clip will be built around. Measure the rate on names specifically.

It also determines the correction strategy. Reading a 6,750-word transcript against the audio at, say, three times real time is 135 minutes per episode — nobody sustains that. Fixing 270 known terms with a find-and-replace list is ten minutes and removes half the errors that matter.

The vocabulary list is the fix

Build it once, extend it every episode. It has three uses: as a custom vocabulary hint where your transcription service accepts one, as a post-correction map, and as an audit list.

# vocab.txt — the correct term, then the mishearings you have seen
Kubernetes :: Cubernetes, Coobernetties, Kuber netties
PostgreSQL :: Post gray SQL, Postgres QL
Aoife :: Eefa, Eva, Efa
Xhosa :: Kosa, Cosa
p95 :: P 95, p ninety five
BYOK :: B Y O K, byock

# fix_transcript.py — Python 3.9+, standard library only.
# Usage: python fix_transcript.py vocab.txt transcript.txt > fixed.txt
# Reports what it changed on stderr so you can audit the corrections.

import re, sys

pairs = []
for line in open(sys.argv[1], encoding="utf-8"):
    line = line.strip()
    if not line or line.startswith("#") or "::" not in line:
        continue
    right, wrongs = (part.strip() for part in line.split("::", 1))
    for wrong in (w.strip() for w in wrongs.split(",")):
        if wrong:
            pairs.append((wrong, right))

# Longest first, so "Post gray SQL" is tried before "SQL".
pairs.sort(key=lambda p: -len(p[0]))

text = open(sys.argv[2], encoding="utf-8").read()
counts = {}
for wrong, right in pairs:
    pattern = re.compile(r"\b" + re.escape(wrong) + r"\b", re.IGNORECASE)
    text, n = pattern.subn(right, text)
    if n:
        counts[wrong + " -> " + right] = n

sys.stdout.write(text)
for k, n in sorted(counts.items(), key=lambda kv: -kv[1]):
    print(f"{n:4d}  {k}", file=sys.stderr)

Two rules keep this safe. Sort by length descending, or a short entry swallows the long one it is contained in. And print every substitution to standard error: a silent corrector will eventually replace a word that was correct, and you want the list of what it did in front of you.

Before recording, add every guest name, company, place and technical term you expect to come up. Ten minutes of preparation removes the errors that would otherwise take an hour to find.

Show notes with a grounding requirement

Show notes fail in a specific way: the summary describes a version of the conversation that is slightly more coherent, slightly more conclusive, and occasionally attributes a position to the wrong person. The control is to require a timestamp and a quoted span for every point, so a claim without evidence cannot be produced.

From the transcript below, produce show notes.

Return JSON only:
{
  "summary": "3 sentences on what the episode covers",
  "points": [
    {
      "point": "one sentence, in your own words",
      "speaker": "exact speaker label from the transcript",
      "timestamp": "MM:SS from the transcript",
      "quote": "verbatim, 8 to 30 words, copied exactly"
    }
  ],
  "mentioned": [
    {"thing": "book, paper, product or person named",
     "timestamp": "MM:SS", "quote": "the verbatim mention"}
  ]
}

Rules:
- Every point must have a quote that appears verbatim in the transcript.
- Do not attribute a point to a speaker unless the transcript shows that
  speaker saying it.
- Do not add context, correct anybody, or resolve a disagreement that
  was not resolved in the conversation.
- If a claim was hedged, the point must carry the hedge.

TRANSCRIPT (with speaker labels and timestamps)
...

Run the quotes through the same substring verifier used in the research workflow — the transcript is the source document and the check is identical. The last rule matters most. Summaries systematically sharpen: a guest who said something was probably true appears in the notes as having asserted it, and that is how a podcast acquires a reputation for misrepresenting its guests.

The mentioned array is worth its own field because it is what listeners actually come to the notes for — the book, the paper, the tool — and because those are proper nouns, which is exactly the category the transcript got wrong. Check every one against the audio.

Chapters: the bar is five seconds

A chapter marker is a navigation promise. The quality bar is positional: the marker should land within about five seconds of the actual topic change, and it should land before it rather than after.

• Early is much better than late. A marker five seconds early costs the listener a sentence of the previous topic. A marker five seconds late means they have already missed the start of the thing they navigated to, which is the failure that makes people stop using chapters.
• Titles are descriptive, not teasing.“Why the migration failed” is navigation. “A surprising turn” is not.
• Three to eight per hour. More than that and the list stops being scannable, which was the entire point.
• A chapter boundary is a topic change, not a pause. Silence-based segmentation puts markers where people breathed.

Check by clicking every marker and listening for five seconds. On a 45-minute episode with six chapters that is under a minute of work, and it is the only test that measures the thing the feature is for.

What stays human

• Anything a guest said that they might not want published. Editing for consent is a judgement about a relationship. A model cannot see the wince.
• Corrections. If someone stated something false on air, the fix is a correction in the notes and, where it matters, a note in the audio. Never a silent edit of the transcript — that produces a record that disagrees with the recording.
• Clip selection. Extracting candidate moments is a fine machine task; deciding which one represents the conversation fairly out of context is not, and a clip is the version most people will see.
• Guest names and affiliations. Check against something the guest wrote, not against the transcript. This is the error people remember.

All four belong to whoever holds the single human gate before publication, and the same transcript-first ordering applies unchanged to video production, where the transcript is also the artefact every downstream saving runs through.