Extracting Speaker Turns From a Chat Transcript

“Extract who said what” sounds like one problem. It is three, because the thing that identifies a speaker is different in each of the exports you will be handed: a role, a direction relative to one phone, or an opaque identifier that means nothing without a second file.

Three formats, three kinds of speaker

It is worth being precise about the difference before writing any code, because it decides what your schema has to be able to represent.

• A support widget export identifies speakers by role. The transcript has a visitor, an agent and a system, and the same label can cover two different humans across a handoff.
• An SMS export identifies speakers by direction. There are exactly two parties, one of them is the phone’s owner, and they are never named — the record says sent or received, plus the other party’s number.
• A messaging-app export identifies speakers either by an opaque id that requires a join against a member list, or by a display name printed at the start of a line, depending on whether you got structured data or a text render.

The consequence is that speaker identity has different reliability in each. In one it is a stable key you can trust across the whole export; in another it is a label that is stable only within one conversation; in a third it is a string typed by whoever exported the file. A schema that stores all three as a name column throws away the difference, and then somebody joins on it.

Support widget: roles, not people

A live-chat transcript is generated for a human to read, so it renders participants as roles. The specific things that break extraction:

• System lines are not turns. “Chat started at 14:02”, “Alex joined the chat”, “Visitor is typing”, “Chat ended”. They look exactly like messages and they carry no speech. Classify them as events, keep them, and exclude them from anything that counts turns — they are also the best available evidence of when an agent changed.
• Agent handoff. A transcript that says “Agent” throughout may have had two agents. The join event names them; the turns do not. Carry a running current-agent value forward from the last join event and attach it to each agent turn, rather than trusting the label.
• Bots that present as agents. The first several turns are frequently an automated triage flow using the same styling. If the export has a flag for it, keep it; if it does not, the handoff event is again the marker.
• Canned responses and quick-reply buttons. A visitor turn that is a button click is not free text, and treating it as an utterance will distort anything you extract from visitor language.

SMS: a direction flag and a phone number

SMS archives are the odd one out. A typical XML backup stores each message with the correspondent’s address and a type code distinguishing received from sent, so the speaker is defined entirely relative to the device that produced the file. Nobody is named. If you do not record whose phone the export came from, half the turns are attributed to an unknown person forever.

Phone numbers then bring their own identity problem. The same correspondent appears as a national-format number in one thread, with a country code in another, and with punctuation in a third, because the number is stored as it was dialled. Normalise to E.164 before grouping — and note that normalisation needs a default region, which you can only get from the device, so it is another reason to record the export’s provenance. Group MMS threads carry several addresses on one message and the sender is one of them, which is a different shape again.

One field in these archives deserves a warning: the readable date string that sits alongside the numeric timestamp is generated by the exporting app in the device’s locale and timezone, and it is a rendering rather than a source of truth. Parse the numeric field. Resolving mixed timestamp formats is a whole problem of its own.

Messaging apps: an id or a line prefix

Structured workspace exports give each message an author field containing an opaque user id and nothing human-readable, plus a separate members file mapping ids to profiles. That indirection is a feature — the id is stable when somebody changes their display name — but three things reliably go wrong. A profile can have an empty display name, in which case you need a documented fallback chain rather than an empty string. Messages posted by integrations have no user id at all and instead carry a bot or app identifier, sometimes with a per-message username override. And membership files often omit people who left the workspace, so a share of ids resolve to nothing and must render as the raw id rather than as null.

Text renders of a group chat are the other shape: a timestamp in brackets, a display name, a colon, the message. Here the name is whatever the exporting person had that contact saved as, so the same conversation exported from two phones produces two different “identities” for the same human. Treat a text-render name as a label local to that file and never as a key, and expect system lines with no name and no colon at all — encryption notices, group membership changes, media placeholders where a file was not included in the export.

One normalised turn

{
  "source": { "format": "sms_xml", "owner_party_id": "party:device_owner",
              "default_region": "GB", "exported_at": "2026-07-14T08:00:00Z" },
  "parties": [
    { "party_id": "party:device_owner", "kind": "human",
      "label": "Device owner", "identifiers": [], "is_export_owner": true },
    { "party_id": "party:+441632960041", "kind": "human",
      "label": "+44 1632 960041", "identifiers": ["tel:+441632960041"] }
  ],
  "turns": [
    { "index": 0, "party_id": "party:+441632960041", "role": "correspondent",
      "kind": "message", "text": "can you resend the invoice",
      "ts_text": "12/07/2026 09:41", "ts_instant": null }
  ],
  "events": [
    { "index": 4, "kind": "agent_joined", "text": "Alex joined the chat",
      "party_id": "party:agent_alex" }
  ]
}

The load-bearing decisions are that parties is a separate table from turns, that party_id is synthesised by you rather than taken from the source, and that events are not turns. The party table is where the three formats reconcile: a role, a phone number and a workspace id all become one party record with the source identifier kept in identifiers. It is also where you can legitimately merge two parties later, having discovered that the agent in one transcript and the user id in another are the same person, without rewriting a million turns.

Keep kind on the party as well as role on the turn. Whether a speaker is a human, a bot or the system is a fact about the party; whether they were the visitor or the agent in this particular conversation is a fact about the turn, and one person can be both across a dataset.

The parse bug everybody writes

For any text-render format, the obvious parser splits each line on the first colon and calls the left side the speaker. It works on the sample and fails on real data, because message bodies contain colons and newlines. A message reading “three things: first, the invoice” wrapped onto a second line produces a speaker called “three things”. A pasted URL produces a speaker called https. A pasted stack trace produces dozens.

Anchor on the timestamp instead. In every text render worth parsing, a new message begins with a timestamp in a predictable position, and a line that does not start with one is a continuation of the message above it. Match the timestamp pattern at the start of the line; if it matches, start a new turn and take the speaker from between the timestamp and the first colon after it; if it does not, append the whole line to the current turn including its own colons.

Two refinements make it robust. Strip Unicode format characters before matching, because some exports embed invisible directional marks around bracketed timestamps and a regex anchored at the line start will not match past one. And validate afterwards: the set of distinct speaker labels in a conversation should be small, so a parse producing forty speakers in a two-person chat has failed, and that check costs one aggregate query. Only once the turns are right is there any point asking a model about their content — the segmentation is not a model problem, and using one to do it converts a deterministic bug into an intermittent one.