Triage Prompts for Meetings, Calls and Tickets

A support ticket, a meeting transcript and a call recording are different texts with the same downstream question: what happened, how urgent is it, who owns it, and what did somebody commit to. One schema handles all three. What changes is what counts as evidence.

The prompt

Read the input and fill <schema>. The input type is given in <source_type>;
the schema does not change with it.

<schema>
{"summary": "<at most 25 words, what happened>",
 "type": "incident" | "request" | "question" | "decision" | "commitment"
         | "noise",
 "urgency": "now" | "today" | "this_week" | "none",
 "customer_impact": "none" | "one_customer" | "many" | "all",
 "owner_hint": "<a team name from <teams>, or null>",
 "actions": [{"what": "...",
              "who": "<a name or team, or null>",
              "due": "<ISO date, or null>",
              "quote": "<verbatim span that establishes this action>"}],
 "quotes": {"type": "<verbatim span that establishes the type>",
            "urgency": "<verbatim span that establishes the urgency>",
            "impact": "<verbatim span, or null>"},
 "unclear": ["<what you could not determine and why>"]}
</schema>

Rules that hold for all source types:
- Every quote must be a verbatim substring of the input.
- urgency is what the input establishes, not what the speaker feels. "This is
  urgent" is a claim; "the till has been down since 09:00" is evidence.
- customer_impact "all" requires a statement that every customer is affected.
  Do not infer it from the severity of the description.
- Never invent a due date. If a commitment has no date, "due" is null.
- If you cannot fill a field from the input, put it in "unclear". Do not
  guess.

Rules that depend on <source_type>:

  ticket    One person's account, unverified. Treat every claim as reported,
            not established. type is never "decision" - a ticket cannot
            record a decision.

  meeting   Multiple speakers. A commitment counts only when the person who
            would do it says so; "someone should look at that" is not a
            commitment and has no owner. type is "decision" only when the
            meeting explicitly agreed, not when one person proposed. Attribute
            every action to the speaker who accepted it.

  call      One customer, one agent. Only the customer's words establish
            impact and urgency. The agent's reassurances, apologies and
            promises are not evidence about the problem - though an agent
            promise IS a commitment with the agent as owner.

<source_type>{{source_type}}</source_type>
<teams>{{team_list}}</teams>
<input>
{{text}}
</input>

Why one schema for three inputs

The temptation is three prompts, because the inputs look different. Resist it, for reasons that are about your system rather than about the model:

• One schema, one set of consumers. Routing, dashboards, alerting and search all read the same shape. Three schemas means three of everything downstream and a translation layer between them.
• One eval set logic. You still need examples of each type, but the grading code, the field-level metrics and the regression harness are shared.
• Comparable metrics. “How many urgent customer-impacting incidents came in this week” is a question across all three sources. With three schemas it becomes three questions and a reconciliation.
• One place to change. Adding a field means editing one prompt and one schema, not three that will drift apart within a quarter.

The per-type rules are short precisely because the schema absorbs the differences. Three paragraphs of evidence rules is a far smaller surface than three prompts.

What counts as evidence, per source

The urgency rule that runs across all three is the one worth adopting even if you take nothing else from this page. “This is urgent” is a claim about urgency; “the till has been down since 09:00” is evidence of it. Requiring the quote to be evidence rather than assertion changes what the field means, and without it your urgency distribution measures how emphatic your customers are.

Routing belongs outside the model

The model classifies. Your code routes. Keeping the routing table out of the prompt is not a stylistic preference — it has three consequences you will feel.

# routing.py - deterministic, versioned, unit-tested, changed without
# touching the prompt or re-running the eval set.

def route(t: dict) -> tuple[str, str]:
    if t["type"] == "noise":
        return ("drop", "no ticket created")
    if t["urgency"] == "now" and t["customer_impact"] in {"many", "all"}:
        return ("page_oncall", "immediate")
    if t["type"] == "incident" and t["customer_impact"] == "one_customer":
        return ("support_queue", "1h")
    if t["type"] == "commitment" and within_days(t.get("due"), 3):
        return (owner_queue(t["owner_hint"]), "before due")
    if t["owner_hint"] is None:
        return ("triage_queue", "4h")
    return (owner_queue(t["owner_hint"]), "standard")

• Rota changes are not prompt changes. Teams reorganise every few months. A routing rule in the prompt means every reorganisation invalidates your eval set, because the expected output changed.
• Routing must be reproducible. “Why was this paged at 3am” has an exact answer when a function decided it, and a probabilistic one when a model did.
• It is testable for free. The routing function is pure. A table of inputs and expected destinations runs in milliseconds and catches the rule that shadows another.

This is the same separation argued for in prompts that apply business rules: use the model for the part that needs judgement over unstructured text, and put the deterministic part in code where it belongs.

The noise label

noise exists because most of a meeting transcript is not an item, and without the label everything becomes one. A model asked to triage a transcript with no way to say “this stretch is scheduling chat” will produce actions from small talk, and the resulting list is discarded wholesale — which loses the three real actions with it.

It carries the same diagnostic value as the escape label in classification. Track the noise rate per source type. It should be high for meetings, low for tickets, and moderate for calls. A meeting noise rate that falls means the model has started manufacturing items; a ticket noise rate that rises means something upstream is submitting junk.

The same incident, two sources

One outage, arriving twice: once as a ticket and once inside a meeting transcript. Same schema, different evidence rules, different outputs — and the difference is the point.

-- source_type: ticket ------------------------------------------------------
"Card payments have been failing in our Utrecht store since about 09:00.
 Every transaction declines. We're turning people away. This is losing us
 hundreds an hour and I've had no response to my last two emails."

{"summary": "Card payments declining at Utrecht store since 09:00; store
             turning customers away.",
 "type": "incident",
 "urgency": "now",
 "customer_impact": "one_customer",
 "owner_hint": "payments",
 "actions": [],
 "quotes": {"type": "Every transaction declines",
            "urgency": "since about 09:00",
            "impact": "Card payments have been failing in our Utrecht store"},
 "unclear": ["whether other stores are affected - the ticket only reports
              this one"]}

customer_impact is one_customer, not "many". One store reporting is one
report. The unclear entry is what makes the limitation visible rather than
lost, and it is the thing a triage engineer acts on first.

-- source_type: meeting -----------------------------------------------------
"MAARTEN: ...and the Utrecht card thing is still open.
 PRIYA:   Utrecht and Delft both, actually. Both since this morning.
 MAARTEN: Someone should check whether it's the terminal firmware.
 PRIYA:   I'll pull the terminal logs for both after this.
 MAARTEN: Great. And we should probably tell the other stores.
 JOOST:   Anyway, are we still doing the offsite in October?"

{"summary": "Card failures at Utrecht and Delft since the morning; terminal
             firmware suspected.",
 "type": "incident",
 "urgency": "now",
 "customer_impact": "many",
 "owner_hint": "payments",
 "actions": [{"what": "pull terminal logs for Utrecht and Delft",
              "who": "PRIYA", "due": null,
              "quote": "I'll pull the terminal logs for both after this"}],
 "quotes": {"type": "the Utrecht card thing is still open",
            "urgency": "Both since this morning",
            "impact": "Utrecht and Delft both, actually"},
 "unclear": ["whether the other stores were told - it was raised and nobody
              accepted it"]}

Four decisions in that meeting output are the per-source rules doing their work. “Someone should check the terminal firmware” is not an action, because nobody accepted it. “I’ll pull the terminal logs” is, and its owner is the speaker who said it. “We should probably tell the other stores” produced no action and instead an unclear entry, which is more useful than an action with no owner. And Joost’s offsite question contributed nothing — it is the noise the label exists for, in a transcript that also contained a genuine incident.

The impact field differs between the two sources for a good reason rather than an inconsistency: the meeting establishes two sites, the ticket establishes one. Both are correct about what their source says, which is what makes it safe to route on the field.

When it stops working

• Actions per meeting rises. The single most reliable signal, because the commitment rule is the first thing to erode. Check whether each action’s quote contains the owner accepting it; when it does not, restate the rule with the counter-example.
• Urgency quotes become assertions. Sample ten. If they are mostly the customer saying “urgent”, the evidence rule has stopped biting and your paging volume will follow.
• customer_impact: all appears more than rarely. It requires an explicit statement. A rise means the model is inferring scope from severity, which is exactly the inference that wakes people up unnecessarily.
• owner_hint names a team not in teams. Assert it against the list. This breaks after every reorganisation and it fails silently — the routing function sends the item to a queue nobody reads.