Build a Support Autoresponder With a Kill Switch

The question that decides whether an autoresponder can be deployed is not how good the answers are. It is: when it starts answering badly at two in the morning, how does a person who is not you stop it, and how long does that take? Build that first and the rest of the system can be improved in production. Build it last and it will never be trusted enough to reach production at all.

Design the stop first

There are four ways to stop a running system and only one of them is available at two in the morning to a support lead.

The row-in-a-database switch is the design, and the important property is that it is checked at the point of the side effect — just before the reply is sent — not at process start.

The kill switch

# control.py
import sqlite3, time

DB = sqlite3.connect("support.db", check_same_thread=False)
DB.executescript("""
CREATE TABLE IF NOT EXISTS control (
  key     TEXT PRIMARY KEY,
  value   TEXT NOT NULL,
  changed REAL NOT NULL,
  who     TEXT NOT NULL
);
INSERT OR IGNORE INTO control (key, value, changed, who)
VALUES ('mode', 'shadow', 0, 'install');
""")

MODES = ("off", "shadow", "suggest", "live")

def mode():
    row = DB.execute("SELECT value FROM control WHERE key='mode'").fetchone()
    return row[0] if row else "off"

def set_mode(new, who):
    assert new in MODES
    DB.execute("UPDATE control SET value=?, changed=?, who=? WHERE key='mode'",
               (new, time.time(), who))
    DB.commit()

def may_send():
    return mode() == "live"

Four modes rather than a boolean, because “stopped” is not one state:

• off — nothing runs. No model calls, no cost, no logs. This is the panic position.
• shadow — the bot drafts an answer and stores it, sends nothing. This is how you evaluate on real traffic without any risk, and it is where the system should live for its first week. Shadow traffic is the cheapest evaluation there is.
• suggest — the draft appears in the agent’s reply box, a human presses send. Most of the value, almost none of the risk.
• live — the bot sends, subject to the gate below.

Decide the drain semantics explicitly, because this is the part people get wrong under pressure. When somebody sets off, a reply that has already been generated but not sent should not go out — which is why may_send() is called immediately before the send and not when the job was picked up. A switch with a thirty-second tail is a switch that keeps embarrassing you after it has been flipped.

The confidence gate

Sending is allowed only when four independent conditions hold. Each one rules out a different failure, and combining them is what makes the false-send rate low enough to leave running.

def should_send(draft, hits, ticket):
    reasons = []
    if not hits or hits[0].score < 0.35:
        reasons.append("no strong source")
    if draft.get("confidence", 0) < 0.85:
        reasons.append("low self-confidence")
    if draft["answer"].strip() == NO_ANSWER:
        reasons.append("model abstained")
    if ticket["category"] in ESCALATE_ALWAYS:      # billing, legal, safety,
        reasons.append("category requires human")  # anything with a refund
    if not citations_check(draft["answer"], hits):
        reasons.append("citation not in sources")
    return (not reasons), reasons

ESCALATE_ALWAYS is the list that lets everyone else relax. Money, legal exposure, safety, account deletion, anything regulated — those never auto-send regardless of how confident anything is, and saying so in one visible constant is worth more than a paragraph of policy.

The abstention check depends on the model having a way to abstain. Instruct it to reply with one exact sentence when the documents do not answer the question, then test for that sentence. An explicit refusal token beats a confidence score because it is unambiguous in a log.

The automatic brake

A human kill switch needs a human to notice. Add a circuit breaker that notices first: a rolling window, a threshold, and an automatic downgrade from live to suggest.

# Trip on any of these, checked before each send:
#   - > 20 replies sent in the last 5 minutes      (runaway loop)
#   - > 3 replies followed within 10 minutes by a  (bad answers)
#     customer reply containing a negative signal
#   - > 15% of drafts in the last hour failed the gate
#   - spend in the last hour > 3x the trailing 24h hourly mean

def check_breaker():
    if sent_in_last(minutes=5) > 20:
        trip("rate")
    if failed_gate_ratio(hours=1) > 0.15:
        trip("quality")
    if hourly_spend() > 3 * mean_hourly_spend(hours=24):
        trip("spend")

def trip(why):
    if mode() == "live":
        set_mode("suggest", "breaker:" + why)
        alert("Autoresponder downgraded to suggest: " + why)

Downgrade to suggest rather than to off. The failure mode you are protecting against is bad answers, and suggest stops those while keeping the drafts that help the agents who are now handling everything manually. Breakers that fail into a degraded mode get left enabled; breakers that fail into an outage get disabled by the third false trip.

The escalation path

Everything the gate rejects has to go somewhere, and the somewhere determines whether the system reduces work or moves it.

1. Attach the draft to the ticket as an internal note, with the reasons it was blocked and the sources it found. The agent starts from something.
2. Route by the category the classifier produced, and let the agent correct it in one click. Those corrections are your training set for the routing rules.
3. Tell the customer nothing about the escalation. “An agent will be with you shortly” is fine; “our AI was not confident enough to answer” is an invitation to argue with the system.
4. Track time-to-first-human-reply on escalated tickets separately. If the bot handles the easy 60 per cent, the remaining queue is entirely hard tickets and the average handle time goes up — which looks like a regression to anyone reading one dashboard.

What the bot must never say

• A promise. Refunds, delivery dates, exceptions to policy. If the model can write “we’ll refund that”, somebody will hold you to it. Post-filter for a small list of commitment phrases and block the send.
• A number it was not given. Prices, balances, order totals. These come from your database into the prompt, and if they are not in the prompt the answer must not contain one. A regex for currency amounts not present in the retrieved context is a crude check that catches a real class of error.
• Anything about its own confidence. “I think” and “it appears that” in a support reply transfer the uncertainty to the customer, who cannot resolve it. If you are not confident enough to state it, escalate instead.

The four numbers to watch

Deflection rate is the number everybody reports and it is the one most easily gamed — a bot that replies to everything deflects everything, right up until the reopen rate is counted. These four together cannot be gamed by any single change:

Sample satisfaction separately for auto-answered and human-answered tickets, and resist combining them. A blended score that goes up because easy tickets are now answered instantly tells you nothing about whether the hard ones got worse, and the hard ones are where customers are lost.

One more measurement, taken once rather than continuously: read fifty auto-sent replies a month, by hand, forever. Every automated metric above measures whether the process worked. Only a person reading the output notices that the tone has drifted, that the bot has started apologising for things it did not do, or that a model update changed how it handles a category nobody thought to instrument.

Rolling it out without a bad week

1. A week in shadow on all traffic. Read fifty drafts a day by hand. You are looking for categories where it is confidently wrong, not for an average score.
2. suggest for the two or three narrowest categories where shadow was consistently right. Measure how often agents send the draft unedited — that ratio is the honest quality metric and it needs no annotation.
3. live for one category, out of hours only, with the breaker on and one named person watching. Out of hours because the alternative for the customer is waiting until morning, so the bar the bot has to clear is lower.
4. Widen one category at a time, and never widen and change the model in the same week. When quality moves you need to know which change moved it.