Incident Response for AI Features

The standard incident playbook assumes the failure is visible: errors, saturation, a red dashboard. The characteristic AI incident has none of those. Everything is green, the latency is fine, and the product is telling customers things that are not true.

Incident classes classical SRE does not have

Worth enumerating in advance, because the first ten minutes of an incident is not when you want to be inventing a taxonomy. Each of these has a different first action.

Severity when the service is up

Severity scales anchored on availability do not fit. A workable rewrite anchors on user harm and reversibility instead:

• Sev1 — the feature is producing output that could cause real harm if acted on (wrong prices, wrong medical or legal content, another tenant’s data), or it is entirely unavailable for most users, or spend is compounding without a cap. Page, and the first move is to stop the output rather than to understand it.
• Sev2 — noticeably degraded quality or availability for a substantial share of users, with no lasting harm. Page during business hours.
• Sev3 — a measurable regression on a proxy signal, a single tenant affected, or a provider degraded but absorbed by failover. Ticket.

The distinction that matters and is easy to miss: an AI incident can have persistent effects that an outage does not. Wrong answers were read, acted on, saved into records, or emailed to customers. That is why the Sev1 first move is containment, and why the remediation plan includes finding and correcting the outputs already delivered — a step with no analogue in a normal availability incident.

The first ten minutes

Four questions, in this order, all answerable from the request log if it has the right columns.

-- 1. When did it start, and is it still happening?
select date_trunc('hour', started_at) as hr, count(*),
       round(avg((not schema_valid)::int)::numeric, 4) as schema_fail,
       round(avg(refused::int)::numeric, 4)            as refusal,
       round(avg((error_type is not null)::int)::numeric, 4) as errors
from llm_request
where environment = 'prod' and feature = $1
  and started_at > now() - interval '48 hours'
group by 1 order by 1;

-- 2. What changed? Everything that could have, in one result.
select 'prompt' as kind, prompt_version as v, min(started_at) as first_seen
  from llm_request where started_at > now() - interval '48 hours' group by 2
union all
select 'model',   served_model, min(started_at)
  from llm_request where started_at > now() - interval '48 hours' group by 2
union all
select 'release', release,      min(started_at)
  from llm_request where started_at > now() - interval '48 hours' group by 2
order by first_seen desc;

-- 3. Who is affected — everyone, or one tenant / one route?
select tenant_id, count(*),
       round(avg((not schema_valid)::int)::numeric, 4) as schema_fail
from llm_request
where environment = 'prod' and started_at > now() - interval '3 hours'
group by 1 having count(*) > 50 order by schema_fail desc limit 20;

-- 4. What is it costing while we look?
select date_trunc('hour', started_at) as hr, round(sum(cost_usd), 2)
from llm_request
where environment = 'prod' and started_at > now() - interval '12 hours'
group by 1 order by 1;

Query 2 is the one to build a saved view for. In practice the answer to almost every quality incident is a row in that result whose first_seen lines up with the start of the graph in query 1, and finding it takes seconds if the query already exists and twenty minutes if it does not.

Five switches that must already exist

An incident is a bad time to write a deploy. Each of these should be a runtime configuration change that takes effect in under a minute, and each should have been exercised in staging.

• Pin the model. Force a specific snapshot id, overriding whatever routing normally decides. The mitigation for an entire incident class.
• Pin or roll back the prompt. Point the environment at the previous version. Instant if you have a registry, a deploy if you do not.
• Degrade the feature. Not off — down a level. A cheaper model, a cached answer, a template response, or the deterministic non-AI path that existed before the feature. Users tolerate a diminished feature far better than a broken one.
• Cap spend. A hard hourly ceiling enforced in code that starts rejecting or degrading when crossed. The only control that works while everyone is asleep.
• Disable a single guardrail rule. Per-rule, not the whole layer, so that an over-blocking rule can be removed without turning off safety entirely.

A useful test of readiness: pick one of those and ask how long it takes right now, in production, without a deploy. If the answer is more than a minute for any of them, that is the highest-value work on this page.

Note that all five are mitigations rather than fixes, and that is the point. The goal of the first thirty minutes is to stop the harm, not to understand the cause — an incident where you pinned the previous model and went back to bed is a good outcome even though nobody knows yet what changed. Diagnosis is much easier the next morning against a system that is no longer producing bad output, and much harder against one that is, because everybody is being interrupted.

Communicating a quality incident

Status-page language is built for outages: impacted, degraded, resolved. A quality incident needs different content, and vagueness reads as evasion.

• Say what was wrong, concretely. “Between 09:10 and 11:40 UTC, summaries generated for documents over 40 pages omitted the final section” is actionable. “Degraded quality” is not, and leaves every customer wondering whether everything they received is suspect.
• Name the blast radius by time and scope. Users can check their own outputs if they know which window and which feature to check. That is the single most useful thing you can give them.
• Say whether outputs were persisted. If bad answers were written into records, emailed, or acted on downstream, that is the important part of the notice and it needs its own remediation plan.
• Do not lead with the model. “The provider changed the model” may be true and it reads as blame-shifting. You chose the alias, and the fix is on your side.

The postmortem question that is different

Standard postmortems ask why it broke and why detection took so long. Both apply. The one to add is: what would have had to be true for us to have caught this automatically?

For quality incidents the answer is almost always a missing proxy signal — nobody was computing citation rate, or refusal rate, or schema validity by prompt version. That makes the action item concrete and cheap: add the column, add the panel, add the threshold. Over a few incidents this converges, and the class of incident that used to be found by a customer starts being found by a query.

Add a second question for the mitigation side: which switch did we wish we had? Almost every AI incident postmortem contains a moment where somebody had to deploy to change a value that should have been configuration. Those moments are the cheapest possible action items — one flag each — and they compound, because the next incident of any kind gets shorter by however long that deploy would have taken.