Pattern: Gradual Autonomy

Nobody should ship an autonomous feature on day one, and nobody should keep a human confirming every action forever. The useful design is a ladder with written criteria for climbing it, decided before there is any pressure to climb.

Four levels, not a slider

Autonomy is discrete in practice because each step changes who is accountable, and accountability does not come in fractions.

The jump that matters is three to four, because that is where the default flips from “nothing happens unless someone acts” to “something happens unless someone intervenes”. Every argument about autonomy is really an argument about that flip, and treating it as a smooth slider disguises the discontinuity.

Level one is skipped almost universally and is the highest-return level in the list. It provides real-traffic accuracy data with zero product risk, and it is available the moment the feature computes anything at all — including while the interface is still being designed.

Promotion criteria you write in advance

Promotion decisions made in the moment are made under pressure — the confirmations are tedious, the team is convinced it works, the approval rate looks high. Criteria written before the data exists are the only ones that constrain anything. Four, and all four must hold.

• Volume. A stated number of decisions observed at the current level. Not a duration — a count, because a week of low traffic is not evidence. The count should be large enough that the error rate you care about would have appeared several times; the arithmetic for that is a sample size question with a real answer.
• Accuracy against a decided threshold. The agreement rate between the system and the human, measured at the current level, above a number chosen in advance. Choosing the number afterwards is choosing the number that the data supports.
• A bounded worst case. Written down: the most expensive single thing this can do wrong at the next level, and the control that bounds it. If the answer is unbounded, promotion is not available at any accuracy.
• A working reversal. Somebody has actually undone one of these actions, in production, and timed it. Not a design for a reversal — an executed one. This criterion catches the features whose rollback path exists only in a document.

Two refinements make the criteria harder to game. Promote by segment rather than globally — one customer type, one action type, one confidence band — so the first level-four traffic is a slice you chose rather than everything at once. And separate the person who proposes promotion from the person who approves it, for the same reason deployments have that separation.

The path back down

Almost every treatment of autonomy levels describes going up. The omission matters, because a system that can only be promoted has a ratchet in it: the level only ever rises, and the response to a problem becomes a patch rather than a step back.

Demotion needs three things to be real. A trigger, defined as a metric crossing a threshold rather than as somebody’s judgement — a post-hoc rejection rate above a bound, a reversal rate above a bound, a validation-failure rate above a bound, or a single incident of a named severity. A mechanism that is a configuration change rather than a deploy, because a demotion that requires a release will not happen during the twenty minutes when it should. And an owner with the authority to pull it who is not the person who built the feature.

The interaction with feature flags for models and prompts is direct: the autonomy level is a flag value, demotion is setting it to a lower number, and the kill switch is the same control at its bottom setting. Building it that way means the emergency path is the path you use routinely, which is the only kind of emergency path that works.

One further rule. A demotion should be reversible only by satisfying the promotion criteria again from the start. Otherwise the response to a demotion is to wait a day and put it back, and the ladder becomes decorative.

What each level must record

Each level generates the evidence that the next level needs. If the instrumentation is not in place, the level is not doing its job, and you will arrive at the promotion decision with nothing to evaluate.

level 1 · observe   what the system proposed
                    what the human actually did
                    -> agreement rate, and WHERE it disagrees

level 2 · suggest   proposal shown / accepted / edited / ignored
                    -> take-up rate, and the edit diff, which names the
                       failure modes better than any rubric you would write

level 3 · confirm   proposal, verdict, correction, seconds to decide
                    -> approval rate per confidence band, which is the
                       curve the level-4 threshold is read off

level 4 · act       action, outcome, reversals, post-hoc sample verdicts
                    -> reversal rate and sampled error rate; without the
                       sample, the error rate on the majority of traffic
                       is simply unknown

The edit diff at level two is the underrated one. When a person accepts a suggestion but changes two words, those two words are a precise, unprompted description of what the system got wrong, produced by an expert at no cost. Most implementations record acceptance as a boolean and discard the diff.

Three ways this goes wrong

• Confirmation becomes a formality. Level three with a high approval rate degrades into clicking, and the human protection you are relying on quietly stops existing while the metrics still show it. The counters are the ones from the review queue: make the evidence visible, localise the uncertainty, and treat a very high approval rate as a signal to route fewer items rather than as proof the system is safe.
• Promotion by exhaustion. The most common path to level four is not a criterion being met; it is that the confirmations were annoying and somebody turned them off. This is exactly what written criteria prevent, and it is why they must exist before the annoyance does.
• Uniform autonomy across non-uniform actions. A feature that does five things does not have one autonomy level. The cheap reversible action can sit at four while the irreversible one stays at three indefinitely, and collapsing them to a single setting means the riskiest action is governed by the average. Autonomy is a property of an action type, not of a feature.