A Checklist Before You Ship Anything AI

Every item here is phrased so that the answer is a fact somebody can state, not an intention somebody can hold. “We handle errors” passes no check on this list; “here is the code path taken when the provider returns a 429, and here is the last time it ran” does.

How to use it

Not all thirty apply to every feature, and treating the list as a gate for an internal tool is how checklists get abandoned. Two rules make it usable. First, scale it to the blast radius: an internal summarisation tool needs the cost and operations sections and can skip most of disclosure; anything that touches money, customer communication or personal data needs all of it.

Second, and more importantly: several of these cannot be retrofitted cheaply, and those are marked (now or never). They are the items whose absence means a question is permanently unanswerable rather than merely unanswered — evidence that has to be recorded as it happens, and design decisions that are expensive to reverse once traffic depends on them. Everything else can genuinely be added in month two.

Cost — 6

• 1. Cost per successful outcome is written down, not cost per call, including retries, fallbacks, guardrail calls and any classifier or verifier in the path.
• 2. The p99 request has been costed, not just the median. Token counts have a long tail and your caps are sized by it.
• 3. A hard spend cap exists and has been tested by actually hitting it in a non-production environment. An untested cap is a configuration value, not a control.
• 4. Per-user and per-tenant limits exist, so one actor cannot consume the budget — the denial-of-wallet case, which does not require malice to occur.
• 5. Spend is attributable to this feature in your own records. (now or never) — a metadata tag that was not applied at call time cannot be recovered from a monthly invoice.
• 6. Somebody is named to look at the cost weekly for the first month, and the alert threshold is a rate rather than a total, so a step change is visible before the month ends.

Correctness — 6

• 7. An evaluation set exists, built from real inputs, and it ran before the last change to the prompt.
• 8. The acceptable failure rate is a number somebody agreed to, and the eval is large enough to distinguish it from a worse one.
• 9. Output is validated against your own schema on every path, including the fallback path — which is the one that has never run in anger.
• 10. Business invariants are checked in code: totals match line items, identifiers resolve, values fall in permitted ranges. Cheap, deterministic, and the highest-yield check on this list.
• 11. The prompt has a version, an owner and a gate, and the version is recorded on every request. (now or never) — without it, no future quality question is investigable.
• 12. A regression suite runs in CI on prompt and model changes, and it has failed at least once, so you know it can.

Safety and security — 6

• 13. Untrusted content is not treated as instruction. Anything retrieved, uploaded or received from a third party is marked as data, and the feature has been tested with an injection payload in that content.
• 14. The model cannot take an irreversible action directly. Code or a person authorises; the model proposes. (now or never) — this is an architecture decision, not a check to bolt on.
• 15. Tools are capability-scoped to the acting user’s permissions, evaluated server-side, per call.
• 16. Rendered output cannot exfiltrate. Markdown images and links to arbitrary hosts are stripped or proxied at the render layer.
• 17. Private data, untrusted content and an outbound channel do not co-exist in one context — the lethal trifecta check, applied to each call site rather than to the feature as a whole.
• 18. Logs are safe to keep. Redaction happens at capture, not in a later cleanup, and somebody has read a sample of what is actually stored.

Operations — 7

• 19. There is a degraded path, and it has been exercised in production by deliberately disabling the model during a quiet hour.
• 20. A kill switch exists that needs no deploy and no engineer, is evaluated without calling the failing dependency, and produces the degraded feature rather than an error.
• 21. Timeouts are set from a deadline that is shorter than the shortest timeout in front of you, including the platform’s own.
• 22. Retries cannot double-charge. An idempotency key or a claim-before-call path exists wherever a retry could duplicate a billable operation.
• 23. Every request logs model id, prompt version, token counts, cost, latency and validation outcome. (now or never) — this is the single most consequential item on the list, because it is what makes every future investigation possible.
• 24. A canary set runs on a schedule and its results are retained. (now or never) — a time series cannot be reconstructed after the fact, and without one you cannot date a behaviour change.
• 25. Alerts are on rates, not on individual failures, and one of them fires on the share of traffic served by a fallback — the failure that produces no errors.

Disclosure — 5

• 26. The user can tell that output is model-generated, where knowing would change what they do with it. This is about accuracy of expectation rather than about a badge on everything.
• 27. There is a way to report a bad output that captures the request id, so a complaint arrives attached to something you can look at rather than as a description.
• 28. Data handling is stated and true: what is sent to which provider, whether it may be retained, and whether it may be used for training — checked against the terms actually in force for your account, not the marketing page.
• 29. Someone has confirmed the regulatory position for the domain, if the feature touches health, finance, employment, credit, education or legal matters. The answer may be “nothing applies”; the check is that it was asked before launch rather than after a complaint.
• 30. Support knows it shipped, knows what it does, and knows the one sentence to say when a customer asks why the output was wrong. The most-skipped item on the list and among the cheapest.

If the list is too long for the feature in front of you, the reduced version is the six items marked (now or never): spend attribution, prompt versioning, per-request logging, the canary series, the proposal-not-action boundary, and — implied by all of them — deciding these before traffic exists rather than after. Everything else on this page can be added in month two at roughly the same cost as adding it today. Those six cannot, because each one is either a record that has to be written as events happen or a boundary that gets harder to draw with every call site that assumes its absence.