API Key Management for AI Applications

An inference key is a payment instrument with an API. That is the property that makes it different from most credentials you manage: the person who steals it does not need your data to profit, because the key itself buys something they want.

What makes an inference key different

• It is directly monetisable. A stolen database credential needs a buyer for the data. A stolen inference key is resold as capacity within hours, and automated scanners harvest public repositories continuously looking for exactly this.
• The loss accrues while you sleep. Usage-based billing means the damage is a function of elapsed time and rate limit, not of a single event. This is the argument for hard caps over careful monitoring.
• It is passed around more than most secrets. Notebooks, evaluation scripts, a colleague’s laptop, a CI job, an agent’s own environment. Every one of those is a copy you do not control.
• The blast radius is often the whole account. Where a provider offers one key with full access, a leak is total. Where it offers scoped keys, use them — this is the single biggest lever available.

Leak paths specific to AI applications

Generic advice — do not commit secrets, use a manager — is correct and widely published. These are the paths that only exist because there is a model in the system, and they are the ones that survive a conventional review:

• The key in the context. A key pasted into a system prompt so a tool “has access to it”. It is now one paraphrase from the transcript, and the transcript is stored.
• Traces and observability. LLM tracing tools capture full request bodies by default. If a header, a tool argument or an environment dump ends up in a span, your key is in a third-party dashboard with a broader access list than your secret manager.
• Prompt and response logs. Same problem, your own infrastructure. A logger that prints the request object on error will print the Authorization header.
• Evaluation datasets. Captured production traffic reused as an eval set, then shared with a vendor or committed to a repository. Redact before it is stored, not before it is shared — the shared copy is never the only copy.
• Client-side calls. Any key shipped to a browser or a mobile binary is public. Obfuscation and a proxy domain do not change this; only a server-side broker does.
• The model quoting its own configuration. If the agent can read files or environment variables, a document can ask it to print them. Deny the capability rather than the request.
• Notebook output cells. Committed .ipynb files keep the output of the cell that printed the environment.

Scoping beats rotation

Rotation limits the window of a leak you have detected. Scoping limits the damage of a leak you have not — and undetected is the normal case. A rotation schedule with unscoped keys is a control optimised for the wrong scenario.

Practically: one key per workload, never one shared key per company. Separate keys for production, staging, CI and each developer, so that revoking one costs one workload an outage instead of costing everyone an incident. Where the provider supports it, attach a spend limit, a rate limit, an allowed-model list and an IP or origin restriction to each key. And give every key a name that identifies its owner and purpose, because a key you cannot attribute is a key nobody will dare revoke.

That last point decides whether the other controls are usable under pressure. During an incident the question is always “can I kill this key right now without taking down something I care about?”, and the answer depends entirely on whether one key serves one workload. A single shared key makes the honest answer no, which is how organisations end up leaving a credential they know is compromised in service for another day while somebody works out what it was for.

The lifecycle, concretely

The field-allowlist detail in the “use” row matters more than it reads. Denylisting secret field names fails the moment someone adds a header you did not anticipate; allowlisting what may be logged fails safe by construction.

Detecting a leaked key

Assume detection will come from a pattern rather than from an alert saying “key leaked”. The signals worth wiring up: spend rate departing from its own baseline rather than crossing a fixed threshold; requests from unfamiliar regions or ASNs; usage of models the workload never calls; traffic at hours the workload never runs; and a request mix — very long prompts, unusual sampling parameters — that does not look like your application.

Have the response written down before you need it: revoke first and investigate second, since the cost of a wrongly revoked key is a deploy and the cost of a live stolen key compounds hourly.