Secrets in a Python AI Project

An LLM API key is a bearer credential attached to a balance. Anyone holding it can spend your money, and the usual result is not a spectacular breach but a bill. Choose the storage that prevents the leak you are actually exposed to.

The five ways a key actually leaks

Ranked by how often each is the one that happened, rather than by how serious it sounds.

Four of those five are not solved by encryption at rest. They are solved by the key never being in a place that gets copied — which is what makes the ordering below the way it is.

The ladder, in order

1. Process environment, injected by the platform. The production answer everywhere. Your hosting provider, container orchestrator or systemd unit puts the value in the environment; there is no file, and no path by which the repository can contain it.
2. A .env file listed in .gitignore. The development answer. Convenient, shareable in structure via a committed .env.example with empty values, and safe exactly as long as the ignore rule was written first.
3. The OS keyring. Better than .env on a laptop, because the secret is not a plaintext file that a backup tool, a sync client or a shared screen can pick up. Detailed below.
4. A secret manager. Vault, or a cloud provider’s own. Worth it when you need rotation, audit and per-service access rather than just storage.
5. Never: a constant in source, an argument on the command line. The first is committed. The second appears in your shell history and in the process table, where any user on the machine can read it with ps.

# config.py — one place that reads the environment, and it fails loudly
import os
from dataclasses import dataclass


@dataclass(frozen=True)
class Settings:
    base_url: str
    api_key: str
    model: str

    def __repr__(self) -> str:                 # so a traceback cannot print it
        return f"Settings(base_url={self.base_url!r}, api_key='***', model={self.model!r})"


def load_settings() -> Settings:
    missing = [name for name in ("LLM_BASE_URL", "LLM_API_KEY")
               if not os.environ.get(name)]
    if missing:
        raise RuntimeError(
            f"missing environment variables: {', '.join(missing)}. "
            "Copy .env.example to .env and fill it in."
        )
    return Settings(
        base_url=os.environ["LLM_BASE_URL"].rstrip("/"),
        api_key=os.environ["LLM_API_KEY"],
        model=os.environ.get("LLM_MODEL", "openai/gpt-4o-mini"),
    )

The custom __repr__ is small and pays for itself the first time an exception is raised with the settings object in a local variable. Frameworks and error reporters print locals; a dataclass’s default repr prints the key.

Using the OS keyring

pip install keyring. It talks to the macOS Keychain, the Windows Credential Locker, and Secret Service (GNOME Keyring, KWallet) on Linux. The secret is stored by the operating system and released to your process, so no file in your project ever contains it.

# secrets_store.py
import getpass
import os

import keyring

SERVICE = "multigrid-llm"
ACCOUNT = "default"


def store_key_interactively() -> None:
    """Run once, by a human, in a terminal. Never in a script or a notebook."""
    value = getpass.getpass("API key (input hidden): ")
    keyring.set_password(SERVICE, ACCOUNT, value)
    print("stored in the OS keyring")


def get_api_key() -> str:
    """Environment wins, so CI and containers do not need a keyring."""
    from_env = os.environ.get("LLM_API_KEY")
    if from_env:
        return from_env
    try:
        value = keyring.get_password(SERVICE, ACCOUNT)
    except keyring.errors.KeyringError as exc:
        raise RuntimeError(
            f"no LLM_API_KEY set and the keyring is unavailable ({exc})"
        ) from exc
    if not value:
        raise RuntimeError("no key stored; run store_key_interactively() first")
    return value

getpass.getpass rather than input, so the key is not echoed to the terminal and does not end up in a screen recording, a shared session or a terminal scrollback that gets pasted somewhere.

Environment before keyring, always. A headless server or a CI runner has no keyring backend, and code that reaches for one first fails there in a confusing way — the exception mentions D-Bus, not your missing configuration.

Notebooks, which are their own hazard

A .ipynb file stores cell outputs. Anything printed is saved, committed, and rendered by every git host with a preview.

• Never print(os.environ), and never %env with no argument. Both dump every variable, and the output cell keeps it.
• Never paste a key into a cell, even to delete it a minute later. Jupyter checkpoints save automatically, and the checkpoint directory is easy to commit by accident.
• Beware tracebacks. An exception raised inside a call that took a key as an argument prints that argument in the frame. This is the leak people do not anticipate, and it is why the settings object above has a custom repr.
• Strip outputs before committing. nbstripout --install adds a git filter that removes outputs on the way into the index while leaving your working copy intact. It also makes notebook diffs readable, which is worth it on its own.

Notebooks for LLM work without the usual mess covers the rest of the notebook discipline, including getting the code out into a module where it can be tested.

Stopping the commit before it happens

Detection beats discipline. A pre-commit hook takes five minutes to set up and catches the case where somebody clones the repository and does not read the README.

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: detect-private-key
      - id: check-added-large-files
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets
        args: ["--baseline", ".secrets.baseline"]

pip install pre-commit
pre-commit install                      # installs the git hook

# create the baseline of known, reviewed findings
detect-secrets scan > .secrets.baseline

Add the server-side half too, if your host offers it — GitHub’s push protection blocks pushes containing recognised key formats, and it works for the person who has not installed your hooks.

If a key has already been committed

The order matters, and the first step is the one people leave until last.

1. Revoke the key. First. Now. Before deciding how it happened, before rewriting history, before telling anyone. A key in a git object is a key in every clone, every fork, every CI cache and every mirror; rewriting history does not reach any of those. Revocation is the only action that actually stops the spend.
2. Issue a replacement and deploy it. Through the environment, not the file that leaked.
3. Check the usage log for the exposure window. From first commit to revocation. Unexpected models, unusual hours or a volume spike tell you whether it was used, which changes what you have to disclose.
4. Only now, clean the history — git filter-repo or the equivalent — and force-push. This is housekeeping so the secret is not re-leaked by a later fork, not remediation.
5. Add the guard that would have caught it, and lower the spending limit on the key while you are there. Budget controls and denial of wallet are about capping the damage a leaked key can do before you notice it.