Designing an AI Audit Trail That Holds Up

An audit trail exists to answer a question asked months later by somebody who was not there: what did this system do for this person on this date, and on what basis. Application logs answer a different question — what went wrong — and are built for a different lifetime. The two should not be the same pipeline.

This page is engineering design guidance rather than legal advice. No logging design guarantees that any particular reviewer will be satisfied; what a good one guarantees is that the record exists, is complete, and can be shown not to have been edited.

The question a log has to answer

Write down the questions before the schema, because they determine it completely. For a system with a model in it, the recurring set is: what was decided, by which model version and which prompt version, what inputs contributed, whether a human was involved and what they did, what the user was told, and who has looked at the record since.

Notice that reconstructing the exact model output is on the list but reconstructing the user’s original text is not. That distinction is the whole design. An audit trail that stores every prompt is a second, longer-lived, more widely readable copy of your most sensitive data — built for compliance reasons and creating a compliance problem.

The fields, and the ones to leave out

{
  "seq":          8814219,
  "ts":           "2026-08-03T09:14:02.881Z",   // UTC, monotonic seq
  "event":        "decision.issued",
  "subject_ref":  "sub_9f3a...",                // pseudonymous, resolvable
  "actor":        "svc:[email protected]",
  "decision":     "route:manual-review",
  "confidence":   0.71,
  "model_id":     "vendor/model-name@2026-05-11",
  "prompt_sha":   "b91c4e...",                  // hash of the template
  "prompt_ver":   "git:7f2c9d1",
  "params":       { "temperature": 0, "max_tokens": 400 },
  "input_sha":    "3ad0f1...",                  // hash, NOT the text
  "input_ref":    "conv_4471#msg_12",           // pointer to the store
  "retrieved":    ["doc_882@v3", "doc_119@v7"], // ids and versions
  "output_sha":   "c7e2b8...",
  "human":        { "reviewed": true,
                    "by": "usr_331",
                    "action": "accepted",
                    "at": "2026-08-03T09:20:44Z" },
  "disclosed":    "ai-assisted-notice-v2",
  "prev":         "0f5b...",                    // previous entry hash
  "hash":         "a19d..."                     // this entry hash
}

The pattern to copy is input_sha plus input_ref. The hash proves that whatever the conversation store holds today is what the decision was made on; the reference says where to find it. If the subject exercises erasure, the text goes and the audit entry stays intact and still verifiable — it simply records that the input it hashed is no longer available. That is the property that lets an audit trail and a deletion obligation coexist, and it is very hard to retrofit.

Never put in it: prompt or completion text, retrieved document content, direct identifiers, credentials or tokens, or anything you would not be comfortable with the whole audit-reader population seeing. That population is larger than the engineering team, and grows.

Making it tamper-evident

Append-only is a policy; a hash chain makes it checkable. Each entry commits to its predecessor, so altering or removing any entry breaks every hash after it:

import { createHash } from "node:crypto";

const GENESIS = "0".repeat(64);

// Canonical form matters: both writer and verifier must serialise
// identically, so sort keys and fix number formatting once, centrally.
function canonical(entry) {
  return JSON.stringify(entry, Object.keys(entry).sort());
}

function seal(prevHash, entry) {
  const body = canonical({ ...entry, prev: prevHash });
  const hash = createHash("sha256")
    .update(prevHash)
    .update("\n")
    .update(body)
    .digest("hex");
  return { ...entry, prev: prevHash, hash };
}

function verify(entries) {
  let prev = GENESIS;
  for (const e of entries) {
    if (e.prev !== prev) return { ok: false, at: e.seq, why: "chain break" };
    const { hash, ...rest } = e;
    const body = canonical({ ...rest, prev });
    const expect = createHash("sha256")
      .update(prev).update("\n").update(body).digest("hex");
    if (expect !== hash) return { ok: false, at: e.seq, why: "entry altered" };
    prev = hash;
  }
  return { ok: true, head: prev };
}

A chain detects tampering; it does not prevent it, since whoever can rewrite the log can recompute the chain. Two additions close that gap cheaply. Publish the head hash somewhere the log’s operator cannot rewrite — a daily entry in a separate system, sent to a second party, or written to storage with an enforced retention lock — so any rewrite must also alter a record outside the writer’s control. And sign the head with a key the application does not hold.

Run verify on a schedule and alert on failure. An integrity check nobody runs is a comment.

Storage, clocks and retention

• Write-once storage. Object storage with an object-lock or immutability policy, or an append-only table where the application’s credentials have insert and select and nothing else. Whichever you choose, the application must not be able to delete.
• Sequence, not timestamp, for ordering. Clocks move. Use a monotonic sequence for order and a timestamp for meaning, and record both.
• UTC, always. A log spanning a daylight-saving transition in local time contains an hour that happened twice, which is exactly the sort of detail that discredits an otherwise good record.
• Retention that is a decision. Long enough for the obligation, no longer, and enforced by a lifecycle policy rather than by intention. Because the entries hold hashes and references rather than content, a long retention here is much less costly than a long retention on the conversation store.

Four mistakes that void the value

• Logging into the same system as application logs. Different lifetime, different access population, different integrity requirement. Sharing a pipeline means the audit trail inherits the weakest property of both.
• Recording the model name without the version. “It used the standard model” is not an answer to “why did it do that in March”. Pin the identifier and log the exact string the provider returned, not the one you requested.
• Not logging the human step. If a person reviewed and accepted, that is frequently the most important fact in the record, and it is the one most often left in a ticketing system nobody will correlate later.
• No access log on the audit log. Reads of a record about people are themselves events worth recording, and the first question after an internal incident is who looked.