Model Cards and System Cards: Reading Them Critically

The model card began as a proposal for disaggregated, honest reporting of a model’s limits. It has drifted into a launch document. The useful way to read one is to hold it against what the original proposal asked for and notice what is missing.

What the genre was for

Model cards were proposed by Mitchell and colleagues in 2019, in a paper whose central idea was disaggregated evaluation: report performance not as a single number but broken down across the groups and conditions where it might differ, so that a user can tell whether the headline figure applies to their case. The companion proposal for data — Gebru and colleagues’ datasheets for datasets — asked for the provenance, motivation, composition and known limitations of a training corpus.

The original sections are worth remembering because they are the checklist: intended use, out-of-scope use, factors the performance varies across, metrics with the reason for choosing them, evaluation data, training data, quantitative results broken down by group, and ethical considerations. That is a document written to be useful to someone deciding whether the model is safe for their purpose.

What it became

Three related documents now circulate under overlapping names, and distinguishing them saves confusion:

The drift is not simply cynicism. A card that reports a weakness in detail hands a map to anyone looking for one, and a card that quantifies a disparity creates a discoverable document in future litigation. Those are genuine incentives against disclosure and they are why voluntary transparency tends to converge on what is safe to say.

The deeper problem is that one document is being asked to serve audiences with incompatible needs. A regulator wants evidence of process. A researcher wants reproducibility detail. A deploying engineer wants operating limits. A journalist wants a summary. A competitor is reading it too. A document written to satisfy all five satisfies none of them well, and the compromise consistently favours the audiences least able to check it. Where cards are genuinely useful, it is usually because someone published a separate evaluation appendix for the audience that needed one — which is the practical fix, and it costs nothing but a willingness to be specific.

Reading for the absences

• Training data. The standard formulation is some variant of “publicly available and licensed data.” That phrase carries no information and its ubiquity is the disclosure: it tells you the composition is considered commercially or legally sensitive, which is a fact about the industry rather than about the model.
• Evaluation configuration. A benchmark number without the prompt, the number of samples, the decoding settings and whether it is single-attempt or best-of-many is not reproducible and not comparable to anyone else’s number for the same benchmark. The presence of this detail is the strongest single signal of a card written to inform.
• Contamination checks. Whether the evaluation sets were checked for overlap with training data, and how. Rarely present, and it is the check that determines whether the number means anything.
• Disaggregated results. The entire point of the original proposal, and the section most often reduced to a paragraph of prose about a commitment to fairness.
• Both refusal numbers. Safety sections often report how often the model refused genuinely harmful requests, and rarely report how often it refused benign ones. The second number is where the cost of the safety training lands on users, and reporting one without the other makes the trade-off invisible.
• Red-teaming provenance. Who did it, with what access, over how long, and whether they could publish. “External experts” without those four facts constrains nothing.
• Version identity. Which exact checkpoint the numbers describe, and whether that checkpoint is the one behind the name you call. Where a name points at successive checkpoints over time, the card documents a model that may no longer be the one answering you, and nothing in the document itself will say so.

Verifiable, unverifiable, and the audit gap

Sort the claims. Benchmark results are verifiable if — and only if — the configuration is given and the model version is still served. Context length, modalities and API behaviour are verifiable by anyone with an account. Refusal behaviour is verifiable statistically. Everything about provenance, process and internal decision-making is an assertion, and no amount of detail in the document changes that.

This is exactly where transparency obligations in regulation run into trouble, and it is worth being precise about why. A rule can require publication of a summary of training data. It cannot, by itself, make that summary accurate. Accuracy requires either an audit right with access to the underlying pipeline, or a penalty for misstatement severe enough to price the risk — and the second only works if someone can detect the misstatement. A disclosure regime without one of those two is a formatting requirement. Whether current regimes have either is a question worth asking of the actual text.

The card describes weights; you consume an endpoint

A gap that catches production teams. A card describes a set of parameters evaluated under the developer’s conditions. What reaches your application is an endpoint, and between the two sit quantisation, a serving stack, possibly a system prompt you did not write, possibly a moderation layer, a context window that may be configured shorter than the maximum, and a routing decision. A model can match its card exactly and still behave differently for you.