The NIST Generative AI Profile: What It Adds to the RMF

NIST AI 600-1, the Artificial Intelligence Risk Management Framework Generative Artificial Intelligence Profile, was released on 26 July 2024. It is a cross-sectoral profile of the base AI RMF, which means it adds no new functions and no new structure: it names risks that are either unique to generative AI or sharply exacerbated by it, and then lists suggested actions keyed to existing RMF subcategories.

What the profile is, structurally

A profile in RMF terms is a use-case-specific or sector-specific application of the framework. This one is cross-sectoral: it applies wherever generative AI is developed or deployed rather than to one industry. Its first half enumerates risks. Its second half is a long table of suggested actions, each tagged with the RMF subcategory it serves — identifiers of the form GV-1.3-001 or MS-2.6-004 — and with the risks it addresses. That tagging is the mechanically useful part, because it means you can start from a risk you care about and land on the governance artefact that addresses it, or start from a subcategory you already implement and find the generative-specific actions attached to it.

The profile does not replace the base framework and cannot be implemented on its own; it assumes you have the four functions from the base AI RMF in place. It is available as NIST AI 600-1, and it was published alongside a companion on secure software development practices for generative AI, NIST SP 800-218A, which handles the supply-chain and code side of the same problem.

The twelve risks it names

The base RMF talks about trustworthiness characteristics in the abstract. The profile’s contribution is to name concrete risks, listed alphabetically in the document rather than by severity:

• CBRN information or capabilities — lowered barriers to chemical, biological, radiological or nuclear weapons information, including the aggregation of individually public material.
• Confabulation — confidently stated false content.
• Dangerous, violent or hateful content — generation at a scale and specificity that was previously effortful.
• Data privacy — leakage of training data, inference of sensitive attributes, and user-supplied data ending up somewhere unexpected.
• Environmental impacts — the energy and water cost of training and serving.
• Harmful bias or homogenisation — the second half of that pairing being the profile’s own addition: outputs converging on a narrow range, and that narrowing feeding back into future training data.
• Human-AI configuration — misuse, over-reliance, automation bias, anthropomorphisation, and mismatched expectations of what the system is.
• Information integrity — synthetic content at scale degrading the ability to tell what is authentic.
• Information security — prompt injection, model extraction, data poisoning, and the expanded attack surface of a system that executes text.
• Intellectual property — training data and outputs, and the exposure that runs in both directions.
• Obscene, degrading or abusive content — including non-consensual intimate imagery and material depicting child sexual abuse.
• Value chain and component integration — the opacity that comes from building on components you did not make.

Several of these have no counterpart in the base framework at all. CBRN uplift, environmental impact and value-chain opacity are not derivable from “valid and reliable” or “secure and resilient”; they had to be added.

Confabulation, and why the word was chosen

The profile uses “confabulation” where most writing uses “hallucination”, and the choice is deliberate: hallucination implies perceiving something that is not there, whereas the model is producing a fluent continuation that happens to be false. The terminology matters less than what the profile then asks for, which is that the risk be treated as a measurement problem rather than a prompt problem. Suggested actions cluster around the Measure function: define what a factual error means for your use case, sample outputs, track the rate over time, and record the conditions under which it rises.

That framing has a practical consequence. If your only control for confabulation is a system prompt telling the model not to make things up, you have a Manage-function treatment with no Measure-function evidence, and the profile is fairly direct that this is not a control. The mechanism underneath is worth understanding independently — see what happens when a false statement about a person is generated, where the risk stops being a quality issue and becomes a legal one.

Value chain and component integration

This is the risk most likely to describe your organisation exactly. If you call a hosted model, you do not know its training data, you did not choose its safety tuning, you cannot reproduce its behaviour after the provider updates it, and your incident response depends on somebody else’s status page. The profile’s suggested actions here are about documentation and contract: record what you know about each component, record what you do not, and put the parts you need into agreements rather than assumptions.

The specific documentation gap the profile keeps returning to is provenance — of training data, of the model version actually serving your traffic, and of the outputs your own system then stores and reuses. A silent model version change on the provider’s side invalidates every evaluation you ran, and if you did not record which version produced which output you cannot tell afterwards whether a regression is theirs or yours.

Using it without drowning in it

The suggested-action table runs to hundreds of entries and reading it end to end is the wrong first move. The order that works is to start from the risk list, mark the ones that are live for your system, and only then filter the actions by those risk tags. A retrieval assistant over internal documents has a serious confabulation, data privacy and information security profile, a real human-AI configuration problem, and essentially no CBRN exposure. A model provider fine-tuning open weights has almost the inverse.

Two habits make the difference between a profile that is used and one that is filed. The first is writing down, for each risk you exclude, why it does not apply — that record is what an auditor or a customer actually asks for, and it is cheap to produce while the reasoning is fresh. The second is keeping the subcategory identifiers in your own documentation. When a customer questionnaire or an internal review asks how you handle a generative risk, an answer that names MG-3.1 and points at a real artefact is a different quality of answer from a paragraph of assurance, and it survives the person who wrote it leaving.