The NAIC Model Bulletin on Insurers' Use of AI

The National Association of Insurance Commissioners cannot regulate anybody. It is a standard-setting body of state regulators, and its model bulletin on artificial intelligence has exactly as much force in your state as your state’s commissioner has chosen to give it. Knowing that is the difference between a compliance plan and a reading exercise.

A bulletin is not a model law

The NAIC adopted the Model Bulletin on the Use of Artificial Intelligence Systems by Insurers on 4 December 2023, at its Fall National Meeting. The NAIC produces several kinds of instrument and they are not interchangeable. A model act is draft legislation for a state legislature to enact. A model regulation is draft text for a department to adopt through its rulemaking process. A model bulletin is a template for a commissioner to issue as a statement of the department’s expectations under existing law — primarily each state’s unfair trade practices act and its market conduct authority.

That last point is the one that decides whether the bulletin bites. Because it rests on existing unfair-trade-practice and market-conduct statutes rather than on new authority, a commissioner can issue it without a legislature and without notice-and-comment rulemaking. It also means the bulletin does not create a new cause of action; it tells insurers how the department will read powers it already had.

The definitions that set the scope

Section 2 of the bulletin carries the definitions, and they are where the scope is really decided. The bulletin defines an Artificial Intelligence System broadly enough to cover machine-learning models, generative AI and simpler predictive models, and it is not limited to systems that make a decision without a human. A model that ranks, scores, or recommends and is then actioned by an adjuster is in scope.

The definition doing the most work is Adverse Consumer Outcome: a decision by the insurer that is subject to insurance regulation and that adversely affects the consumer in a way the relevant law does not permit or that is unfair, inequitable or unfairly discriminatory. The bulletin scales its expectations to the risk of an adverse consumer outcome, which is the hinge for proportionality. A generative model drafting internal summaries that never reach a coverage decision sits at one end; a model scoring claims for referral to a special investigation unit sits at the other, because a referral changes what happens to a claimant.

This matters practically because insurers routinely inventory AI by department rather than by consumer effect. The bulletin’s frame asks a different question: for each system, what is the worst thing it can do to a policyholder, and how many steps of human judgement stand between the model output and that outcome? An inventory built the first way does not answer an examiner asking the second.

The AIS Program: what has to exist on paper

Section 3 sets the substantive expectation: an insurer using AI systems should maintain a written Artificial Intelligence Systems Program designed to mitigate the risk of adverse consumer outcomes, and should be able to produce it. The word doing the work is written. Practices that exist but are not documented cannot be produced on request, and the bulletin’s enforcement mechanism is entirely about production.

The model text organises the programme in three parts, and each maps to artefacts a governance team can actually build.

• Governance. Named accountability at a senior level, board or committee oversight, defined roles for development, validation and approval, policies covering the full lifecycle, and documented decisions about which systems are in scope. The bulletin references the NAIC’s 2020 AI Principles — fair and ethical, accountable, compliant, transparent, and safe, secure and robust — as the values the programme implements.
• Risk management and internal controls. An inventory of AI systems, pre-deployment validation and testing appropriate to the risk, ongoing performance monitoring including drift, data governance covering the provenance and suitability of training data, documented model updates and version control, and an audit function independent of the people who built the model.
• Third-party systems and data. Due diligence on vendors, contractual terms giving the insurer the information and the audit rights it needs, and a process for the insurer to satisfy itself about models it did not build.

There is a strong family resemblance between this list and the model-risk-management discipline banks have run for years under the Federal Reserve and OCC supervisory guidance, treated in the page on effective challenge, and to the function structure of the NIST AI Risk Management Framework. An insurer that already runs one of those has most of the artefacts; what it usually lacks is the mapping from its own documents to the bulletin’s headings, which is what makes an examination painful.

Third-party models and data

The bulletin does not let an insurer transfer responsibility to a vendor, and this is the provision most likely to require contract renegotiation rather than a policy document. The insurer is expected to have done diligence on third-party AI systems and data, and to have contract terms that let it get at what it needs — which in practice means audit or information rights, notice of material model changes, and enough documentation to answer a regulator about a model it cannot inspect.

“Enough documentation” is where the friction is real. A foundation-model provider will not disclose training data, and an insurer cannot make that a condition of purchase. What is negotiable is narrower and more useful: which model version served a given request, when versions change, what evaluation the insurer ran itself on its own book, and whether the insurer’s data is used for training. The general form of these terms is set out in the page on audit rights in AI vendor contracts.

What Section 4 lets an examiner ask for

Section 4 is the part to read before an examination. It states that the department may request information about an insurer’s AIS Program during an investigation or market conduct examination, and it gives an illustrative list. In practice it converts the written programme into a document request:

• The AIS Program document itself, and the governance policies under it.
• The inventory of AI systems and their mapped consumer effects.
• Documentation of the data used — sources, suitability, limitations, and any external consumer data or information sources.
• Model development and validation records, testing for unfair discrimination, and monitoring results over time.
• Vendor contracts and diligence files for third-party models and data.
• Records for individual consumer decisions, sufficient to explain what the system contributed to a specific outcome.

The last one deserves emphasis because it is the one systems are least often built for. Aggregate fairness testing answers a question about the portfolio. A market conduct examination frequently starts from a complaint by one policyholder, and the question is what happened to that person. If your logging retains scores but not the inputs and the model version, you can describe your process and cannot reconstruct the decision — and reconstruction is what Section 4 anticipates. The quantitative end of this, where a regulator prescribes the test rather than asking to see yours, is Colorado’s approach, covered in the Division of Insurance testing rule.