California’s AB 1008: Applying the CCPA to AI-Generated Output

The argument AB 1008 was written to end went like this: model weights are floating-point numbers, numbers are not personal information, so the CCPA has nothing to say about a trained model. The amendment does not say that argument is wrong on the physics. It says the format is irrelevant.

What actually changed

Assembly Bill 1008, signed on 28 September 2024 and operative from 1 January 2025, amended the California Consumer Privacy Act’s definition of personal information at Civil Code section 1798.140. The definition has always included a clause about the formats in which personal information can exist. AB 1008 extended that list to make clear that personal information can exist in abstract digital formats, including compressed or encrypted files, metadata, or artificial intelligence systems that are capable of outputting personal information. The bill record is at California Legislative Information.

The operative words are “capable of outputting”. The test the amendment sets is behavioural, not structural. Nobody has to demonstrate that a particular name is stored in a particular weight; it is enough that the system can produce personal information as output. That is a much easier threshold to cross, and it is the reason the amendment matters.

It removed an argument, it did not add a right

This is the part most coverage gets backwards. AB 1008 created no new consumer right, no new business obligation, no new penalty and no new exemption. Every right discussed below existed in the CCPA before it. What the amendment did was close off a definitional escape route, so that the pre-existing rights have to be answered rather than argued out of scope.

Reading it that way also sets the correct limit on it. The amendment does not say a model is personal information; it says a model can be, depending on what it can output. A model that has demonstrably never seen personal data, or one whose outputs are constrained such that personal information cannot emerge, is not brought into scope by this provision. The question is factual and the burden of being able to answer it sits with the business.

Note also what it means for training data specifically. The same amendment makes explicit that personal information in a training corpus is personal information — which was always the better reading, but which is now not worth disputing. That connects directly to the disclosure a developer must publish about whether its datasets contain personal information as defined in section 1798.140, under AB 2013. A developer that answers “yes” there has, in effect, answered part of the AB 1008 question about itself in public.

The rights that are now hard

• Deletion, section 1798.105. A consumer may request deletion of personal information the business collected from them. If the model is within scope, the request logically reaches it, and there is no reliable technique for removing one individual’s contribution from a trained model. Machine unlearning is an active research area, not a shipping capability. Retraining is a real answer and an expensive one. What the statute expects here is genuinely unresolved.
• Correction, section 1798.106. A consumer may request correction of inaccurate personal information. A model that confidently generates a false statement about a named person is the obvious case, and the obvious fix — output filtering rather than model modification — is a correction of the system’s behaviour rather than of stored data. Whether that satisfies the right is untested. The same question arises under the GDPR and is covered in the rectification page.
• Access and disclosure, sections 1798.100 and 1798.110. A consumer may ask what personal information the business has collected. Answering for a model raises the same extraction problem in reverse: you cannot enumerate what a model “has” without probing it, and probing it to find out is itself processing.
• Opt-out of sale or sharing, section 1798.120. Relevant where model access is provided to third parties on terms that meet the CCPA’s broad definitions of selling or sharing for cross-context behavioural advertising.
• Limits on sensitive personal information, section 1798.121. If the model is capable of outputting sensitive categories, the right to limit use and disclosure is engaged.

The limits and exceptions that still apply

Three things narrow the practical effect, and a compliance position built without them will be more alarming than the law requires.

First, the CCPA’s deletion right is not absolute: section 1798.105(d) lists exceptions permitting a business to retain personal information for enumerated purposes, including security, error correction, legal compliance and certain internal uses reasonably aligned with consumer expectations. Whether any of them covers retaining a trained model is an argument, not a certainty, but it is the argument that will be run.

Second, the regulations issued under the CCPA contemplate that a business may not always be able to comply in the most direct way, and include a concept of disproportionate effort in responding to certain requests. That does not excuse a response; it can shape what an adequate response is.

Third, the CCPA applies to businesses meeting its thresholds and contains exemptions — for deidentified and aggregate consumer information, and for certain data covered by other regimes. Genuinely deidentified data is outside the definition of personal information entirely, which makes the deidentification standard in section 1798.140, and whether a model trained on such data can nonetheless output identifying content, the hinge of the whole analysis. The parallel EU question is in the anonymisation page.

Separately, the California Privacy Protection Agency’s regulations on automated decision-making technology are a distinct instrument with their own scope, access and opt-out requirements and their own compliance timeline. They are not what AB 1008 did, and they are covered in the ADMT regulations page.

What to do about it

The useful work is upstream of the request, because every hard question above becomes easier if less personal information reaches the model in the first place. In rough order of leverage: know which training and fine-tuning corpora contain personal information, at the corpus level, because you cannot answer any request without that; keep a record of which model versions were trained on which corpora, so a deletion request can at least be scoped; separate the retrieval layer from the model, since personal information held in a vector store or a database and injected at inference time can actually be deleted, which is a strong argument for keeping it there rather than training it in; and decide in advance what your response to a deletion request touching a model will be, because inventing one under a statutory clock is worse.