Retention Limits for AI Training Datasets Under GDPR

Training corpora are kept forever by default, because deleting one feels like destroying an asset and nobody has ever been asked to justify the decision. Article 5(1)(e) asks.

What storage limitation actually says

Article 5(1)(e) of the GDPR requires personal data to be kept in a form which permits identification of data subjects for no longer than is necessary for the purposes for which it is processed. It is a principle, not a schedule: the Regulation nowhere states a number, and any vendor who tells you the GDPR mandates a specific retention period is wrong.

What the Regulation does instead is make you state the period and stand behind it. Article 13(2)(a) and Article 14(2)(a) require you to tell data subjects the period, or the criteria used to determine it, at the time of collection. Article 30(1)(f) requires the envisaged time limits for erasure to appear in your record of processing activities where possible. Article 5(2) makes you responsible for demonstrating compliance. So the obligation in practice is: pick a period, write it down, be able to defend it, and actually delete.

The phrase “in a form which permits identification” is the escape hatch and it is a real one. The principle constrains identifiable retention. Data that is genuinely anonymous falls outside the GDPR altogether, and pseudonymised data does not — pseudonymisation is a safeguard under Article 4(5) and Article 32, not an exit. The distinction is developed in anonymisation and pseudonymisation of training data, and it is the difference between a defensible retention position and a claimed one.

The moment the corpus stops being necessary

Necessity is judged against the stated purpose, which makes the wording of the purpose decisive. If the purpose is “to train model version 3”, the corpus stops being necessary when version 3 is trained and validated, and the clock is short. If the purpose is “to develop and improve our models”, the corpus arguably remains necessary while development continues — but that purpose is vague enough to strain the specificity requirement in Article 5(1)(b), and a regulator asked to accept an indefinite retention period on the strength of an indefinite purpose is being asked to accept the thing the principle exists to prevent.

There is also a real asymmetry in the artefacts. The raw scraped or collected data, the cleaned intermediate, the tokenised shards and the final curated training file are four datasets with four different necessity arguments. Almost nobody needs the raw crawl after cleaning is complete, and it is usually the largest and most sensitive of the four. Deleting the raw layer while keeping the curated one is often the single highest-value retention decision available, and it is invisible if the schedule treats “training data” as one entry.

The four reasons given for keeping it

Reproducibility. The argument that you must retain the corpus to reproduce the training run. This is genuine for a while and weakens over time: reproducibility matters until the model version is retired, not forever. It also does not require identifiable data in every case — a hash manifest of the corpus establishes what was used without retaining the contents, which is enough for many audit purposes and not enough for a full retrain.

Regulatory documentation. The argument that the AI Act requires it. Read the provisions before accepting this. Article 11 and Annex IV of Regulation (EU) 2024/1689 require technical documentation describing the data and the data-governance choices, and Article 10 requires data-governance practices for high-risk systems, but describing a dataset is not the same obligation as retaining it. Article 12 and Article 19 concern logs generated by the system, with a retention floor of at least six months unless other law provides otherwise, and logs are not the training corpus. See the Article 11 documentation duty and Article 10 data governance. Where a genuine documentation obligation does require retention, that is a legal obligation and it supports the retention — but cite the provision rather than gesturing at the Act.

Bias testing and evaluation. A strong argument, and the one that most often justifies retaining a subset rather than the whole. A held-out evaluation set with the attributes needed to test for disparate performance is a much smaller and more defensible retention than the full corpus, and Article 10(5) of the AI Act contemplates processing special-category data where strictly necessary for bias detection and correction, subject to conditions. Retain the evaluation set on that basis and delete the rest.

Future models. The weakest of the four. Retaining data because you may want to train something else later is retention for an unspecified future purpose, which fails necessity against the stated purpose and raises the compatibility question in Article 6(4) separately. If you genuinely intend serial reuse, say so in the purpose at collection and in the privacy notice; retrofitting it as a retention justification does not work.

The research exception, read properly

Article 5(1)(e) contains a qualification: personal data may be stored for longer periods insofar as it will be processed solely for archiving purposes in the public interest, scientific or historical research purposes, or statistical purposes, subject to implementation of the appropriate technical and organisational measures required by Article 89(1). Article 5(1)(b) contains a matching compatibility presumption for further processing for those purposes.

This is the provision most often reached for and most often misapplied. Three constraints limit it. The word is “solely”, so a corpus used both for research and to train a commercial production model is not within it. Article 89(1) requires safeguards including data minimisation and, where the purpose can be fulfilled that way, pseudonymisation or anonymisation — so the exception comes with an obligation to reduce identifiability rather than a licence to keep everything as collected. And member states may legislate further conditions under Article 89(2) and (3), so the position is not uniform across the EU. Commercial R&D is not automatically scientific research, and the EDPB has been unenthusiastic about elastic readings of the term.

Writing a schedule that survives an audit

A defensible schedule for an AI programme is per-artefact, not per-project, and it states a trigger rather than only a duration — because “24 months” without a start event is not operational.

1. Enumerate the artefacts separately: raw collected data, cleaned intermediates, curated training files, evaluation and held-out sets, human-annotation records, inference logs, fine-tuning uploads held by a vendor, and backups of each.
2. For each, write the purpose it serves in one sentence, the retention trigger (model version retired, evaluation cycle complete, contract terminated), and the period after that trigger.
3. Name the legal ground for anything kept beyond the obvious necessity window — a specific statutory documentation duty, a litigation hold, or Article 89(1) research with its safeguards described.
4. Include the vendor-held copies. Article 28(3)(g) requires the processor to delete or return personal data at the end of the provision of services at the controller’s choice, and an uploaded fine-tuning file sitting in a provider account is the copy that gets missed. This is one of the questions in the subprocessor checklist.
5. Include backups explicitly, with the honest position: most backup systems cannot delete selectively, so the defensible answer is that restored data is re-deleted on restore and that backups age out on a stated cycle. Saying that is better than a schedule that silently excludes them.
6. Record the deletion. An unexecuted schedule is worse than none, because it is documented evidence of an obligation you did not meet.

One structural point to finish on. Deleting the corpus does not delete the model, and if the model is not anonymous — a proposition the EDPB declined to grant by default in its Opinion 28/2024 of 17 December 2024 — then personal data survives the deletion of every dataset in your schedule. Retention policy for the corpus and lifecycle policy for the checkpoints are two documents, and only one of them is usually written.