Master Data Management for AI Teams
11 min read · updated August 4, 2026
Master data management is the discipline of deciding, across every system in an organisation, which record is the real one and which value is the right one. It has a reputation as a multi-year enterprise programme. The part an AI team actually needs is much smaller, and they usually discover they need it about three weeks after a retrieval system starts giving three different answers to one question.
How AI teams arrive here
The symptom is specific and recognisable. An assistant is built over the CRM, the billing system and the support desk. A user asks for a customer’s address and gets a confident answer. It is the wrong address — or rather, it is one of three addresses, all of which are in the corpus, and the one that surfaced is the one whose chunk scored highest. Nothing in the pipeline is broken. Retrieval found a document that says what it says.
This is not fixable with better chunking, a reranker or a larger context window, because the ambiguity is in the sources rather than in the retrieval. Three systems hold three answers and no system holds the answer. Somebody has to decide, and MDM is the name for deciding systematically instead of per-question.
The related symptom is counting. “How many customers do we have” has three answers, and the difference is not a bug in any system — it is that each defines a customer differently. That is a definitional problem, and it is what a shared glossary and MDM solve between them.
Four architectural styles
| Style | Description |
|---|---|
| registry | Sources keep their data. A central index stores only the cross-references: this CRM account is that billing account. Lightest, least invasive, and the right first move almost always. |
| consolidation | Data is copied into a central store where a golden record is computed, read-only, for analytics and AI. Sources are unchanged and unaware. This is what most AI teams actually need. |
| coexistence | The golden record is computed centrally and written back to the sources. Sources agree, at the price of an integration per source and a synchronisation problem. |
| centralised | The MDM system owns the data outright and sources read from it. Cleanest model, largest change programme, and the one whose failure mode is a two-year project. |
The distinction that matters for an AI team: registry and consolidation require nothing from the owners of the source systems, and the other two require everything. A retrieval or graph pipeline needs the right answer for reading, not for writing, so consolidation is usually sufficient — and it can be built by one team without a steering committee, which is the actual constraint.
Mechanically, consolidation is entity resolution to decide which records are one thing, plus canonical ids so that the decision is stable, plus survivorship to pick values.
Survivorship: which value wins
Given three records agreed to be one customer, the golden record needs one value per field. Survivorship rules decide, and they are per-field — a single rule for the whole record is the most common mistake in this area:
| Rule | Description |
|---|---|
| source precedence | The billing system wins on legal name and tax id; the CRM wins on contact preferences; the support desk wins on nothing. Usually the best rule and the easiest to defend. |
| most recently updated | Reasonable for contact details, dangerous elsewhere: a system that rewrites rows nightly always looks newest regardless of whether anything changed. |
| most complete | Prefer the non-null, or the longer value. Attractive and frequently wrong — the longest address is the one with the note appended to it. |
| most frequent | Where three or more sources hold a value, take the majority. Genuinely useful, and only available once you have several independent sources. |
| verified wins | A value confirmed by a person or by an external check beats any automatic rule until it is re-verified or expires. |
Two disciplines make survivorship survivable. Record which source each field came from in the golden record, so that “why does it say that” has an answer — the same argument as provenance. And never let a rule silently overwrite a steward’s decision; a verified value beats every automatic rule until somebody re-verifies it.
The stewardship queue
Every automated system in this area produces cases it should not decide: pairs in the review band from entity resolution, fields flagged as contested, plausibility violations, splits. That output needs somewhere to go and somebody to work it, and a queue with nobody assigned to it is a slow way to discover the project has ended.
- Size it before shipping. Estimated cases per week times minutes per case gives hours per week. If that number exceeds what your stewards have, narrow the review band or raise the auto-merge threshold — do not ship it and hope.
- Show the evidence, not the records. A steward deciding a merge needs the fields that differ, the scores, and the sources, on one screen. Two raw records side by side is a twenty-minute task; a diff with scores is a twenty-second one.
- Capture every decision as a label. This is the only training data that will ever exist for your matcher, and it is free if you store it and gone if you do not.
- Make decisions reversible and make reversal easy. Stewards who fear a mistake work slowly; stewards who can undo work quickly.
- Report the queue depth as a system metric. A growing queue means the automated thresholds are wrong, and it is visible weeks before the data quality degrades.
Why these programmes fail
The failures are consistent and none of them is technical:
- No consuming use case. A golden record built because data quality is good produces a dataset nobody reads and nobody maintains. Build it for one named consumer that is currently getting wrong answers.
- Ownership without authority. The team told to fix the data cannot change the systems producing it and cannot refuse a feed. Registry and consolidation styles sidestep this; the other two do not.
- The definition was never agreed. Half of what looks like a data quality problem is two departments meaning different things by one word. That is a glossary problem, and no amount of matching fixes it.
- Scope by domain, not by system. “All master data” never finishes. Customer, this quarter, for the support assistant, does.
A version that fits in a quarter
- Pick one entity type and one consumer. Customers, for the assistant that keeps giving three addresses.
- Stand up the identity layer: canonical ids and an alias table mapping every source key onto them. This alone fixes the counting question.
- Run entity resolution with a deliberately conservative auto-merge threshold, and a review band sized to the stewards you actually have.
- Write survivorship rules for the six fields the consumer needs. Not all forty; the other thirty-four can wait until something reads them.
- Expose the golden record as a read API and point the consumer at it, replacing the three-source retrieval for those fields.
- Measure one number — the share of answers about those fields that a steward judges correct — before and after. That number is what buys the second quarter.