Provenance in a Knowledge Base

Provenance is the answer to “where did that come from?”, stored as data rather than remembered by whoever built the pipeline. It is skipped in every tutorial and required in every system that has been running longer than a year, for one reason: eventually a source turns out to be wrong, and you have to find everything it touched.

The operation that pays for it

Suppose a supplier feed was misconfigured for six weeks and mapped country codes wrong. Without provenance, the recovery is to reload everything from every source and hope the overwrite order is favourable. With provenance, it is a query: find every statement whose source is that feed within that window, retract it, and re-derive whatever depended on it.

Provenance also buys three smaller things that add up:

• Citation. An answer that can point at the source record is one a user can check, which is the same discipline as citing retrieved sources but applied to facts rather than to passages.
• Precedence. When two sources disagree, a rule can only be written if the source is on the fact.
• Debugging extraction. A wrong edge from LLM extraction is only diagnosable if you can get back to the document, the model and the prompt version that produced it.

Choosing the grain

The single biggest decision, and it is a storage-cost decision as much as a modelling one:

Per-statement is the right default. The multiplier is real but the alternative is a graph in which no individual fact can be traced, and the first time that costs you a week of reconciliation it has paid for the disk many times over.

Four ways to store it

Named graphs (RDF)

Put each source’s statements in its own graph and describe the graph. This is the reason RDF stores are quad stores rather than triple stores, and it is the cleanest of the four:

# the data, in a per-source graph
GRAPH <urn:src:sap:2026-08-04> {
  ex:acme ex:country ex:germany .
  ex:acme ex:supplies ex:vandenberg .
}

# the description of that graph, in the default graph
<urn:src:sap:2026-08-04>
  prov:wasGeneratedBy   <urn:job:sap-load-7741> ;
  prov:wasAttributedTo  <urn:system:sap> ;
  prov:generatedAtTime  "2026-08-04T02:14:00Z"^^xsd:dateTime ;
  ex:trustTier          "system-of-record" .

RDF-star

RDF-star allows a triple to be the subject of another triple, which expresses per-statement metadata without reification boilerplate:

<< ex:acme ex:supplies ex:vandenberg >>
  prov:wasDerivedFrom ex:pressRelease44 ;
  ex:confidence       0.82 .

Edge properties (property graph)

MERGE (a:Company {id:$a})-[r:SUPPLIES {source: $source_id}]->(b:Company {id:$b})
SET r.doc_id     = $doc,
    r.extractor  = '[email protected]',
    r.confidence = $confidence,
    r.asserted_at = datetime()

Note the source id is in the MERGE key, not only in the SET. Without that, a second source asserting the same relationship overwrites the first source’s provenance and the corroboration is lost.

An assertion table (relational)

CREATE TABLE assertion (
  id           bigserial PRIMARY KEY,
  subject_id   text NOT NULL,
  predicate    text NOT NULL,
  object_id    text,
  object_value jsonb,
  source_id    text NOT NULL REFERENCES source(id),
  doc_id       text,
  evidence     text,                  -- the quote, for extracted facts
  confidence   numeric(4,3),
  asserted_at  timestamptz NOT NULL DEFAULT now(),
  retracted_at timestamptz,
  CHECK (object_id IS NOT NULL OR object_value IS NOT NULL)
);

CREATE INDEX ON assertion (subject_id, predicate) WHERE retracted_at IS NULL;
CREATE INDEX ON assertion (source_id, asserted_at);

Unfashionable and extremely effective. The current graph becomes a view over non-retracted assertions, retraction is an update rather than a delete, and the second index is exactly the one the retraction query needs.

PROV-O, in the part you will use

PROV-O is a W3C vocabulary for provenance. It is large; three classes and three properties cover almost every practical case:

Using the standard names rather than inventing ex:source and ex:createdBy costs nothing and means a tool or a colleague can read the graph without a glossary. That is close to the entire practical argument for shared vocabularies.

The retraction query

Retraction has two halves and everybody remembers only the first. The first is withdrawing what the bad source asserted directly:

UPDATE assertion
   SET retracted_at = now()
 WHERE source_id = 'sap-supplier-feed'
   AND asserted_at >= '2026-06-15'
   AND asserted_at <  '2026-07-27'
   AND retracted_at IS NULL;

The second is everything computed from those assertions — materialised inferences, aggregates, resolved entities, denormalised copies. Following the derivation chain requires that derived facts record what they were derived from, which is the whole reason wasDerivedFrom is worth storing:

WITH RECURSIVE tainted AS (
  SELECT id FROM assertion
   WHERE source_id = 'sap-supplier-feed'
     AND asserted_at >= '2026-06-15'
     AND asserted_at <  '2026-07-27'

  UNION

  SELECT d.derived_id
  FROM derivation d
  JOIN tainted t ON d.source_assertion_id = t.id
)
SELECT count(*) FROM tainted;

Run the count version first, always. The number tells you whether this is a retraction or an incident, and a retraction that turns out to touch forty per cent of the graph is a decision for a person rather than for a job.

Then re-derive rather than restore. Restoring a backup discards everything correct that has happened since; re-deriving from the surviving assertions produces a graph that is consistent with what you currently believe, which is the state you actually want.

Two sources, two answers

Provenance turns a conflict from a bug into a decision, but only if the decision is written down. Give every source a tier and make the rule explicit and per-field:

resolution rules, in order:
1. a human steward decision wins, until superseded by another steward
2. otherwise the highest source tier wins:
     system-of-record  >  verified-third-party  >  extracted  >  inferred
3. within a tier, the most recent assertion wins
4. if two assertions tie on all three, surface both and mark the field
   contested rather than picking one

Rule 4 is the one that gets deleted for being inconvenient, and it is the one that prevents the worst failure: a field that flips between two values on every load because two sources disagree and the tie-break is effectively random. A contested field displayed as contested is information; a field silently oscillating is a bug report six months from now. The same rules, applied at the record level rather than the field level, are the substance of master data management.

Confidence scores, and how not to use them

Almost every provenance schema grows a confidence column, and most of them are filled with numbers that do not mean what the column name implies. Three distinct things end up in there:

The practical rules that follow. Never mix the three in one column; store what produced the number alongside it, or use three columns. Never multiply confidences along a chain of derivations as if they were independent probabilities — they are not independent and they are not probabilities, and the product falls towards zero for reasons that have nothing to do with the facts. And never use a raw score as a threshold in a user-facing decision without calibrating it against labelled outcomes first, which is the same discipline as picking a threshold in entity resolution.

The version that works in practice is coarse and defensible: three or four ordinal tiers with written definitions — verified, corroborated, single-source, unverified — assigned by rule rather than by a continuous score. A tier a person can explain beats a decimal nobody can defend, and it survives a change of matcher.