Wikidata as a Free Backbone

Wikidata is a free, structured, CC0-licensed knowledge base with a public SPARQL endpoint, covering people, places, organisations, works, taxa and much else. It is the most useful free backbone available for linking your own entities to the outside world, and it is community-maintained, which is both why it exists and the source of every caveat on this page.

How it is structured

Finding the Q-id or P-id you need is a lookup, not a guess. Search on the site, open the item, and the identifier is in the URL and the header. Do not infer identifiers from patterns; there is no pattern, and a wrong Q-id produces a query that runs and returns the wrong thing.

wdt: versus p:, and why it matters

The query service exposes the same data through several prefixes, and choosing wrongly is the single most common source of quietly incorrect Wikidata queries:

The consequence: a query written with wdt:P39 asking which positions a person has held returns only the best-ranked ones, with no dates. Somebody who held three offices in sequence looks like they held one. The full form gets everything:

PREFIX wd:  <http://www.wikidata.org/entity/>
PREFIX p:   <http://www.wikidata.org/prop/>
PREFIX ps:  <http://www.wikidata.org/prop/statement/>
PREFIX pq:  <http://www.wikidata.org/prop/qualifier/>

SELECT ?position ?positionLabel ?start ?end WHERE {
  wd:Q_PERSON p:P39 ?statement .          # P39 = position held
  ?statement ps:P39 ?position .
  OPTIONAL { ?statement pq:P580 ?start }  # P580 = start time
  OPTIONAL { ?statement pq:P582 ?end }    # P582 = end time
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}
ORDER BY ?start

Use wdt: when you want the current best value and nothing else — which is genuinely most of the time, and it is much faster. Use p:/ps:/pq: the moment history, context or sourcing is part of the question. This is the same reification pattern discussed in temporal knowledge graphs, deployed at scale.

The label service is the other thing to learn immediately. Adding the SERVICE wikibase:label block makes ?xLabel available for any variable ?x, which saves a join per label and is the difference between a readable result set and a page of Q-ids.

Queries that do real work

Subclass traversal is the pattern to internalise, because Wikidata classes are deep. Asking for instances of a class without following P279* misses everything classified more specifically:

# every item that is an instance of the class, or of any of its subclasses
SELECT ?item ?itemLabel WHERE {
  ?item wdt:P31/wdt:P279* wd:Q_YOUR_CLASS .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}
LIMIT 500

Resolving a batch of your own names against Wikidata, which is the query you will actually run. VALUES injects your list, and the match is against labels and aliases:

SELECT ?name ?item ?itemLabel ?countryLabel WHERE {
  VALUES ?name { "Acme Robotics" "Vandenberg Automation" "Zenith Drives" }
  ?item rdfs:label|skos:altLabel ?label .
  FILTER (STR(?label) = ?name)
  OPTIONAL { ?item wdt:P17 ?country }     # P17 = country
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}

Querying from code is a plain HTTP GET with an Accept: application/sparql-results+json header. Two operational points matter more than the code: the Wikimedia user-agent policy asks for a descriptive User-Agent including a contact address, and the public endpoint enforces a query timeout, so anything expensive must be split with LIMIT and OFFSET or run against a dump.

import requests

ENDPOINT = "https://query.wikidata.org/sparql"
HEADERS = {
    "Accept": "application/sparql-results+json",
    "User-Agent": "acme-kg/1.0 (https://acme.example; [email protected])",
}

def sparql(query: str) -> list[dict]:
    r = requests.get(ENDPOINT, params={"query": query},
                     headers=HEADERS, timeout=90)
    r.raise_for_status()
    return r.json()["results"]["bindings"]

Measuring coverage for your own domain

Any figure quoted for Wikidata’s coverage is out of date by the time it is published, and a global figure would not tell you about your domain anyway. Measure it directly. The pattern is a count of items in a class against a count of those with the property you need:

# what share of items in this class carry the property you depend on?
SELECT (COUNT(?item) AS ?total) (COUNT(?value) AS ?with_property) WHERE {
  ?item wdt:P31/wdt:P279* wd:Q_YOUR_CLASS .
  OPTIONAL { ?item wdt:P_YOUR_PROPERTY ?value }
}

Coverage is also uneven in ways a single number hides — typically by country, by language and by recency. Break it down before you trust it:

SELECT ?countryLabel (COUNT(?item) AS ?items)
                     (COUNT(?value) AS ?with_property) WHERE {
  ?item wdt:P31/wdt:P279* wd:Q_YOUR_CLASS ;
        wdt:P17 ?country .
  OPTIONAL { ?item wdt:P_YOUR_PROPERTY ?value }
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}
GROUP BY ?countryLabel
ORDER BY DESC(?items)
LIMIT 40

And the coverage that actually matters is coverage of your entities, not of the class in general. Take a random sample of 200 of your own records, attempt to resolve each against Wikidata, check the matches by hand, and you have three numbers worth more than any published statistic: the share that can be matched at all, the share matched correctly, and the share whose Wikidata record carries the fields you wanted. Repeat it annually; the answer moves.

Linking your entities to it

Store the QID as an alias, never as a key. The reasoning is in canonical entity ids and it applies with particular force here: Wikidata items are merged and split by volunteer editors, and a QID you depend on can be redirected without notice.

INSERT INTO entity_alias (source, source_key, entity_id, confidence, asserted_by)
VALUES ('wikidata', 'Q42', 'co_01J9XQ...', 0.94, 'linker:v2')
ON CONFLICT (source, source_key) DO NOTHING;

What the link buys, concretely: multilingual labels and aliases for free, which is a large win for search and for the alias tables in entity resolution; cross-walks to other identifier systems, since Wikidata items carry external-identifier properties; and hierarchy through P279 that you do not have to maintain.

What it does not buy: authority. Wikidata is a starting point for resolution and a source of enrichment. It is not a system of record for anything you are accountable for, and a fact taken from it should carry provenance saying so.

What it is and is not

• It is community-maintained. Anyone can edit it. Most edits are good, some are bots, and errors and vandalism both occur and are usually corrected. If a decision depends on a Wikidata value, either verify it against a system of record or accept the risk explicitly.
• Coverage is uneven and reflects who edits. Subjects with active volunteer communities are deep; commercially mundane domains are thin. This is not a criticism of the project; it is a direct consequence of how it is built, and it is why the measurement queries above exist.
• The public endpoint has no service level. It is a free public good with a query timeout and shared capacity. Do not put it in a synchronous user-facing path. Cache results, or load a dump.
• Dumps exist and are the right answer at scale. Full entity dumps are published for download, and loading one into your own store removes the timeout, the rate limits and the availability risk in a single step. It is a large download and a real ingestion job, so budget for it rather than discovering it.
• The licence is CC0. The data is in the public domain, which is unusually permissive and is a substantial part of why it is worth building on.