When to Use a Knowledge Graph: Questions Vectors Cannot Answer

A vector index answers one question: which stored items are nearest to this one. That question is enormously useful and it is not the same question as “which suppliers are exposed to this plant”, “how many”, “which have none”, or “all of them”. Here are five question classes it cannot answer, with the query that can.

What a vector index actually returns

Retrieval by embedding takes a query vector and returns the k stored vectors with the highest similarity. Three properties of that operation determine everything below, and none of them is a criticism — they are the definition:

• It ranks; it does not filter. The result is the top k by score, always of size k, whether or not anything is relevant. There is no notion of “matches” and “does not match”, so there is no set to be complete over.
• It compares one thing to one thing. Similarity is computed between the query and each item independently. Nothing in the operation can combine two stored items, which is what a join is.
• It has no arithmetic. Cosine distance does not count, sum, or compare a value against a bound. Anything numeric in the answer came from text that happened to be retrieved.

Add approximate search on top — HNSW and its relatives, described in how HNSW works — and even the top-k guarantee becomes probabilistic. That is the correct engineering trade for search. It is the wrong basis for an answer that has to be right.

1. Multi-hop: facts that are not in any document

“Which of our customers are exposed, at any depth, to a component made at the Leipzig plant?”

No document says this. The chain lives across four records: the plant makes a component, the component goes into a subassembly, the subassembly is in a product, the product was sold to a customer. Retrieval can find each link if you already know to look for it. It cannot compose them, because composition is a join and there is no join in a nearest-neighbour lookup. Chunk the four records together and you have hard-coded one question; the next question needs a different chunking.

// Cypher
MATCH (plant:Site {id: 'site_leipzig'})
MATCH (plant)-[:MAKES]->(:Component)
      -[:PART_OF*0..6]->(:Component)
      -[:PART_OF*0..3]->(p:Product)
MATCH (p)<-[:PURCHASED]-(cust:Customer)
RETURN DISTINCT cust.name, cust.account_manager
ORDER BY cust.name

-- PostgreSQL, same question, recursive CTE
WITH RECURSIVE affected AS (
  SELECT component_id
  FROM made_at
  WHERE site_id = 'site_leipzig'

  UNION            -- UNION, not UNION ALL: a BOM can contain a cycle

  SELECT b.parent_id
  FROM bom b
  JOIN affected a ON b.child_id = a.component_id
)
SELECT DISTINCT c.id, c.name
FROM affected a
JOIN product_component pc ON pc.component_id = a.component_id
JOIN purchase p           ON p.product_id    = pc.product_id
JOIN customer c           ON c.id            = p.customer_id
ORDER BY c.name;

# SPARQL, same shape, using a property path
PREFIX ex: <https://example.com/id/>

SELECT DISTINCT ?customer ?name WHERE {
  ex:site_leipzig ex:makes ?component .
  ?assembly ex:partOf* ?component .
  ?product  ex:contains ?assembly .
  ?customer ex:purchased ?product ;
            ex:name ?name .
}

The recall property is what matters here. The graph query returns every customer on every chain, or it returns an error. A retrieval pipeline returns the customers whose records happened to score well, and gives you no way to know which ones it missed.

2. Aggregation: how many, which is most

“How many suppliers do we have in each country, and which three countries have the most?”

Embeddings cannot count. If you ask a retrieval-augmented system this, what happens is that it retrieves twenty supplier records and the model counts those twenty — producing a confident, specific, wrong number that is bounded above by k. This failure is particularly nasty because the answer looks exactly like a correct answer, and the only tell is that it is suspiciously close to whatever your retrieval limit is set to.

// Cypher
MATCH (s:Company)-[:SUPPLIES]->(:Company {id: $us})
MATCH (s)-[:HEADQUARTERED_IN]->(:Location)-[:IN_COUNTRY]->(country:Country)
RETURN country.name AS country, count(DISTINCT s) AS suppliers
ORDER BY suppliers DESC
LIMIT 3

# SPARQL
SELECT ?countryName (COUNT(DISTINCT ?supplier) AS ?n) WHERE {
  ?supplier ex:supplies ex:us ;
            ex:headquarteredIn/ex:inCountry ?country .
  ?country ex:name ?countryName .
}
GROUP BY ?countryName
ORDER BY DESC(?n)
LIMIT 3

The same applies to sums, averages, ratios and percentiles. Any question containing “how many”, “what share”, “total”, “average” or “top” needs a query over a complete set, not a sample of a ranked list.

3. Negation: the things that are not there

“Which enterprise accounts have had no support contact in 90 days?”

This one is structurally impossible for similarity search, and the reason is worth stating precisely: absence has no embedding. There is no document describing the ticket that was not filed, so there is nothing for the query vector to be near. Retrieval can only ever return things that exist.

// Cypher
MATCH (a:Account {tier: 'enterprise'})
WHERE NOT EXISTS {
  MATCH (a)<-[:FOR_ACCOUNT]-(t:Ticket)
  WHERE t.created_at > datetime() - duration({days: 90})
}
RETURN a.name, a.owner
ORDER BY a.name

# SPARQL
SELECT ?account ?name WHERE {
  ?account a ex:Account ;
           ex:tier "enterprise" ;
           ex:name ?name .
  FILTER NOT EXISTS {
    ?ticket ex:forAccount ?account ;
            ex:createdAt ?created .
    FILTER (?created > "2026-05-06T00:00:00Z"^^xsd:dateTime)
  }
}

Every question of the form “missing”, “without”, “never”, “not yet”, “overdue” or “gap” is in this class, and those are a large share of the questions an operations team actually asks. This is usually the fastest way to demonstrate why a graph is needed to somebody who is happy with their vector index.

4. Conjunctive constraints: all of these at once

“Which gearboxes fit the RX-40 chassis, are rated above 200 Nm, are certified for the EU market, and have at least two active suppliers?”

Similarity blends. Given four requirements, an embedding produces one point in space and returns items near it overall — which means an item that satisfies three requirements strongly and violates the fourth will often outrank an item that satisfies all four modestly. That is correct behaviour for search and wrong behaviour for a specification, because a constraint is not a preference: a part that does not fit does not fit, however similar it is.

// Cypher
MATCH (g:Product {category: 'gearbox'})
WHERE (g)-[:FITS]->(:Chassis {model: 'RX-40'})
  AND g.torque_nm > 200
  AND (g)-[:CERTIFIED_FOR]->(:Market {code: 'EU'})
WITH g, [(g)<-[:SUPPLIES_PART]-(s:Company) WHERE s.status = 'active' | s] AS suppliers
WHERE size(suppliers) >= 2
RETURN g.name, g.torque_nm, size(suppliers) AS supplier_count
ORDER BY g.torque_nm DESC

# SPARQL
SELECT ?part ?torque (COUNT(DISTINCT ?supplier) AS ?n) WHERE {
  ?part ex:category   ex:gearbox ;
        ex:fits       ex:rx40 ;
        ex:torqueNm   ?torque ;
        ex:certifiedFor ex:marketEU .
  ?supplier ex:suppliesPart ?part ;
            ex:status "active" .
  FILTER (?torque > 200)
}
GROUP BY ?part ?torque
HAVING (COUNT(DISTINCT ?supplier) >= 2)

Note the last constraint in both queries. “At least two active suppliers” is an aggregate inside a filter — a condition on a count of related things. Metadata filtering on a vector store handles the flat attribute predicates, and cannot express this one at all.

5. Completeness: all of them, not the top ten

“List every contract containing an unlimited liability clause that expires this year.”

This is the class that decides whether the graph gets built, because it is the class where being mostly right is worthless. Top-k retrieval returns k items. If the true answer has 340 members and k is 20, the system returns 20 and says nothing about the other 320. Raise k and you eventually exceed the context window; the failure mode moves rather than going away.

A query language returns the set. It is defined by a condition rather than by a ranking, so the answer has a size, and that size is either right or the query is wrong — which is a debuggable state. If the query is too slow you learn that immediately; you do not learn that it silently returned two-thirds of the answer.

SELECT c.id, c.counterparty, c.expires_on
FROM contract c
JOIN contract_clause cc ON cc.contract_id = c.id
WHERE cc.clause_type = 'unlimited_liability'
  AND c.expires_on >= date_trunc('year', current_date)
  AND c.expires_on <  date_trunc('year', current_date) + interval '1 year'
  AND c.status = 'active'
ORDER BY c.expires_on;

The honest caveat, and it is the important one: this only holds if the clause was extracted onto every contract. The completeness of a graph query is completeness over what is in the graph, and a graph built by an extractor with 0.85 recall gives you a complete answer over 85% of the corpus. That is still categorically better than top-k, because the gap is a measurable property of the pipeline rather than an unknowable property of one query — but it is not magic, and telling a stakeholder otherwise is how these projects lose credibility.

The hybrid that actually works

None of this argues against embeddings. The two are good at opposite things and the useful systems use both, in one of three arrangements:

The routing decision between them is mostly mechanical, and can be made from the question’s surface form. Counting words, negations, superlatives, and phrases like “all” or “every” indicate a query; “about”, “like”, “similar” and any request for an explanation indicate retrieval. Getting the model to write the query rather than guess the answer is text-to-Cypher and text-to-SPARQL, and combining both retrieval routes is what GraphRAG is for.

What the graph is worse at

The argument only stays credible if it runs in both directions. Vectors beat a graph, decisively, at all of these:

• Questions your schema did not anticipate. A graph answers what it was modelled to answer. A question about a relationship nobody extracted returns zero rows, and zero rows looks identical to “there are none”. An embedding index over the raw text will still surface the passage.
• Meaning rather than structure. “Complaints about the tone of our renewal emails” is not a relation. It is a fuzzy semantic region, which is exactly what embeddings are for.
• Cost and time to first answer. An embedding index over a corpus is an afternoon. A useful graph over the same corpus is a schema, an extractor, entity resolution and a review process. If the questions are answerable by retrieval, retrieval is the correct engineering decision and the graph is over-engineering.
• Long-tail vocabulary. Users write “the bendy-gear thing”. A graph needs that mapped to a canonical entity before it can do anything; a vector index does not care.

The decision rule that follows: build the graph when the questions are multi-hop, aggregate, negative, constraint-shaped or must be complete, and when being wrong is expensive. Otherwise index the text and spend the effort somewhere it shows.