Text-to-Cypher and Text-to-SPARQL

Getting a model to write a graph query is easy. Getting one you can execute against a production database without a human reading it first is a validation problem, and the validation is most of the work. This page builds the layer between the model and the driver.

The shape of the system

1. Serialise the schema — labels, relationship types, properties.
2. Prompt the model with the schema, the question and a few examples.
3. Validate the generated query against the schema and reject anything that writes, is unbounded, or names something that does not exist.
4. Execute as a read-only user, with a row limit and a timeout.
5. On a database error, feed the error back once. Then stop.
6. Return the rows, the query that produced them, and nothing else.

Step 6 is a product decision worth defending. Show the query. Users who can read it will catch errors you cannot, and users who cannot read it are given an honest signal that a machine wrote it. Hiding the query converts a checkable answer into an unverifiable assertion.

Serialising the schema into the prompt

The model needs the schema and nothing else — not sample data, not the full property catalogue of every node type. A compact, complete, typed listing:

Node labels and properties:
  Company(id: string, name: string, country: string, founded: int, status: string)
  Person(id: string, name: string)
  Product(id: string, name: string, category: string, torque_nm: float)
  Site(id: string, name: string, city: string)

Relationships (start -> end):
  (Company)-[:SUPPLIES {since: int}]->(Company)
  (Company)-[:PRODUCES]->(Product)
  (Company)-[:OPERATES]->(Site)
  (Person)-[:HELD]->(Employment)-[:AT]->(Company)
  (Product)-[:FITS]->(Chassis)

Notes:
  Company.status is one of: active, dissolved, acquired
  Product.category is one of: gearbox, motor, controller, sensor
  Employment has valid_from and valid_to (date); valid_to is null if current

The enumerated values in the notes matter more than they look. Without them the model writes WHERE c.status = 'Active' and the query runs, returns nothing, and reports no error — which is the worst outcome available and has its own section below.

Generate this listing from the database rather than maintaining it by hand, so it cannot drift. Most graph databases expose an introspection procedure that returns labels, relationship types and property keys; the exact name varies by product and version, so check yours. For RDF, a SPARQL query over the data does the same job:

SELECT ?class ?property (COUNT(*) AS ?uses) WHERE {
  ?s a ?class ; ?property ?o .
}
GROUP BY ?class ?property
HAVING (COUNT(*) > 50)
ORDER BY ?class DESC(?uses)

If the schema does not fit in a reasonable prompt — hundreds of labels, thousands of properties — do not truncate it arbitrarily. Retrieve the relevant subgraph schema for the question first, using an embedding index over the label and property descriptions, and put only that in the prompt. A truncated schema produces queries against the half that was cut.

The prompt

SYSTEM = """You translate questions into Cypher for the schema below.

{schema}

Rules:
- Output a single read-only Cypher query. No explanation, no markdown fence.
- Use only labels, relationship types and properties from the schema above.
- Never write CREATE, MERGE, SET, DELETE, REMOVE, DROP, LOAD CSV or CALL.
- Always end with a LIMIT of at most 200.
- Bound every variable-length pattern, e.g. [:SUPPLIES*1..5].
- If the question cannot be answered from this schema, output exactly:
  CANNOT_ANSWER: <one sentence naming what is missing from the schema>

Examples:

Q: Which companies in Germany supply Vandenberg?
A: MATCH (s:Company {country: 'DE'})-[:SUPPLIES]->(b:Company {name: 'Vandenberg Automation'})
   RETURN s.name, s.founded ORDER BY s.name LIMIT 200

Q: Who has been CTO of Vandenberg since 2020?
A: MATCH (p:Person)-[:HELD]->(e:Employment)-[:AT]->(c:Company {name: 'Vandenberg Automation'})
   WHERE e.title = 'CTO' AND e.valid_from >= date('2020-01-01')
   RETURN p.name, e.valid_from, e.valid_to ORDER BY e.valid_from LIMIT 200

Q: How many suppliers does each German company have?
A: MATCH (s:Company)-[:SUPPLIES]->(b:Company {country: 'DE'})
   RETURN b.name, count(DISTINCT s) AS suppliers ORDER BY suppliers DESC LIMIT 200
"""

The CANNOT_ANSWER escape hatch is the highest-value line in the prompt. Without an explicit way to decline, a model asked something the schema does not support will invent a relationship type that sounds right, and you will get a syntactically perfect query against a graph that does not exist. Making refusal a first-class output is the same argument as teaching a model to abstain.

Validation, before execution

Never send model output straight to a driver. Four checks, cheap, in order, and all of them run before the database sees anything:

import re

WRITE_TOKENS = re.compile(
    r"\b(CREATE|MERGE|SET|DELETE|DETACH|REMOVE|DROP|LOAD\s+CSV|"
    r"FOREACH|CALL\s+\{|USING\s+PERIODIC|ALTER|GRANT|DENY)\b",
    re.IGNORECASE,
)
UNBOUNDED_PATH = re.compile(r"\*\s*\]")          # [:REL*] with no bound
UNBOUNDED_TAIL = re.compile(r"\*\s*\d*\s*\.\.\s*\]")  # [:REL*2..]


class Rejected(Exception):
    pass


def validate_cypher(q: str, schema: dict) -> str:
    q = q.strip()
    for fence in ("```cypher", "```"):
        q = q.removeprefix(fence).removesuffix("```").strip()

    if q.startswith("CANNOT_ANSWER:"):
        raise Rejected(q)

    # 1. no writes, no procedure calls, no subquery blocks
    if WRITE_TOKENS.search(q):
        raise Rejected("query contains a write or a procedure call")

    # 2. no unbounded variable-length patterns
    if UNBOUNDED_PATH.search(q) or UNBOUNDED_TAIL.search(q):
        raise Rejected("unbounded variable-length pattern")

    # 3. every label, relationship type and property exists in the schema
    for label in set(re.findall(r":([A-Z][A-Za-z0-9_]*)\s*[){ ]", q)):
        if label not in schema["labels"] and label not in schema["rel_types"]:
            raise Rejected(f"unknown label or relationship type: {label}")
    for prop in set(re.findall(r"\.([a-z_][A-Za-z0-9_]*)", q)):
        if prop not in schema["properties"]:
            raise Rejected(f"unknown property: {prop}")

    # 4. a limit is present and sane
    m = re.search(r"\bLIMIT\s+(\d+)\s*;?\s*$", q, re.IGNORECASE)
    if not m:
        q = q.rstrip("; \n") + "\nLIMIT 200"
    elif int(m.group(1)) > 200:
        raise Rejected("limit too large")

    return q

Two honest caveats about that code. Regexes over a query language are a filter, not a parser — a determined adversary can defeat them, and if untrusted users can reach this endpoint you need a real parser and the database-level protections below rather than this alone. And check 3 is deliberately loose about distinguishing labels from relationship types, because the regex cannot reliably tell them apart; the strict version requires an actual grammar. Both are fine as a first line of defence behind a read-only credential, and neither is fine as the only line.

For SPARQL the equivalent checks are: reject INSERT, DELETE, LOAD, CLEAR and DROP; reject SERVICE unless you intend federation, since it makes your endpoint issue outbound requests to a URL the model chose; require a LIMIT; and check every prefix and predicate IRI against the vocabulary. That SERVICE clause is the one people miss and it is a genuine server-side request forgery vector.

Executing safely

• A read-only database user. This is the control that actually works. Everything above is defence in depth behind it, and if you only do one thing, do this one.
• A server-side timeout. A pattern that passes every check can still be expensive. Configure the timeout on the database or the driver, not with a client-side cancel that leaves the query running.
• A separate connection pool for generated queries, so that one bad query cannot exhaust the connections your application needs.
• Log every generated query with its outcome. This is the only corpus you will ever have for evaluating the system, and it costs nothing to collect from day one.

The zero-rows failure

The dangerous failure of this whole pattern is not a query that errors. It is a query that is valid, executes, and returns nothing — because the model wrote 'Active' where the data says 'active', or reversed a relationship direction, or filtered on a property that exists but is null on every node. The system reports “no results found”, the user concludes there are none, and nobody learns otherwise.

Distinguish empty from wrong by probing. When a query returns zero rows, re-run it with the filters progressively removed and report which predicate emptied the result:

def diagnose_empty(session, query: str) -> str | None:
    """Drop the WHERE clause, then the property maps, and see what comes back."""
    no_where = re.sub(r"\bWHERE\b.*?(?=\bRETURN\b)", "", query,
                      flags=re.IGNORECASE | re.DOTALL)
    if session.run(no_where).peek() is not None:
        return "the pattern matches, but the filters excluded everything"

    no_props = re.sub(r"\{[^}]*\}", "", no_where)
    if session.run(no_props).peek() is not None:
        return "the inline property values matched nothing"

    return "the pattern itself matches nothing in the graph"

Surface that sentence to the user. “The pattern matches, but the filters excluded everything” is an answer somebody can act on; “no results” is not. It is also the fastest route to finding the enumerated values missing from your schema serialisation.

Evaluating it

String-comparing generated queries against reference queries is useless: there are many correct Cypher queries for one question and they share almost no tokens. Evaluate on results.

1. Build a golden set of 80–150 question/query pairs, written by somebody who knows the schema, covering each of the question shapes in questions vectors cannot answer.
2. Score execution match: run the generated query and the reference query and compare the result sets as sets of tuples, ignoring column order.
3. Track four rates separately, because they need different fixes: invalid (rejected by validation), error (rejected by the database), empty, and wrong-but-non-empty. The last is the expensive one.
4. Include unanswerable questions in the set and score CANNOT_ANSWER as the correct output. A system that never declines is a system that guesses.
5. Re-run the whole set on every schema change. The schema is part of the prompt, so a schema change is a prompt change, with the same regression risk as any other.