Extracting Structured Answers From a Regulatory Compliance Questionnaire

Compliance questionnaires look like the easiest document in this cluster: a numbered list, a yes or a no, and some text. They are the one where a wrong output is hardest to spot, because a questionnaire with every answer shifted by one question is entirely plausible on inspection.

The unit is a bound pair

Whether the document is a vendor security questionnaire, a supplier quality survey, a due-diligence pack or a regulator’s self-assessment, its rows share a shape: a question identifier, the question text, a closed response drawn from a small vocabulary, and free text that qualifies the response. The industry examples are familiar — the Cloud Security Alliance publishes the Consensus Assessments Initiative Questionnaire with stable control identifiers, and the Shared Assessments programme publishes the SIG as a workbook with tabbed sections — but the shape is the same in a bespoke spreadsheet a customer emailed over.

The closed response without its explanation is not an answer. A “No” that continues “this control is implemented at the platform layer by our infrastructure provider and evidenced in section 4.2 of their attestation report” and a bare “No” are opposite findings. Pooling explanations into a single comments field per section — which is what happens when a model is asked for “the answers” and “any notes” separately — produces a dataset where every remaining No looks like a gap. The people who read this output are triaging exceptions, and you have just manufactured them.

So the extraction target is a list of triples at minimum: question_id, response, explanation, with the explanation nullable and never merged upward. Everything else on this page follows from protecting that binding.

Yes, no, and the three states in between

Treating the response as a boolean is the second common mistake. Questionnaire response vocabularies routinely carry five distinct states, and collapsing them loses the ones that matter:

• Yes — the control exists as described.
• No — it does not.
• Partial or Compensating control — something else achieves the objective. This is the state whose explanation is load-bearing.
• Not applicable — the question does not apply, and a justification is normally required for it to count. An N/A with an empty justification is itself a finding.
• Unanswered — the cell is blank. Distinct from “No” and it must never be defaulted to one. A blank is a fact about the respondent’s submission; a No is a claim.

Model the response as an enum with an explicit unanswered member rather than as a nullable boolean, and set the enum from the questionnaire’s own legend rather than from a house standard — the legend is usually printed on the first tab or in the instructions page, and different questionnaires use different words for the same state. Where the provider supports it, constraining the model to that enum at decode time is worth doing; the mechanics of enforcing an enum differ between providers, which the structured output support page covers.

Where the binding breaks

There is one failure that accounts for most bad output on these documents, and it is worth naming precisely. A questionnaire is authored in a spreadsheet with wrapped text in the explanation column and then exported to PDF. On export, a long explanation that does not fit is pushed to the next page, where it appears above or beside the next question. Read in visual order, the text now sits with the wrong question. The output is well formed, every question has an answer, and a run of them is off by one.

Three defences, in order of how much they help:

• Extract by row region rather than by reading order. If the source is a real PDF table with rules, the row boundary is a line on the page and the explanation belongs to whichever row region contains it. This is the same coordinate-first principle that fixes multi-column PDF parsing.
• Ask the source. A questionnaire that arrived as a PDF very often exists as a spreadsheet, and one email retrieves a document where the binding is structural rather than visual. The cheapest extraction problem is the one you decline to have.
• Validate the binding by content. The question identifier is usually repeated in the row, and where it is, an explanation whose row lacks an identifier is a continuation — attach it to the previous identified row rather than to the next one, and flag it.

A second, quieter break happens with merged cells: one explanation spanning three questions because the respondent answered them together. Every row of a merged block should receive the same explanation with a flag saying it was shared, rather than the first row taking it and the others appearing unexplained.

Question identity across versions

Questionnaires are versioned and respondents answer against whichever version they were sent, sometimes a year old. If you are aggregating responses across suppliers to see who has a gap in a given control, you are joining on question identity, and question identity is not stable in the way it looks.

Two independent drifts occur. Numbering is renumbered when a section is inserted, so identifier 4.7 in one version is a different question in the next. And wording is edited without renumbering, so the same identifier asks a materially different question — “do you encrypt data at rest” becoming “do you encrypt data at rest using keys you control” is the same number and a different control.

The defence is to store both: the printed identifier and a hash or normalised form of the question text as it appeared on the document you extracted. Then a join that matches on identifier but not on text is visible as a mismatch instead of silently producing a comparison between two different questions. Capture the questionnaire name and version from the cover sheet as document-level metadata at the same time; it is the field most often absent from these datasets and the one that makes the rest interpretable, in the same way schema versioning does for your own output.

A shape that supports a follow-up

{
  "questionnaire": { "name": "Supplier security assessment", "version": "2026.1" },
  "respondent": { "organisation": "Northbay Components", "completed_on": "2026-05-04",
                  "signatory_present": true },
  "answers": [
    {
      "question_id": "AC-04",
      "section": "Access control",
      "question_text": "Is multi-factor authentication enforced for remote administrative access?",
      "question_text_norm_hash": "9f13c2ae",
      "response": "partial",
      "explanation": "Enforced for all administrative accounts; three legacy service accounts are excluded and are restricted to a jump host.",
      "explanation_shared_with": [],
      "evidence_reference": "Appendix C, item 3",
      "page": 7
    },
    {
      "question_id": "AC-05",
      "section": "Access control",
      "question_text": "Are administrative sessions logged and retained?",
      "question_text_norm_hash": "2b70d441",
      "response": "unanswered",
      "explanation": null,
      "evidence_reference": null,
      "page": 7
    }
  ]
}

Two fields there exist purely for the human who follows up. evidence_reference captures the pointer respondents give to an appendix, a policy document or an attestation report, which is the thing an assessor asks for next and which is otherwise buried in prose. And page is the cheapest quality control available on a long questionnaire: it lets a reviewer jump straight to the source region for any answer, which turns a disputed extraction from an argument into a five-second check. Storing a source location alongside every field is worth the space on any document where the output will be challenged.