Extracting Methods and Sample Size From a Scientific Paper

Ask a model for the sample size of a clinical paper and it returns one number. The paper contains four, they disagree, and every one of them is correct. The extraction problem here is not finding a number — it is deciding which stage of the study your field is supposed to describe.

There is more than one n and all are right

A trial report distinguishes, at minimum, the number of people assessed for eligibility, the number enrolled, the number randomised, the number who received the allocated intervention, the number who completed follow-up, and the number included in the primary analysis. These are different quantities by design. Attrition between them is not an error; it is the study, and reporting it is the point.

So an abstract that says “we randomised 480 patients” and a results table headed n = 431 are not in conflict. The second is the analysis population after withdrawals. A validator that flags the mismatch as an extraction error is wrong, and worse, it trains whoever reviews the queue to dismiss the flag.

The same fracture appears outside clinical work. An observational study reports a cohort size and a complete-case size. A survey reports invitations sent, responses received and responses retained after quality filtering. A machine learning paper reports dataset size, training split and evaluation split. In every case the mistake is a schema with one integer field called sample_size.

The flow diagram is the authority

For a randomised trial the numbers are not scattered arbitrarily. The CONSORT statement — the reporting guideline most medical journals require — specifies a participant flow diagram carrying the count at each stage, and requires the numbers analysed for each group to be stated. Where a paper follows it, the flow diagram is the single authoritative source and every other mention of n in the paper is a summary of it. Systematic reviews have the equivalent in the PRISMA flow diagram, which counts records identified, screened, excluded with reasons, and included.

That matters operationally because the flow diagram is a figure. It is a boxed graphic, often vector line art with text objects positioned absolutely, and a text-layer extractor returns its labels in drawing order — which is neither reading order nor visual order. The numbers arrive shuffled and unattached to their boxes. This is a coordinate problem, not a prompting problem: you need the position of each text run to reattach a count to its stage, which is the clustering described in two-column PDF reading order. Rendering the figure region and passing the image to a vision model is usually the shorter path, and it is the one case in this cluster where paying for a page image is clearly worth it.

Where the number hides

Four locations, in descending order of how much you should trust them and ascending order of how easy they are to find:

• The flow diagram or the analysis table. Most reliable, hardest to parse, and the only place that distinguishes the stages explicitly.
• The methods section. Contains the planned sample size and the power calculation that justified it — which is the target, not the achieved count. A paper that says “we calculated that 500 participants would provide 90% power at alpha 0.05” is telling you what it aimed for. Enrolment may have stopped early.
• The results text. Usually the achieved figure, often with the stage named in the same sentence.
• The abstract. Easiest to find, most compressed, and frequently states only one number without saying which stage it is. Extracting from the abstract alone is the source of most silently wrong sample sizes.

Number formatting adds its own noise. n=431, N = 431, 431 participants, four hundred and thirty-one, and 1,431 versus 1.431 in a paper typeset with a European decimal convention are all live. Group counts are commonly given as (n = 216/215), which is two numbers in one token and reads as a date to a careless parser.

Extracting the design, not just the count

A count without a design is close to useless downstream, and the design is a closed vocabulary, which makes it a much better extraction target than free text. Constrain the model to an enumeration — randomised controlled trial, cluster randomised, crossover, non-randomised interventional, prospective cohort, retrospective cohort, case-control, cross-sectional, case series, systematic review, meta-analysis — plus an explicit other and an unclear. An enum that lacks an escape hatch forces the model to pick a wrong neighbour, which is the failure described in schema edge cases.

Two design details are worth their own fields because they change what the sample size means. Whether analysis was intention-to-treat or per-protocol determines whether the analysed count includes people who did not receive the intervention. And the unit of randomisation matters in cluster trials: a study of 40 clinics with 4,000 patients has a sample size of 40 for some purposes and 4,000 for others, and a schema with one integer cannot say which.

A schema that survives disagreement

{
  "design": "randomised_controlled_trial",
  "analysis_population": "intention_to_treat",
  "randomisation_unit": "participant",
  "counts": [
    { "stage": "assessed_for_eligibility", "n": 612, "source": "flow_diagram" },
    { "stage": "randomised",               "n": 480, "source": "abstract" },
    { "stage": "analysed_primary",         "n": 431, "source": "results_table" }
  ],
  "counts_disagree": false,
  "planned_n": 500,
  "planned_n_source": "methods_power_calculation"
}

Three properties make this work. Each count carries the stage, so nothing downstream has to guess. Each count carries the section it came from, so a disagreement is traceable to a location on the page rather than to the model. And planned_n is a separate field from every achieved count, so the power calculation can never be mistaken for the study size.

The checkable assertion is monotonicity: within one arm of one study, the counts should be non-increasing down the stages. Assessed is at least enrolled, enrolled is at least randomised, randomised is at least analysed. A record where the analysed count exceeds the randomised count is either an extraction error or a paper worth a second look, and either way it belongs in review. This is the same discipline as asserting that a total foots — it does not tell you the numbers are right, but it catches the class of error where one of them came from the wrong place.

If your downstream task is summarising or comparing studies rather than building a database, do the extraction as a separate call first and reason over the structured record afterwards, for the reasons in extract then reason. A model asked to extract and compare in one step will quietly resolve the disagreement between two values of n by picking one.