Extracting Structured Data From a Grant Application Form

Most extraction gives you no way to know whether it worked. A federal grant budget is an exception: it is a grid that adds up in two directions, so the arithmetic checks the transcription. The trap is that one line on it is not supposed to reconcile the way you would expect.

The budget is a grid, and grids foot

US federal discretionary applications are submitted on a standard form set: the SF-424 application face sheet, and for non-construction programmes the SF-424A budget information form. The budget form is organised into sections, and the one that carries the detail is the budget-categories section, which lists object class categories down the side and programme or funding sources across the top.

The categories are fixed and lettered: personnel, fringe benefits, travel, equipment, supplies, contractual, construction, other, then total direct charges, then indirect charges, then the total. Two of those lines are computed from the others — total direct charges is the sum of the eight categories above it, and the total is direct plus indirect. That is the property to build on.

Represent the section as a two-dimensional array with explicit row and column labels rather than as a flat list of named fields. The number of columns varies by application and a schema with personnel_amount as a scalar cannot hold a three-programme budget, which is exactly the application where the arithmetic is most worth checking.

The two footing checks

// grid[category][column] -> number, with categories keyed by their letter.
const DIRECT = ["a","b","c","d","e","f","g","h"];   // personnel .. other

function footBudget(grid, columns) {
  const issues = [];

  // 1. Column-wise: direct total, then grand total, per column.
  for (const col of columns) {
    const direct = DIRECT.reduce((s, k) => s + (grid[k]?.[col] ?? 0), 0);
    if (!near(direct, grid.i?.[col])) {
      issues.push({ kind: "direct_total", column: col,
                    computed: direct, printed: grid.i?.[col] });
    }
    const total = (grid.i?.[col] ?? 0) + (grid.j?.[col] ?? 0);
    if (!near(total, grid.k?.[col])) {
      issues.push({ kind: "grand_total", column: col,
                    computed: total, printed: grid.k?.[col] });
    }
  }

  // 2. Row-wise: each category across columns equals its own total column.
  for (const k of [...DIRECT, "i", "j", "k"]) {
    const across = columns.reduce((s, c) => s + (grid[k]?.[c] ?? 0), 0);
    if (!near(across, grid[k]?.total)) {
      issues.push({ kind: "row_total", category: k,
                    computed: across, printed: grid[k]?.total });
    }
  }
  return issues;
}

// Currency on a form is in whole units or to two places; compare with a
// tolerance rather than by equality, and keep the tolerance small enough
// that a transposed digit cannot hide inside it.
const near = (a, b) => b != null && Math.abs(a - b) < 1;

Run both directions, not one. A column check alone misses a value read into the wrong column, because the column still sums correctly if the error was a swap within it. Running rows and columns together localises a single bad cell to its intersection: one row failure and one column failure that cross at a cell is a much stronger signal than either alone, and it is enough to put a reviewer’s cursor on the exact number.

The tolerance deserves thought. Set it too loose and a transposition — 52,100 read as 25,100 — hides inside it; set it to exact equality and every budget with a rounding convention fails. A tolerance of well under one currency unit per check, combined with reporting the computed and printed values rather than just a pass or fail, keeps the check honest and keeps its output actionable.

Why the indirect cost line will not reconcile

Here is the check people write next, and it is wrong. The application states an indirect cost rate; the budget states an indirect charge; so indirect should equal rate times total direct charges. It does not, and the mismatch is not an error.

Federal indirect cost rates are applied to a base, and the common base is modified total direct cost, which is not total direct cost. Under the uniform administrative requirements at 2 CFR part 200, modified total direct cost excludes several things that are nonetheless direct costs sitting on the same budget page: equipment and other capital expenditures, charges for patient care, rental costs, tuition remission, scholarships and fellowships, participant support costs, and the portion of each subaward beyond the first 25,000 dollars.

So the correct check is against the exclusions, and it needs data the budget grid does not carry on its own:

// Worked on a synthetic budget, with every input labelled as an assumption.
//
// assumed: total direct charges (line i)          = 480,000
// assumed: equipment (line d)                     =  60,000
// assumed: participant support, within "other"    =  25,000
// assumed: one subaward of 90,000, so the amount
//          above the first 25,000 is excluded     =  65,000
// assumed: negotiated indirect cost rate          =    32 % on MTDC
//
// MTDC = 480,000 - 60,000 - 25,000 - 65,000 = 330,000
// indirect = 0.32 x 330,000 = 105,600
//
// Checking against total direct instead would predict 0.32 x 480,000
// = 153,600 and report a 48,000 discrepancy that does not exist.

The extraction lesson is not that you must reproduce the arithmetic. It is that the budget grid alone cannot support this check, so either you extract the exclusion components as well — which means reading the budget narrative for the participant support and subaward figures — or you do not run the check and say so. Running it against the wrong base generates a discrepancy on every application, which trains reviewers to ignore your findings, which is worse than having no check.

The budget narrative is a different document

The grid gives totals; the budget narrative or justification gives the line items that produce them — named positions with percentages of effort and salaries, itemised travel with trip counts and per-diem rates, equipment with unit prices. It is prose and tables mixed, and it is the document where the interesting errors are.

Cross-checking the two is where the value is. Personnel in the narrative should sum to the personnel line on the grid; fringe should be the stated fringe rate applied to the salaries in the narrative, not to some other figure; travel line items should sum to the travel line. A narrative that itemises 84,000 dollars of salary against a grid line of 88,000 is the error the check exists to find, and it is exactly the error a reviewer catches on submission day if nobody caught it earlier.

The narrative is unstructured enough that this is a genuine extraction problem rather than a table read, and it is the place on this document where a language model adds most — pulling itemised amounts with their descriptions out of paragraphs. Give it the segmented narrative section rather than the whole application, for the same reason as everywhere else in this cluster.

Where the numbers go wrong on the page

• Parenthesised negatives. An amount shown as (12,500) is minus 12,500. Read as a positive it inverts a subtotal, and the footing check will catch it — provided your parser did not already strip the parentheses as noise.
• Thousands separators and decimal marks. A form completed with European conventions writes 1.234,56 for what a US form writes as 1,234.56. Detect the convention per document from the pattern of separators rather than assuming, and refuse to parse a document where both appear.
• Empty versus zero. A blank category means not requested; a zero is an explicit statement. The distinction survives into the award and into reporting, and coercing blanks to zero makes the two indistinguishable.
• The dollar sign and the digit. On a scanned form the currency symbol sits close to the first digit and is periodically read as one — an S, a 5, an 8. The footing check catches this, which is a good demonstration of why arithmetic beats confidence: the recogniser was perfectly confident.
• Handwritten amendments. A figure crossed out and rewritten by hand on an otherwise typed form. Both are on the page and the printed one is more legible. Flag rather than pick — the same rule as on any form with a handwritten layer.