Categorizing Receipt Line Items for Expense Reports

The hard part of categorising receipt lines is not the classifier. It is choosing a category set that someone downstream can actually use, and handling the tax and tip that belong to no single line.

Do not invent a category set

Most expense pipelines begin by inventing categories: Meals, Travel, Supplies, Other. Then an accountant has to map those onto the lines they file against, by hand, every quarter, and the mapping is ambiguous because nobody designed the two sets to correspond.

Pick a target that already exists. For a US sole proprietor or single-member LLC that is the expense lines of Schedule C, which the business is going to file on anyway — see extracting expense categories from a Schedule C for what those lines are and how they move between tax years. For travel specifically, the vocabulary in IRS Publication 463 (Travel, Gift, and Car Expenses) is the one an auditor uses. For a company with a bookkeeper, the target is their chart of accounts and nothing else.

The rule is the same in every case: your enum should be a projection of a set someone else maintains, with a stable internal id per member and a mapping table you own. That way a change in the tax form is a change to one table rather than a reclassification of every historical receipt.

The category carries a percentage

Categories in this domain are not labels. They carry a deductible percentage, and getting the category wrong is a money error rather than a tidiness error. Business meals are subject to a statutory limit under section 274(n) of the Internal Revenue Code; entertainment expenses were made non-deductible by the 2017 tax act; and the meals limit was temporarily changed for restaurant meals during 2021 and 2022 before reverting. The IRS publishes the current treatment in Publication 463, and the figure has moved twice in a decade.

This is why alcohol on a meal receipt matters, why a hotel folio has to be split, and why “Meals & Entertainment” as a single category is now actively wrong: the two halves of that old category have different treatment.

Building it

1. Freeze the enum as data. A table of category_id, display name, target line, deductible percentage and effective dates. Not a list in a prompt string, because the prompt has to be generated from it and so does the validator.
2. Extract line items with a strict schema. Description as printed, quantity, unit price, extended amount, and a flag for whether the row is a discount or a return. Keep the printed description verbatim; abbreviations like CHIX SAND are the classifier’s input and paraphrasing them destroys information.
3. Classify against the enum, constrained. Inline the valid category_id values in the request and require the output to be one of them, using whatever constrained-decoding or strict schema support the provider offers — see structured output support. An unconstrained model will invent a plausible category name that is not in your table, and the failure surfaces three systems later.
4. Assert the items sum to the subtotal. Before allocation. If the categorised items do not account for the whole subtotal, something was dropped and the allocation below will be wrong in a way that is hard to see afterwards.
5. Allocate tax and tip pro rata. The step almost everyone skips. Tax and gratuity belong to the categories in proportion to the item amounts they were charged on, not to a “Tax” bucket of their own.
6. Route the low-confidence rows. Per-row, not per-receipt. One ambiguous line on a twelve-line receipt should not send the whole document to a human.

subtotal = 84.00
items = [
  ("Grilled chicken",   32.00, "meals"),
  ("House salad",       18.00, "meals"),
  ("Bottle of wine",    34.00, "meals_alcohol"),
]
tax = 7.14
tip = 15.00

# pro-rata share by item amount
for name, amount, category in items:
    share      = amount / subtotal
    alloc_tax  = round(tax * share, 2)
    alloc_tip  = round(tip * share, 2)
    total      = amount + alloc_tax + alloc_tip

# and then, because rounding does not sum:
#   push the residual cents onto the largest line, do not drop them
assert sum(allocated_totals) == subtotal + tax + tip

That last assertion is not decoration. Rounding three shares to the cent will usually miss the true total by a cent or two, and a pipeline that ignores it produces expense reports that do not tie out to the card statement — a difference small enough to be invisible and persistent enough to be maddening.

One receipt, several categories

A supermarket receipt on a business trip has groceries, a phone charger, and a bag of ice. A hotel folio has room nights, a resort fee, a restaurant charge posted from downstairs, parking and a movie. Those are four or five different expense lines on one document with one card charge, and the merchant tells you nothing about which is which — the merchant category applies to the business, not to what was bought, which is exactly the limitation described in extracting merchant category from a receipt.

So the unit of categorisation has to be the line, and the receipt is a container. That has one useful consequence: the container carries the date, the merchant and the payment, and the lines carry the categories, so a hotel folio produces one payment record and five expense records that sum to it. Modelling it the other way round — one category per receipt, picked from the dominant line — is where the mis-filed restaurant charge inside a hotel bill comes from.

Where the mapping breaks

• Ambiguous descriptions. MISC MDSE 1 is a real line on a real receipt and it means nothing. There has to be an uncategorised bucket; forcing every line into a category produces confident nonsense.
• Store-brand abbreviations. Truncated at twelve or sixteen characters by the register. The classifier sees GV SHRD MZRLA and has to work with it; supplying a few real examples from that merchant is worth more than a longer instruction.
• Deposits and returns. A bottle deposit and a returned item are negative or refundable amounts that must not be categorised as expenditure. They also break the subtotal check if their sign is dropped.
• Personal items on a business receipt. Common, legitimate, and the reason the line is the unit. The split is a policy decision your table should be able to express, not something to hide inside a prompt.
• Multi-currency. A foreign receipt has a local subtotal, a possible dynamic-currency-conversion amount printed alongside, and a settled amount in the home currency that matches neither. Store all three and the rate used; the card statement is authoritative for the last.