Extracting Cited Prior Art From a Patent Document

The references cited on a patent front page look like one list. They are three, with three different internal structures, and a schema with a single citation string throws away the distinction that makes the data useful.

One heading, three lists

The block carries INID code (56) and is conventionally divided into headed sub-lists: United States patent documents, foreign patent documents, and other publications — the last being non-patent literature, usually abbreviated NPL. On United States grants each sub-list has its own column layout, and the whole block frequently continues onto a second page when it is long, with the columns shifting where it does — the same continuation problem as column misalignment across pages. Long here can mean hundreds of entries.

Extract the block by finding (56) and then reading its sub-headings, rather than by pattern-matching citations across the page. The sub-heading is what tells you which sub-schema applies, and you lose it if you flatten first and classify afterwards; a bare 2004/0123456 could be a US pre-grant publication or the start of a European document number depending on which list it sat under.

Then keep the whole raw entry alongside the parsed fields, always. NPL in particular cannot be reliably decomposed, and the raw string is the only faithful record. The schema below reflects that:

{
  "citation_type": "us_patent",
  "raw": "5,123,456 A   6/1992   Sorensen et al.   ......  703/2",
  "number": "5123456",
  "kind_code": "A",
  "date": "1992-06",
  "first_named_inventor": "Sorensen",
  "cited_by": "applicant",
  "sublist_heading": "U.S. PATENT DOCUMENTS"
}

US patent documents and number formats

The United States list gives a document number, a kind code, a date, the surname of the first-named inventor and often a classification. The number formats are the part that catches people out, because several series coexist and only one of them is a plain integer.

• Utility patents are numbered sequentially and printed with thousands separators: 5,123,456. Strip them for storage, but store the digits as a string — leading zeros appear in some padded representations and integer storage loses them.
• Design patents carry a D prefix, reissues RE, plant patents PP. These are separate series, so a design patent and a utility patent can share the digits. The prefix is part of the identifier and is not decoration.
• Pre-grant publications are formatted quite differently — a four-digit year, a slash and a seven-digit sequence, as 2004/0123456 — and refer to a published application, not a granted patent. Mixing the two series in one field produces joins that appear to work and connect the wrong documents.

Where a classification symbol is printed alongside the citation, it belongs to a classification scheme that is itself revised — the Cooperative Patent Classification is updated on a regular cycle — so store the symbol as printed and record that it was as of the document’s publication rather than resolving it against today’s scheme.

Foreign documents, country codes and kind codes

A foreign citation is a country code, a publication number, a kind code and a date. Both code systems are standardised and both are published by WIPO alongside the INID standard, at the WIPO standards library: ST.3 gives the two-letter codes for offices, ST.16 gives the kind codes.

The country codes mostly look like ISO country codes and mostly are, but the set is not the same. It includes offices rather than only countries — EP for the European Patent Office and WO for international applications published under the PCT — and it retains historical codes for states that no longer exist, which appear routinely in prior art precisely because prior art is old. Validating extracted country codes against a modern ISO list will reject correct citations.

Kind codes are the subtler trap. A, A1, B1, B2, U and others appear across offices, and their meanings are office-specific. Broadly, an A-series code marks a publication of an application and a B-series code a granted document, but the details differ: whether a B1 was preceded by an A publication, what a second-level code means, and what a U designates — in Germany it marks a Gebrauchsmuster, a utility model, which is a different right from a patent. So a kind code is only interpretable in combination with its country code, and a lookup table keyed on the kind code alone is wrong for some offices no matter how it is populated.

Store country, number and kind as three fields and interpret them together. If you need a single canonical identifier, the conventional form concatenates them — country, number, kind — but derive it, do not extract it.

Non-patent literature does not decompose

The “Other Publications” list is free text. Entries are whatever the applicant or examiner typed, in whatever citation style, frequently truncated to fit the column, and they are not all journal articles. The list routinely contains international search reports and written opinions, office actions from related applications, standards documents, product manuals, conference papers, theses, database records and web pages with retrieval dates.

Attempting to force these into a bibliographic schema produces confident nonsense: a search report’s application number parsed as a volume, a standard’s designation parsed as an author. The workable design is a small typed enumeration — article, search report, office action, standard, book, web page, unknown — with optional fields populated only where the entry clearly supports them, and unknown as a perfectly acceptable answer with the raw string intact. That is schema design for variants you have not seen in its plainest form: the list will contain a document type nobody anticipated, and the schema has to hold it rather than reject it.

Two mechanical hazards specific to this list. Entries wrap across several printed lines in a narrow column, so a line-based reader produces fragments; entry boundaries have to come from the layout, not from newlines. And truncation is real — a title cut off with an ellipsis is cut off in the source document, and an extraction that silently completes it has invented text. Instruct against completion explicitly and keep the ellipsis; the general point about models filling in what they cannot see is in vision hallucination.

Who cited it, and what the list leaves out

United States grants print a legend under the references block explaining symbols that mark how a reference entered the record — conventionally an asterisk for a reference cited by the examiner and a dagger for one submitted by a third party, with unmarked entries having come from the applicant. Read the legend as printed on the document you are processing rather than assuming, since the symbols and the legend text are an office convention rather than a universal one.

The distinction is worth capturing because it changes what a citation means. An applicant-supplied reference reflects what the applicant knew and disclosed; an examiner-supplied one reflects what the office found during search and is more likely to have been used in a rejection. Analyses that treat all citations as equivalent are mixing two different signals, and the symbol that separates them is a single character that OCR loses easily — another reason to keep the raw entry string with its leading characters intact.

Finally, be clear about what this list is not. It is the references printed on the face of the document, which is a subset of the prosecution record: the complete set of what was considered lives in the information disclosure statements and the file wrapper held by the office. And it contains only backward citations. Which later patents cite this one is not on the document at any point and has to come from an office or commercial dataset. A schema field named citations with no direction is an invitation to conflate the two, so name it for what it holds — the references this document cites, as printed — and let the forward set arrive from somewhere else.