Extracting Diagnosis Codes From a Medical Record

A chart contains diagnoses in two completely different forms. Some are printed as codes, and extracting those is transcription with a format check. Some exist only as a clinician’s sentence, and turning those into codes is medical coding — a regulated judgement with rules that sometimes say the code must not be assigned at all.

What an ICD-10-CM code looks like

ICD-10-CM is maintained by the National Center for Health Statistics at the CDC, with CMS, and revised annually with an effective date of 1 October. A code is three to seven characters. The first is a letter, the second is a digit, and the third through seventh may be letters or digits. A decimal point is written after the third character when anything follows it. So I10 is a complete, valid code with no decimal at all; E11.9 has a fourth character; and S52.501A runs to seven.

The first three characters are the category. Characters four to six add etiology, anatomic site, severity and laterality. The seventh, where a code has one, is an extension whose meaning is fixed by the chapter rather than by the code.

Two structural details cause more extraction bugs than anything else about the format. The first is that letters and digits genuinely mix, so a validator that treats characters four onward as numeric rejects real codes — and an OCR pass that helpfully corrects the letter O to a zero, or the letter I to a one, produces a code that is still well-formed and is now a different disease. The second is the placeholder X. When a code requires a seventh character but is shorter than six, the empty positions are filled with X so the extension lands in position seven. A stripped or trimmed X silently changes the code’s meaning, and the string still passes a naive length check.

Valid, and all real shapes:
  I10          essential (primary) hypertension — three characters, no decimal
  E11.9        type 2 diabetes mellitus without complications
  M17.11       unilateral primary osteoarthritis, right knee
  S52.501A     unspecified fracture of lower end of right radius,
               initial encounter for closed fracture

Regex for the shape only — it does not prove the code exists:
  ^[A-TV-Z][0-9][0-9A-Z](\.[0-9A-Z]{1,4})?$

That pattern is a shape test, not a membership test. The only authority on whether a code exists in a given fiscal year is the published code set for that year, and codes are added, deleted and redefined every October. Validate against the release that was current on the date of service, not against the release you downloaded last week; a chart from two years ago legitimately contains codes that are no longer in the current file.

Laterality and the seventh character

Laterality is where extraction quietly loses information. In the chapters that carry it — musculoskeletal, eye, ear, injury — the side is encoded in a character, not in a separate field, and the conventional values are 1 for right, 2 for left, 9 for unspecified, with a distinct value for bilateral where the code set provides one.M17.11 and M17.12 differ by exactly one character and describe two different knees.

This matters because it collides with two other things in the same document. A radiology report or an operative note usually states the side in prose, in a different sentence from the code, and the two can disagree — a genuine, common discrepancy that is worth surfacing rather than silently resolving. And the printed code frequently ends in the unspecified value even when the narrative is explicit, because the coder worked from documentation that did not support the specific code at the time. Overwriting the coded value with what the prose says is not a correction; it is you making a coding decision.

The seventh character is the other place meaning hides. On injury codes it distinguishes the episode of care — an initial encounter, a subsequent encounter, a sequela — and on fracture codes it also carries healing status. Two records with identical first six characters and different seventh characters are not duplicates, and a deduplication step keyed on the category will merge them.

Three places a diagnosis lives

Before writing a prompt, work out which of these you are actually being asked for, because they answer different questions and routinely disagree with each other inside one document.

• The coded problem list. A structured list, usually printed as a table with a code, a description and an onset date. It is a longitudinal record of conditions the patient has, and it is frequently stale — resolved problems linger for years.
• Encounter or billing diagnoses. The codes attached to this visit, often on the last page or in a footer, sometimes with a rank order where the first is the principal diagnosis. These are the closest thing in the document to a claim.
• The narrative. Assessment and plan, history of present illness, past medical history. This is where a condition appears as a sentence and nowhere else, and it is the only one of the three that has no code to transcribe.

Your schema needs a source field distinguishing them, and it needs to allow the same condition to appear more than once with different codes. Collapsing all three into one array of diagnoses destroys the distinction between “treated today” and “has a history of”, which is usually the distinction the downstream consumer cared about.

Negation, history and rule-out

Clinical narrative is dense with statements about conditions the patient does not have. A model asked to “list the diagnoses” will happily return pneumonia from the sentence “no evidence of pneumonia”, and this is the single most common failure in the task. The classic mitigation is not a prompt: it is the NegEx and ConText approach from the clinical NLP literature, which scopes a small set of trigger phrases — no evidence of, denies, ruled out, negative for — forward or backward to a sentence boundary or a conjunction. Running that as a deterministic post-check over the span the model cited is cheap and catches what a prompt does not.

Four modifiers need their own boolean fields rather than being folded into the text: negated, historical, hypothetical (“return if chest pain develops”), and attributed to someone other than the patient (“mother with breast cancer”). Family history in particular has its own ICD-10-CM Z codes and is a different assertion about a different person; extracting it as a patient diagnosis is wrong in a way that survives every schema check you can write.

Then there is uncertainty, and here the rule is external. The ICD-10-CM Official Guidelines for Coding and Reporting, published alongside the code set by NCHS and CMS, treat uncertain diagnoses differently by setting. For outpatient encounters the guidelines direct that conditions documented as probable, suspected, questionable or rule-out are not coded as though they exist — you code the signs and symptoms instead. For inpatient records an uncertain diagnosis documented at discharge may be coded as if established. So the same sentence produces a different correct answer depending on what kind of encounter the document records, and any pipeline that resolves “probable cellulitis” to a code without knowing the setting is guessing.

Transcribe, or flag for a coder

The practical design follows directly. Split the task in two and give each half a different output contract.

Where a code is printed, you are transcribing. Extract the code string, the printed description, the source section, and a character offset or bounding box you can point at later. Validate the shape, validate membership in the code set for the date of service, and compare the printed description to the official description for that code — a mismatch usually means an OCR error in the code or a stale description in the source system, and both are worth a human look.

Where a diagnosis exists only in prose, do not emit a code as though it were extracted. Emit the verbatim span, the negation and historicity flags, and — if you want the assist — a candidate code clearly typed as a suggestion with its own field name, never merged into the same array as transcribed codes. Assigning a diagnosis code from documentation is a coding decision made by a person qualified to make it, and the value of the pipeline is that it arrives on their desk with the span already located. Building the queue for that is covered in how much of the output to review, and the threshold at which a field is routed there in confidence threshold review routing.

One handling note that belongs on this page specifically. A diagnosis list is among the most sensitive fields in a record, and HIPAA’s minimum necessary standard applies to what you request as much as to what you disclose. If the downstream consumer needs a count of patients with a condition, it does not need the narrative that mentions it. Extract the smallest field set that answers the question, and keep the span offsets in a separate store from the codes so a reviewer who needs context can get it, the way a review queue highlights its source, without every consumer of the codes inheriting the surrounding text.