HIPAA and LLMs: What Is Actually Possible

Searching for a HIPAA-compliant language model returns a great many confident answers, and the premise is wrong. Compliance is a property of an organisation and its practices, not of a piece of software, and no model is compliant or non-compliant any more than a database is.

This is engineering guidance and not legal or regulatory advice. US health privacy law is detailed and fact-specific, and anyone handling protected health information needs advice on their own arrangement. What this page can give you is the shape of the decision.

Why the phrase is a category error

The rules apply to covered entities — providers, health plans, clearinghouses — and to business associates who handle protected health information on their behalf. Software is a tool used within that arrangement. A vendor can be willing to enter a business associate agreement, can operate controls that support your compliance, and can configure a service so that it is usable in a regulated workflow. None of that makes the model compliant, and any vendor using the phrase about their own product is telling you something about their precision.

The corollary is that your question is never “is this model compliant”. It is: will this vendor sign a business associate agreement covering this specific service, and can I evidence the controls that agreement assumes.

The business associate relationship

Where a vendor processes protected health information for you, the arrangement is normally governed by a business associate agreement. Two practical points that engineers should carry into the conversation.

First, coverage is per service, not per company. A vendor may be willing to cover some products and not others, and the exclusions are usually the newest features — precisely the ones a team wants to use. Get the covered-service list in writing and check that the specific endpoints and models you call are on it, because a general assurance from a salesperson is not the same document.

Second, the agreement flows down. Everyone in the chain who touches the data needs to be inside the arrangement, which makes the sub-processor question from the DPA page load-bearing here rather than merely diligent. Brokered inference, where the party you contracted with forwards your text to a party chosen at request time, is the case to interrogate hardest.

Never assume the answer for a named vendor from anything you read online, including this page. Ask them, get it in the contract, and re-check when you add a model.

The other road: de-identification

If the data is de-identified to the required standard, it is no longer protected health information and the analysis changes fundamentally. Two recognised routes exist. One is expert determination: a qualified person applies statistical methods and documents that the re-identification risk is very small. The other is the safe-harbour method, which removes a specified list of identifier types — names, geographic subdivisions below a certain granularity, dates more precise than a year, contact details, account and record numbers, device identifiers, biometrics, full-face images, and any other unique identifying number or code — subject to the condition that you have no actual knowledge that what remains could identify the individual.

Safe harbour looks mechanical and is genuinely hard on free text. Clinical notes are dense with dates, ages, place names and narrative detail, and the residual catch-all provision means a technically complete removal can still fail if you know the remainder is identifying. Removing every date from a clinical record also tends to destroy the clinical meaning, which is the trade nobody mentions in the architecture diagram.

Two architectures that work

• Covered path. Protected health information goes to a vendor operating under a business associate agreement, with the configuration that agreement requires — retention settings, access restrictions, logging — and the whole chain covered. Simpler to reason about, restricts your model choice to vendors willing to sign, and puts the burden on contracts and evidence.
• De-identified path. Identifiers are removed or pseudonymised before anything leaves your boundary, and only the de-identified text is sent. Opens up the model catalogue, and moves the burden onto your redaction pipeline, its failure rate, and your documentation of the method. See reversible pseudonymisation for the round trip that keeps the output usable.

Teams often want both: a covered path for the clinical workflow and a de-identified path for everything else. That is reasonable, and it requires the two paths to be separated at a level a mistake cannot cross — different credentials, different services, different network egress if you can manage it. A single environment variable deciding which path a request takes is not that separation.

Questions for a vendor

• Will you enter a business associate agreement, and which services, endpoints and models does it cover today?
• Which sub-processors are in the path for those models, and are they inside the agreement?
• What is retained, for how long, and does content flagged by safety classification follow a different path?
• Can staff read request content, under what circumstances, and what record is kept of it?
• What must we configure for the agreement’s assumptions to hold, and how do we verify from our side that it is configured?
• What is your breach notification commitment and its timeline?

The last two are the ones teams forget. An agreement that assumes a setting you never applied protects nobody, and a notification timeline that does not fit inside your own reporting obligation is a problem you find out about on the worst possible day.

There is also a minimum-necessary principle running underneath all of this that engineers can act on directly without waiting for a contract to be signed. If the task is classifying a message into one of eight categories, the model does not need the patient’s name, address or record number to do it, and sending them because they happened to be in the same object is the sort of habit that turns a modest arrangement into a large one. Build the request payload explicitly, field by field, rather than serialising whatever record you already had in memory. It is a five-minute change that reduces the surface of every subsequent conversation on this page.

One last framing worth carrying into the meeting where this gets decided: nothing about health data forces a single answer. A covered path and a de-identified path are both legitimate, an on-premise deployment is a third option with its own arithmetic, and the right choice depends on how much of your traffic actually carries protected health information. Segment the traffic before you choose the architecture, because the answer for the clinical workflow is rarely the right answer for the other eighty per cent of what the product does.