Protecting Weights You Shipped to Someone Else's Hardware

If your weights are on a device somebody else owns, a sufficiently motivated person will obtain them. Every protection available raises the cost of doing so; none of them makes it impossible. Designing on that premise produces better decisions than designing on the hope that this time it will be different.

The premise nobody wants to state

The reason is structural rather than a weakness in any particular scheme. To run a model, the hardware must eventually hold the actual weight values in memory in a form arithmetic can be performed on. Whatever key decrypts them, the device must be able to obtain that key without your help, because it runs offline and in the user’s hands. So the key is on the device too, and an attacker who controls the device controls both.

This is the same argument that applies to every form of client-side content protection, and it has the same conclusion: the objective is not prevention but cost. That reframing is useful, because cost is something you can reason about and trade against effort, while prevention is something you can only fail at.

Attacker tiers, and what each costs

Decide which tier you are defending against before choosing anything, because the measures that stop one tier are irrelevant to the next.

Most published discussion of protecting on-device models addresses tier 0 and is written as though it addressed tier 2. Being explicit about which tier a measure stops is the single most useful thing you can do in a design review.

What each measure actually buys

• Encryption at rest. Stops tier 0 completely and costs almost nothing. Always do it. Understand that the key must be recoverable by the app, so it stops nobody above tier 0 — and be careful that decrypting to a temporary file on disk does not undo the entire measure, which is a common implementation mistake.
• Key storage in the platform keystore. Better than a constant in the binary, because extraction now requires the device to cooperate rather than a hex editor. Raises tier 1 to real work. Hardware-backed keystores can make the key itself non-exportable — but the app can still ask for a decryption, so an attacker in the app’s context gets the plaintext anyway.
• Obfuscation, packing and anti-debugging. Buys time against tier 1 and tier 2 and nothing else. It costs you real engineering, makes crash reports harder to read, and can trigger security software. Worth it only if a delay of weeks has commercial value to you.
• Integrity and root detection. Detects a modified application or a compromised device and refuses to run. Reasonable as a layer, and it is an arms race you do not win permanently. It also has a real cost in false positives against legitimate users on unusual configurations.
• Splitting the model. Ship most layers and keep a small, high-value part server-side. This genuinely defeats a file-copying attacker, because what they took is incomplete. It costs you the offline capability that was probably the reason for going on-device, so it is a trade rather than a win.
• Licence terms. Not a technical control and not nothing. A clear licence is what makes commercial redistribution a legal problem for the redistributor, and it is what your own model licensing position has to be coherent with in the first place.

Secure hardware, and its real limits

Trusted execution environments and secure enclaves are frequently proposed as the answer. They are a real mechanism with real limitations, and the limitations decide the matter:

• Capacity. Secure enclaves are designed for keys and small sensitive operations, not for gigabytes of weights and the memory bandwidth a model needs. Running an entire language model inside one is not, in general, available to you.
• Accelerator access. The NPU and GPU that make on-device inference viable generally sit outside the secure boundary. Protecting the weights inside the enclave and then handing them to an accelerator outside it re-exposes them.
• Availability. Platform-provided model encryption, where the operating system holds the key and the application never sees it, is a genuinely stronger position — where it exists. Its availability and its guarantees are platform- and version-specific, so check what your actual deployment targets offer rather than designing for a capability that covers a third of your fleet.
• Published attacks exist. Trusted execution environments have been broken repeatedly by academic and commercial researchers. They raise cost substantially. They are not a boundary you should treat as absolute, which is the same conclusion reached in confidential computing for inference.

Where to put the value instead

The productive response is to arrange things so that the weights are not the asset. That is a design decision available to almost everyone, and it is more durable than any control.

1. Ship the commodity part. If the on-device model is a fine-tune of a widely available base, an attacker who extracts it has obtained your fine-tune — meaningful, but far less than obtaining something they could not otherwise approximate. Keep genuinely differentiated capability server-side.
2. Put the moat in the data and the loop. A model is a snapshot. The pipeline that produced it, the evaluation set that proves it works, and the feedback that improves it next month are not on the device and cannot be copied from it. Teams that treat the checkpoint as the asset are protecting the least durable thing they own.
3. Make the product the integration. For most applications the value is the surrounding experience — the data it is connected to, the workflow it fits, the reliability. A competitor with your weights and none of that has a file.
4. Update faster than extraction pays. If your model improves meaningfully every quarter, a stolen copy depreciates. That is a genuine defence and it is one you were going to build anyway.
5. Be honest in the business case. If a project only works if the weights stay secret, and the weights are going on consumer hardware, the project has a problem that no amount of obfuscation solves. Better to find that out during planning than after launch.

Proving it was yours, afterwards

Since prevention is not achievable, attribution is worth something — particularly if your recourse is commercial or legal rather than technical.

• Fingerprint each distributed build. Small, behaviour-preserving variations across distribution channels or licensees mean a recovered copy can be traced to where it leaked. This is cheap and it is the measure most likely to actually be useful.
• Embed a verifiable signature in behaviour. A model can be trained to respond in a specific, innocuous way to a specific improbable input. That response survives redistribution and, unlike a file hash, survives re-quantisation and format conversion — which is exactly what somebody covering their tracks will do.
• Record what you shipped, with hashes and dates. The provenance record is what turns “that looks like ours” into an argument somebody else has to answer. The same provenance discipline described in tracking where training data came from applies to what leaves your build system.
• Know the limits of watermarking. Behavioural signatures can be removed by further fine-tuning, and the general robustness picture is mixed — the honest assessment in does AI watermarking work applies here too. Treat it as evidence, not as proof, and size your expectations accordingly.