Confidential Computing for Inference

Confidential computing promises that the operator of the machine your workload runs on cannot see inside it. Applied to inference that is a genuinely interesting property, and it is also the kind of claim where the difference between the strong and weak versions is invisible unless you know what to ask.

Engineering guidance rather than legal advice. Nothing here says whether a given technology satisfies any obligation you are under — that depends on the obligation, and the answer is frequently that it helps without being sufficient.

The threat it addresses

Ordinary cloud security protects your data from other tenants and from outsiders. It does not protect it from the platform: a sufficiently privileged operator, or code running in the hypervisor, can in principle read the memory of a running workload. For most workloads that residual trust is accepted and managed contractually.

A trusted execution environment narrows it technically. Memory is encrypted with keys held by the processor, the hypervisor and host operating system are excluded from the trust boundary, and the hardware can produce a signed statement about what is running inside. The threat model being addressed is specifically the infrastructure operator — which is exactly the party a residency or confidentiality conversation keeps running into.

What an attestation actually says

Attestation is the load-bearing part, and it is narrower than the word suggests. A report signed by a key rooted in the hardware asserts, roughly: this is genuine hardware of this type, at this firmware level, with these security features enabled, and the code image loaded into the protected environment hashes to this value.

Read that carefully. It says the loaded image has a particular hash. It says nothing whatever about whether that image is trustworthy. If you do not know what the expected hash corresponds to — if you cannot map it to source you can inspect or to a build you can reproduce — then the attestation has proved that something ran unobserved, which is a different and much weaker property than proving the right thing ran.

This is why the meaningful question is never “is it attested”. It is: can I verify the attestation myself, against a measurement I can independently associate with code I can examine, at the time I use the service, before sending data.

Why inference makes it harder

The mature part of this technology is CPU-based. Inference runs on accelerators, which introduces the problem that the interesting data spends its time in GPU memory rather than in the protected CPU enclave. Confidential computing for accelerators exists and is advancing, and it requires the protection to extend across the whole path: the CPU environment, the link between CPU and accelerator, and the accelerator’s own memory, with the accelerator participating in attestation rather than being trusted implicitly.

When evaluating any offering, that path is the thing to trace. A protected CPU enclave that hands plaintext activations to an unprotected accelerator has moved the exposure rather than removed it, and the description of such a system can be entirely accurate at every individual step.

Expect a performance and availability cost too. Encrypted memory and attested boot are not free, hardware support is a subset of available capacity, and the constraint on which models can be served in such an environment is real. Ask what it costs in throughput and in catalogue, because a vendor who has measured it will tell you and one who has not is describing an aspiration.

What stays outside the boundary

• The code you attested. A trusted environment running a service that logs your prompts to disk is a trusted environment doing exactly that, faithfully and confidentially.
• Metadata and traffic patterns. Request sizes, timings and frequencies are visible outside the enclave and can be informative on their own.
• The output path. Data is protected in use; what happens to the completion after it leaves is a normal engineering question with normal answers.
• Side channels. This is an area of continuing research, and specific attacks against specific implementations have been published over the years, with mitigations following. Treat it as a strong control with a research frontier, not as an absolute.
• The verifier. If the attestation is checked by the same party operating the service, and you only see the result, the chain of trust has a link in it you did not inspect.

Questions that separate the claims

• Which components are inside the trust boundary — CPU only, or CPU and accelerator including the link between them?
• Can we obtain and verify the attestation ourselves, and is that available per request or only at provisioning?
• What is the expected measurement, and how do we associate it with code we can inspect or a build we can reproduce?
• What is the key release policy — is the key that decrypts our data released only against a successful attestation, and who controls the policy that defines success?
• Which models can be served in this configuration, and what is the throughput cost relative to the unprotected path?
• What happens on attestation failure: does the request fail, or fall back to an unprotected path?

The last one is the sharpest. A silent fallback converts the entire guarantee into a best-effort feature, and it is the sort of behaviour that gets added for availability reasons by somebody who was not in the security conversation. If the answer is that it fails closed, you are looking at a real control; if it is anything else, you are looking at an optimisation.