Private Inference Beat

Comparison

Who can read your prompts and replies, and what does the client verify? Compare the evidence and unfinished checks for each request path.

Expand a provider for its paths. What private inference requires · Companion audit findings · Stage criteria (proposed). Dated assessments; no stages awarded.

Cells name the finding or our remaining work · select one for evidence and success criteria

Scroll sideways; provider names stay visible.

Provider / pathHardwareSessionCodeOperator accessDownstreamUpdatesCollectionNew referenceReview
3 paths Our boot check missingOur backend test to doOur rebuild proof to doRoutes differRoutes differApproval not established 3/3 returned 10-224 changed since review 10-14
3 paths Routes differRoutes differRoutes differOur access review to doRoutes differRoutes differ 1/2 returned 10-303 changed since review 10-14
1 path Our AMD check missingOur session proof to doOur rebuild proof to doOur re-review to doOur worker test to doOur refusal test to do 1/1 returned 10-292 changed since review 10-14
2 paths Our hardware test to doOur session test to doOur code proof to doOur access review to doOur worker test to doOur policy test to do 4/5 returned No history 10-14
1 path Our GPU/RTMR checks missingOur request test to doServing code unmeasuredCode can change unmeasuredModel not committedProvider set only 5/6 returned 10-271 changed since review 10-14
3 paths Routes differRoutes differRoutes differRoutes differRoutes differRoutes differ Not sampled 10-303 changed since review 10-14
2 paths Our hardware test to doOur session test to doOur EIF proof to doOur access review to doOur worker test to doOur policy test to do Not sampled 10-2716 changed since review 10-14
3 paths Our hardware test to doRoutes differOur code proof to doOur access review to doOur worker test to doRoutes differ Not sampled 10-292 changed since review 10-14
1 path Our hardware test to doOur session test to doOur release check to doOperator access pathsOur worker test to doOur refusal test to do Not sampled 10-304 changed since review 10-14

9 providers · 19 paths · Dates refer to source review. Cells name documented system findings or work still missing in our method. Select one for evidence, limits and a linked task. Collection counts do not measure privacy.

Reference dates are provider-wide first sightings of new sets. Later reference differences flag review work; they never authorize a new value. Explore reference history →

All 17 model / endpoint measurements →

What makes inference private?

A trusted execution environment (TEE) can protect memory from the machine's host. A private inference client also needs to establish which code receives plaintext, what that code can do, and whether the actual prompt and reply use the verified keys. A genuine enclave can still run operator-controlled code that logs prompts.

1. Authenticate the environment

Check hardware evidence, its trust roots and security state. Bind it to the connection's keys and establish freshness.

2. Identify the plaintext-handling code

Compare authenticated measurements with expected software and configuration. Include the serving application, model where claimed, and every necessary worker hop.

3. Establish who can change it

Inspect operator access, unmeasured inputs and client update policy. A provider publishing a new hash does not mean your policy accepts it.

The table separates these obligations. A missing check in our adapter describes our method. A documented protection gap describes the reviewed system. Open a cell for the evidence, the exact missing obligation and its scope; method work is linked to issues with completion criteria. Collection describes returned attestation evidence, not inference uptime or privacy.

The Reference values explorer follows identities and policies over time. Audits explain what those values cover. Neither a stable hash nor a fresh observation renews an audit.

What the companion audits establish

The DevProof case studies ask whether a user can verify the limits on a service operator's access and ability to change software. Beat connects that analysis to route assessments and retained reference history. These selected lessons explain the evidence behind the caution, including strengths that a generic label would hide.

Selected lessons from committed companion DevProof reports. Dates describe their evidence; this synthesis is not a fresh deployment audit and does not import their stage ratings. Curated 2026-10-14.

Chutes: genuine hardware can still run unmeasured operator code

The companion study credits genuine TDX, key binding and protection from the miner host. Its application finding is different: operator-supplied serving code received decrypted prompts without that code being identified by the client's measured references.

Evidence 2026-06-16/17 · Pinned report and evidence ↗ · chutes-tee

Finding and limits

Reviewed scope: Controlled demonstration on the author's own deployed chute, plus SDK source review; consumer trust in the chute operator.

  • Operator-added middleware collected controlled callers' decrypted prompts and returned them to another controlled caller through the ordinary encrypted reply channel.
  • A matching base-image measurement did not distinguish that serving code. Preventing network egress alone did not close the cross-request channel.

Limits: This was not an observed breach of third-party production users. It establishes the demonstrated operator capability under the reviewed boundary; today's workload and any remediation need separate evidence. The daily registry adapter also omits GPU appraisal, a limitation of our instrument rather than this application finding.

Consumer report and threat model ↗Hardware strengths and miner containment ↗

Tinfoil: trace what an operator-controlled setting can actually affect

The report separated the router from model enclaves and traced two externally supplied router settings through the code. For the reviewed deployment, it found those settings off the prompt path; the examined model configurations had no equivalent external slots.

Evidence 2026-04-26 · Pinned report and evidence ↗ · tinfoil-router

Finding and limits

Reviewed scope: Dated managed-inference deployment/configuration inspection and router source trace; two listed model enclaves were unreachable for direct inspection.

  • The hostname setting and billing-HMAC secret had different effects from a hypothetical setting that redirects or logs plaintext; the mere existence of an external input did not establish prompt exposure.
  • The report distinguished AMD verification in Tinfoil's reference Go verifier from the missing VCEK-chain check in this repository's Python adapter.

Limits: Those configuration conclusions apply to the examined versions. They do not complete our present hardware checks, independently reproduce every build, or prove an unobserved worker. The report's historical stage language is not a stage award by this site.

ACI gateway: a valid gateway quote is not a model attestation

The August study compared ACI's explicit gateway and session evidence with a legacy endpoint that accepted arbitrary model names while returning the gateway's quote. The same genuine hardware evidence could be interpreted as a stronger claim than it supported.

Evidence 2026-08-18 · Pinned report and evidence ↗ · aci-gateway-tee-redpill · aci-gateway-phala · aci-gateway-api-redpill

Finding and limits

Reviewed scope: Dated shared RedPill/Phala gateway deployment, protocol/source inspection, attestation and receipt checks.

  • Naming a model in an attestation URL did not bind that model to the quoted gateway workload.
  • A client's proof must follow the selected upstream and actual request; gateway identity, route policy and worker identity are different obligations.

Limits: The legacy-endpoint finding is historical. This repository now probes ACI hostnames and separately records remaining boot-policy and downstream-proof gaps. The earlier report's broader verification language does not close those instrument gaps.

Phala verifier: inspect what a successful result actually checks

The April source review found attestation checks without a complete encrypted client exchange. It also identified checks that compared server-supplied fields without closing the intended hardware or live-channel binding.

Evidence 2026-04-24 · Pinned report and evidence ↗

Finding and limits

Reviewed scope: Source review of Phala private-ai-verifier at revision 2d2f5cf; not a verdict on every later SDK or every provider using Phala infrastructure.

  • A verifier's success flag needs an explicit contract: which signatures, fields, trust roots and live session are actually checked?
  • Authenticating some attestation evidence is insufficient if the client sends its prompt to a key or connection outside that verified chain.

Limits: This is a dated verifier study. Current implementations must be inspected independently; the registry applies the same criticism to its own adapters and tracks their unfinished checks separately.