Private Inference Beat

Verifiers

Clients that verify TEE attestation for private inference, reviewed from source at a pinned commit. A cell grades what the client establishes on that provider: 'serving' = reaches the code that reads the prompt and blocks before sending; 'partial' = checks real evidence but stops short of the serving code or doesn't block; 'weak' = little is authenticated. No cell is 'serving' today. Reviewed 2026-11-15; each row links its review with file:line citations.

Two questions recur in every review: are event-log values hashed before they are trusted (the dstack RTMR3 replay must recompute each digest from its payload), and does a failed check stop the prompt from being sent?

serving reaches the serving code, blocks before sending partial real evidence, stops short of the serving code or doesn't block weak little is authenticated — not supported

Clients and proxies

Client NEAR AITinfoilPhala Cloud (ACI)ChutesVeniceNanoGPTPrivatemodeIts own service Event log hashedBlocks on failure
teep 6548996 · review partial
Every instance appraised, E2EE v2 to the attested model key, static compose pin. Deployment log not checked (#166).
partial
Router only; latest Sigstore release adopted. Direct mode disabled 2026-11-12.
partial
Gateway and sessions pinned; receipt verified after the stream. Upstream workload policy not checked.
partial
MRTD/MRSEAM pinned; ML-KEM to the instance key. Serving code is outside the measurement.
partial
Legacy dstack E2EE blocks. ACI/1: gateway only, reply forgeable.
weak
E2EE off by default, so NanoGPT reads the prompt.
— — yes yes
pap (ACI CLI) e9b6ae8a · review — — partial
TLS pinned to the attested keyset; receipt checked after printing. Key custody only with two flags.
— — — — — n/a no
jooray venice-e2ee-proxy 4b5602ef · review — — — — partial
Refuses GPU models without NVIDIA evidence. If DCAP fails to load it warns and continues. Doesn't parse ACI/1.
— — — n/a yes
AxLabs venice-e2ee-proxy 70ef39f6 · review — — — — weak
Trusts Venice's own GPU verdict; missing GPU evidence only logs a warning.
— — — n/a no
clawdbotatg venice-e2ee-proxy 4311d554 · review — — — — weak
Trusts Venice's verified flag (attestation.ts:66); passes plaintext replies and empties chunks that fail to decrypt.
— — — n/a no
M0dEx venice-e2ee-proxy 0999259d · review — — — — weak
No DCAP or NVIDIA check linked; nonce checked in JSON only; Venice's verified flag is the real gate.
— — — n/a no
Privatemode proxy 3c98e0ab · review — — — — — — partial
Default manifest comes from a CDN, unsigned and auto-adopted; pin it with --manifest-path.
— n/a yes
hermes-agent (unmaintained) 1f92ab10 · review weak
Checks the deployment log's quote but never hashes the list. E2EE v1; sends without E2EE unless strict.
— — — — — — — no no
pi-tee 64e86da · review partial
Opt-in only. Every instance quoted, E2EE v2, reply held until the model signature verifies. No compose/MRTD/app-id pin; GPU not gated; OutOfDate accepted.
partial
Appraises the model worker itself per request: nonce, local GPU check, release and container chain, SPKI pin plus EHBP. Stronger than teep here; auto-adopts tagged releases.
— — — — — — yes yes
pi-privacy 93adf50 · review weak
No quote verification: green when the JSON contains the nonce and words like 'nvidia' (attestation.ts:115-127). A green badge turns off its PII gate.
weak
Compares unsigned report bytes with the first connection's SPKI; no AMD chain or Sigstore; inference connections unpinned; fail-open.
— — — — — — n/a no
opencode-tinfoil 0bb3d3d · review — partial
tinfoil-js SDK: router only, AMD chain, Sigstore auto-adopt, EHBP; no nonce, no CRL, verdict cached for the process lifetime.
— — — — — — n/a yes
tinfoil-cli 97c76a7 · review — partial
tinfoil-go rc.8: nonce, SPKI pin, 7-day witness. Router and latest release by default; --enclave with --repo @sha256 pins a model enclave.
— — — — — — n/a yes
Thinkr SDK and verifier 6d8c1c89 · review — partial
Own TypeScript verifier: AMD chain, Sigstore tag, TLS cert carrying the attested keys, EHBP with authenticated reply frames. Router only; no nonce or CRL; adopts any signed tag.
— — — — — — n/a yes
Maple proxy and SDKs (OpenSecret) 0944a86 · review — — — — — — — partial
Nitro gateway enclave that decrypts the prompt: nonce, E2EE to the attested key, authenticated replies, blocks on failure. Accepts any PCR0 OpenSecret has signed; GPU host checked only by the enclave.
n/a yes

Libraries and SDKs

Client NEAR AITinfoilPhala Cloud (ACI)ChutesVeniceNanoGPTPrivatemodeIts own service Event log hashedBlocks on failure
private-ai-gateway verifier-ts e9b6ae8a · review — — weak
Replays RTMR3 from stored digests, then trusts the compose-hash payload unhashed (report.ts:173-182). Vendored by abalone.
— — — — — no n/a
tinfoil-go / tinfoil-python 99112212 · review — partial
Local AMD chain, client nonce, SPKI pin, 7-day freshness. Router route covers the router, not the model enclave; adopts any release Tinfoil signs.
— — — — — — n/a yes
elkimek venice-e2ee d8e37d15 · review — — — — weak
By default the quote isn't authenticated: DCAP and NVIDIA checks are opt-in. Strong when the caller enables them. ACI/1 receipt verifier unused.
— — — n/a n/a
nearai-cloud-verifier 94554726 · review weak
Demo CLI that prints booleans and never refuses. Its E2EE example encrypts to a key fetched with no nonce or quote check.
— — — — — — — n/a no
Adverserial confidential-sdk and policy 26fb9a5d · review — — — — — — — partial
Live policy pins MRTD/RTMR0-2/compose, but is served unsigned, kept out of Git and adopted immediately.
yes n/a
NEAR inference-sdk 9cc67f72 · review partial
Every instance quoted, event log replayed, deployment-log hash checked when served but skipped if absent (attestation-model.ts:98-107). Defaults: E2EE off (v3 when on), no compose pin, OutOfDate and missing GPU accepted.
— — — — — — — yes yes
Nillion nilai-py / nilai-ts e49b528 · review — — — — — — — weak
No attestation checks; nilai-py fetches the server key with TLS verification off (client.py:104).
n/a no
Secret AI secretvm-verify a5d0d7c · review — — — — — — — partial
TDX quote, TLS key bound in the quote, RTMR3 replayed from payloads, NVIDIA. Compose unpinned, dev images and an auto-downloaded registry accepted, GPU optional, no client nonce.
yes n/a

Gateways, routers and markets

Client NEAR AITinfoilPhala Cloud (ACI)ChutesVeniceNanoGPTPrivatemodeIts own service Event log hashedBlocks on failure
private-ai-gateway (as a client of its upstreams) e9b6ae8a · review Blocks only requests marked required; otherwise forwards and records the failure in the receipt. weak
NEAR's gateway quote and TLS key only; model attestations not fetched; any dstack CVM passes.
weak
Router via Tinfoil's Python SDK, no nonce, one session cached for an hour.
partial
Is the ACI gateway; signs receipts naming its upstream sessions.
weak
Measurement profiles fetched live from Chutes, unauthenticated.
— — weak
Delegated to Privatemode's own proxy.
— yes no
BitRouter attestation plugin (removed) f8763b1 · review weak
Removed in #701, never wired into requests. Pins read from unhashed event-log values; no key binding.
— — — — — — — no no
BitRouter inference auctions 583ccd9c · review weak
Wraps NEAR's SDK with no measurement pins; the 2026-11-11 run used plain HTTPS.
— — partial
Their own Chutes TEE, checked per request by the Market (all RTMRs, NVIDIA, ML-KEM); the buyer never sees it.
— — — partial
Buyer pins the Market's MRTD/RTMR0-2, compose and nonce, TLS pinned. The Market sees prompts. Verify command skips event-log hashing.
yes yes
AntSeed cda310ed · review — — — — — — — weak
Strict check of its own relay (MRTD/RTMR0-2, compose whitelist, Sigstore image, channel-bound quote), but the relay forwards plaintext to any upstream URL. Fail-open by default.
yes no
AnonRouter SDK 39e416b6 · review weak
Provider quote and NVIDIA signatures not checked; evidence arrives through AnonRouter's relay.
partial
Official SDK plus pinned TLS; router only; latest release; prompt passes through AnonRouter's relay.
— weak
Quote signature not checked; allowlist adopted from Chutes's list.
weak
Quote parsed, signature not checked; relay-fetched evidence.
— — partial
Own relay checked well (DCAP UpToDate, event digests recomputed, TLS on the content socket), but off by default in chat().
yes no
Alien Fleet 6a6be6f8 · review — — — — — — — weak
No TEE: consumer GPUs whose operators see plaintext. Node keys come from the gateway over plain HTTP, unauthenticated.
n/a no
Axiom web gateway (Astrea) 1b535ed8 · review partial
NVIDIA signature and signed completion receipt checked; code identity only mr_config_id = manifest hash; no debug check.
partial
Via tinfoil-rs; router only; latest release; no client nonce.
— — — — — partial
Browser verifies the gateway (a Tinfoil container) with tinfoil-go; the gateway verifies upstreams the browser doesn't re-check.
n/a yes
Routstr SDK / routstrd f847f91 · review Blocks only when the model id starts with tinfoil-; otherwise the node reads the prompt. — partial
Client attests the router with tinfoil-js; EHBP blinds the Routstr node. A reply without the EHBP nonce is passed through (TinfoilSecure.ts:323-325), so the node can forge replies. No nonce; cached for the process.
— — weak
On main the node sees plaintext. Open PRs trust a node-relayed verified flag.
— — — n/a yes
routstr-confidential (2PC TLS, no TEE) 430604a · review — — — — weak
Hides the prompt from the Routstr node via two-party TLS and ZK proofs; Venice still sees plaintext; no attestation. Open PRs; not externally reviewed.
— — — n/a yes
ritual-judge-protocol 6edba4bd · review — — — — — — — weak
Integrity, not privacy: median of three Ritual TEE executors. No client verification; trusts the registry's isValid (teeType unread); relayer-trusted scores; prompt is public calldata.
n/a no

Apps

Client NEAR AITinfoilPhala Cloud (ACI)ChutesVeniceNanoGPTPrivatemodeIts own service Event log hashedBlocks on failure
teemoon (Android, iOS) 63cef559 · review partial
On-device DCAP, NVIDIA token signature, E2EE v2. Only the first instance; OutOfDate accepted; deployment list unhashed.
— — — — — — — n/a no
abalone 53655796 · review weak
TCB policy warns and sends; NVIDIA check skipped on outage; E2EE v1; event-log values unhashed.
— weak
Vendored verifier-ts; no custody, no session pins; receipt check only in tests.
— — — — — no no
tee-proof-demo 5226ef12 · review partial
Browser checks quote, TCB, 8-GPU NVIDIA, exact RTMRs, and the reply signature. Reply key unbound (v2); the relay serves the verifier JS.
— partial
RTMR3 replay, source commit, KMS custody; receipt checked before display. Upstream session audited only after sending.
— — — — — yes yes
NEAR AI Gateway / IronWire df2e821 · review weak
Plain HTTPS to NEAR's API. Checks each reply's receipt after delivery, for trace provenance. Pins only in a staging build.
— — — — — — — no no
near-private-triage 01fcfe33 · review partial
Best of the hackathon apps: UpToDate only, every instance verified, MRTD/RTMR1-2/app-id/KMS pinned from captures, runs on localhost.
— — — — — — — yes yes
sealed-review 33f0ff35 · review partial
Recomputes MRTD/RTMR0-2 from NEAR's published OS image; refuses machines nobody reviewed; E2EE v2.
— — — — — — — yes yes
private-dna-reading (indaga) 905216b9 · review partial
Browser app; the only client found that sends E2EE v3. First instance only; accepts OutOfDate; trusts a stored event digest (event-log.mjs:15).
— — — — — — — no yes
Symbolon c1baaead · review partial
Quote verified on-chain by NEAR One's TDX verifier contract; gates payment only, after the prompt has been sent.
— — — — — — — n/a no
NEXORA 17601fad · review weak
Trusts Phala's remote quote.verified with no TCB check; NVIDIA token unsigned; first instance only; remote check can be switched off.
— — — — — — — n/a no
antrepreneur a751a6c7 · review weak
No quote signature check; report_data matched with includes() (tee.js:66-67); NVIDIA token unsigned; sends regardless. Copies near-examples' gaps.
— — — — — — — n/a no
NEAR Private Chat (web, iOS) 5cf5b0d7 · review weak
Prompt goes in plaintext to chat-api, then to cloud-api; no hop verifies. 'Verified' means the report fetched without error (ModelVerifier.tsx:222).
— — — — — — — no no
Tinfoil Chat (webapp, harness) bdfa6372 · review — partial
Browser attests the harness enclave (latest release), which holds plaintext and attests the router once at boot; the router checks the model replica.
— — — — — — n/a yes
Veil dd656b02 · review partial
NEAR inference-sdk 0.3.0 in the browser with E2EE v3; OutOfDate accepted unless strict mode.
— — — — — — — yes yes
Styx c61ddcbe · review weak
Own Python verifier with dcap-qvl and TLS fingerprint binding; NVIDIA token not signature-checked; any one passing instance accepted.
— — — — — — — n/a yes
Brave Leo (browser, ai-gateway, near-verifier) 72a1a994 · review Recipient is NEAR's gateway CVM, not the model. Brave issue #55247 calls client-side attestation out of scope. partial
Brave's server verifies deeply (pinned OS, reproducible builds, deployment-log hash, NVIDIA). The browser verifies nothing: OHTTP to whatever gateway key Brave serves, signature fields ignored, 'Verified' badge hardcoded true.
— — — — — — — yes yes
Nillion n8n-nodes-nilai f5dac94 · review — — — — — — — weak
SEV-SNP chain and reply signature checked, AMD root unpinned; no nonce or TLS-key binding; measurement pin keyed by a server-claimed version; prompt sent whatever the result.
n/a no
Secret AI chatbot (reference app) e2da664 · review — — — — — — — weak
Chats to raw IPs with TLS verification off process-wide (server.js:2); attestation only feeds a badge, which shows valid when checks are missing.
yes no

Monitors

Client NEAR AITinfoilPhala Cloud (ACI)ChutesVeniceNanoGPTPrivatemodeIts own service Event log hashedBlocks on failure
awesome-private-inference monitor 406b535 · review partial
Every instance and the deployment log, with public files aged 24 h. Sends no prompts.
partial
Router, local AMD chain and TCB check.
partial
Gateway keyset and sessions.
partial
Full register match and census.
partial
Legacy dstack rows; ACI/1 rows marked gateway-only.
— — — yes n/a

Reviewed and out of scope