awesome-private-inference a registry of TEE-verified inference

What can these providers actually prove?

Every day we ask each confidential-inference provider for an attestation and check what it contains. A row is verified only when every layer that provider's architecture should be able to prove, it proves. Anything we could not reach is marked as such rather than scored — an unreachable endpoint is not a privacy finding. Methodology →

How much to trust this page

Over 112 days and 2379 observations, this instrument has produced 0 verification failures. Every failing observation in its history was a transport error (583) or an invalid response with no recorded reason (130). By the rule of three, zero events in 2379 trials puts the 95% upper bound on the per-observation verification-failure rate at 0.126%. That is the honest reading: not "providers are safe", but "if this check fails, it fails less often than about once in 793 observations — or it is not wired to anything." We cannot yet distinguish those two cases.

Targets verified
0 / 21
10 partial · 11 unreachable
Builds reviewed
2 / 19
backlog 17 · last 54d ago
Cells carrying signal
26%
139 of 189 matrix cells are dashes
Unmeasurable providers
2
redpill, venice expose no stable version identity

Coverage by target

Proven counts the layers this provider's shape is expected to prove — the expectation is set per architecture in REQUIRED_LAYERS_BY_SHAPE, and that bar is an editorial judgment, not a measurement. Different providers have different denominators, so compare the missing column, not the fraction alone.

Provider Model Status Proven Unproven / reason
chutes DeepSeek-V3.2-TEE partial 3/4 serving_code_attested
chutes Kimi-K2.6-TEE partial 3/4 serving_code_attested
chutes Qwen3-32B-TEE partial 3/4 serving_code_attested
chutes gemma-4-31B-TEE partial 3/4 serving_code_attested
near-ai openai/gpt-oss-120b partial 6/7 backend_attested
near-ai zai-org/GLM-5.1-FP8 partial 6/7 backend_attested
tinfoil router partial 3/5 client_nonce_supported, runtime_config_fully_attested
not shown on the matrix below: client_nonce_supported, code_measurement_reproducible, hpke_pubkey_attested, runtime_config_fully_attested, tls_pubkey_pinned
venice e2ee-glm-5 partial 6/9 code_measurement_reproducible, prod_os_image, serving_code_attested
not shown on the matrix below: code_measurement_reproducible
venice e2ee-gpt-oss-120b-p partial 5/9 code_measurement_reproducible, compose_hash_committed, prod_os_image, serving_code_attested
not shown on the matrix below: code_measurement_reproducible
venice e2ee-qwen3-5-122b-a10b partial 6/9 code_measurement_reproducible, prod_os_image, serving_code_attested
not shown on the matrix below: code_measurement_reproducible
chutes GLM-5-TEE unreachable RuntimeError: HTTP 404 on /e2e/instances/e51e818e-fa63-570d-9f68-49d7d1b4d12f: {"detail":"
venice e2ee-uncensored-24b-p unreachable no TEE attestation available for this model (404)
not shown on the matrix below: code_measurement_reproducible
redpill phala/deepseek-v3.2 unreachable HTTP 502: 502 Bad Gateway

502 Bad

redpill phala/glm-4.7 unreachable HTTP 502: 502 Bad Gateway

502 Bad

redpill phala/gpt-oss-120b unreachable HTTP 502: 502 Bad Gateway

502 Bad

redpill phala/gpt-oss-20b unreachable HTTP 502: 502 Bad Gateway

502 Bad

redpill phala/kimi-k2.5 unreachable HTTP 502: 502 Bad Gateway

502 Bad

redpill phala/qwen-2.5-7b-instruct unreachable HTTP 502: 502 Bad Gateway

502 Bad

tinfoil gemma4-31b unreachable ConnectionError: HTTPSConnectionPool(host='gemma4-31b-inf6.tinfoil.containers.tinfoil.dev'
tinfoil gpt-oss-120b unreachable SSLError: HTTPSConnectionPool(host='gpt-oss-120b-0.inf6.tinfoil.sh', port=443): Max retrie
tinfoil llama3-3-70b unreachable ConnectionError: HTTPSConnectionPool(host='llama3-3-70b.tinfoil.containers.tinfoil.dev', p

How often does the attested code change?

Any client that pins an enclave measurement is betting the measurement holds still. It does not. Deploys counts transitions to a version string never seen before. Corrected accounts for deploys that begin and end between two probes: with T deploys seen over n−1 daily intervals the rate estimate is −ln(1−T/(n−1))/Δ.

Read revisits only on instance-sampled rows. There the version comes from whichever backend answered, so a return to an earlier value means we reached a different instance — Chutes' 19 revisits are one deploy plus a fleet that never finished draining, not 19 rollouts. On control-plane rows the version is read from a single document (NEAR from the gateway's compose, Tinfoil from the release feed), so it cannot show fleet structure at all and zero revisits there is guaranteed by construction rather than observed. Fleet size is not identifiable from once-a-day sampling on any row.

Target Version source Observations Distinct versions Deploys Revisits Days per deploy Corrected
chutes/DeepSeek-V3.2-TEE instance-sampled 54 2 1 0 54.0 53.5
chutes/GLM-5-TEE instance-sampled 43 2 1 6 43.0 42.5
chutes/Kimi-K2.6-TEE instance-sampled 54 2 1 4 54.0 53.5
chutes/Qwen3-32B-TEE instance-sampled 54 2 1 18 54.0 53.5
chutes/gemma-4-31B-TEE instance-sampled 54 2 1 16 54.0 53.5
near-ai/Qwen/Qwen3-30B-A3B-Instruct-2507 control-plane 32 10 9 0 3.4 2.9
near-ai/openai/gpt-oss-120b control-plane 84 19 18 0 4.8 4.2
near-ai/zai-org/GLM-5-FP8 control-plane 31 10 9 0 3.4 2.9
near-ai/zai-org/GLM-5.1-FP8 control-plane 84 18 17 0 4.9 4.4
redpill/phala/gpt-oss-20b control-plane 33 1 0 0
redpill/phala/qwen-2.5-7b-instruct control-plane 34 1 0 0
tinfoil/gemma4-31b control-plane 87 7 6 0 14.5 14.0
tinfoil/gpt-oss-120b control-plane 82 2 1 0 82.0 81.5
tinfoil/llama3-3-70b control-plane 93 3 2 0 46.5 46.0
tinfoil/router control-plane 107 30 29 0 3.7 3.2

Per-layer detail

Cells: verified, rejected, awaiting our review, required but not exposed, not applicable to this architecture. This matrix shows 9 of the 15 layers the probe records; the remainder are listed per row in the coverage table above.

Provider Model Shape Nonce bound TDX quote report_data binds key GPU attested Key derives to addr compose_hash committed Prod OS image Serving code attested Backend attested lat
chutes DeepSeek-V3.2-TEE chutes-tee 7.56s
chutes Kimi-K2.6-TEE chutes-tee 4.46s
chutes Qwen3-32B-TEE chutes-tee 5.2s
chutes gemma-4-31B-TEE chutes-tee 4.19s
near-ai openai/gpt-oss-120b tdx+gpu 5.48s
near-ai zai-org/GLM-5.1-FP8 tdx+gpu 3.6s
tinfoil router tinfoil-sev-snp-v2 0.31s
venice e2ee-glm-5 venice 1.93s
venice e2ee-gpt-oss-120b-p venice 3.59s
venice e2ee-qwen3-5-122b-a10b venice 2.01s
chutes GLM-5-TEE chutes-tee 2.22s
venice e2ee-uncensored-24b-p venice 0.17s
redpill phala/deepseek-v3.2 redpill 0.09s
redpill phala/glm-4.7 redpill 0.09s
redpill phala/gpt-oss-120b redpill 0.1s
redpill phala/gpt-oss-20b redpill 0.1s
redpill phala/kimi-k2.5 redpill 0.09s
redpill phala/qwen-2.5-7b-instruct redpill 0.09s
tinfoil gemma4-31b error 0.0s
tinfoil gpt-oss-120b error 0.0s
tinfoil llama3-3-70b error 0.0s

Editorial notes

Hand-written, not computed, and therefore able to go stale. Each carries the date it was last checked against the data. Anything not on this list that you see above is machine-derived.

How this page is generated

Daily at 13:17 UTC, probe-daily.yml runs python -m probes.collect, writes data/snapshots/YYYY-MM-DD.json, recomputes data/quality.json (the self-check driving the four tiles above), re-renders this page, and commits. Reproduce with run a probe locally; grade this page with python -m probes.quality.