# Private AI Inference: Who Else Sees Your Prompt? > Private AI inference means nobody but you can read your prompt while the model runs. The five levels, from plaintext logging to TEEs and local models. - Published: 2026-03-14 ยท Updated: 2026-10-17 - Canonical: https://ownyourmind.ai/why-deai/private-ai-inference/ - Cite as: Own Your Mind (ownyourmind.ai), licence CC BY 4.0. Link the canonical URL. Charts and interactive widgets are omitted here; project data is at https://ownyourmind.ai/api/scores.json. --- ## What is private AI inference? Private AI inference means running an AI model on your prompt so that nobody except you can read the prompt or the answer while the model works. There are three ways to get there: run the model on your own hardware, run it inside a hardware enclave the operator can't look into, or compute on encrypted data. Anything short of that is privacy by policy: someone can read your prompt and promises not to. Every AI privacy project says it's private. Most of them are using the word differently. Venice alone offers three meanings. In Private mode your identity is stripped before the prompt reaches a GPU, but the GPU operator still reads the text. In TEE mode the model runs inside an enclave the operator can't inspect. In E2EE mode the prompt is encrypted on your device and only decrypted inside a verified enclave. Run a model on your own Mac and there is no operator at all. Privacy in AI is a spectrum, and most projects sit further down it than their marketing suggests. The use case sets the level you need. A blog post draft needs far less protection than your company's financial projections, a patient's records, or your legal strategy. Where a platform sits on the spectrum tells you what you can safely send it. ## The five levels of AI inference privacy ### Level 0: No privacy. OpenAI, Anthropic (default) The provider sees your prompt in plaintext and keeps records of it. How long depends on the product and the settings you pick. OpenAI's API keeps abuse-monitoring logs for up to 30 days by default, and zero data retention needs OpenAI's approval, per its data controls guide. Anthropic's August 2025 consumer terms keep Claude chats for five years if you allow them to be used for training, and 30 days if you don't. The leaks have already happened. In April 2023 Samsung staff pasted faulty source code into ChatGPT, and the company restricted generative AI tools on company devices the following month. Business Insider reported in January 2023 that Amazon had warned its staff not to share confidential information with ChatGPT. With default settings, a centralised provider knows **who** you are and **what** you asked, and keeps both for as long as its policy says. ### Level 1: Policy-based privacy. Venice (Private mode), Morpheus (standard providers) Most "private" AI platforms sit here, and it pays to be precise about what that buys you. [Venice](https://ownyourmind.ai/projects/venice/) in its default Private mode sends your prompt through a proxy that strips your identity before it reaches a GPU. Venice's privacy page says inference runs on Venice-controlled GPUs or zero-data-retention partner infrastructure, with privacy enforced by contractual commitments. In practice: - **Your identity is hidden.** The GPU operator has no way to tie the prompt back to you. - **Your prompt content is visible.** The GPU must see plaintext to run the model. Nothing is encrypted during processing. - **Nothing is stored, by policy.** Venice says it doesn't log prompts and its partners are contracted not to. No hardware or cryptography stops an operator copying a prompt while it sits in GPU memory. To use a construction analogy: it's like sending a confidential document by courier with the return address torn off. The recipient doesn't know who sent it, but they open the envelope and read the contents. They promise not to photocopy it, and you trust them on that. Venice's Private mode gives you anonymity, while confidentiality needs its TEE or E2EE modes. The "who" is hidden. The "what" is exposed to the compute provider. Level 1 is a big step up from Level 0. There's no persistent record, no identity link, and no log to subpoena. For most work (writing, coding, analysis, creative work) it's enough. If you're pasting a cap table, a patient's records or a client's legal strategy into a Level 1 system, understand that a GPU operator could read it during inference. They don't know it's yours and they say they won't keep it, but the text passes through their hardware in plaintext. With a standard provider, [Morpheus](https://ownyourmind.ai/projects/morpheus/) is a Level 1 service. Its peer-to-peer routing is encrypted in transit, per Morpheus's docs, but the provider's GPU sees the prompt. Route to a TEE-tagged provider and it moves to Level 2, covered below. ### Level 2: Hardware-enforced privacy. Venice (TEE/E2EE), NEAR AI Cloud, Phala, Morpheus (TEE providers) At Level 2 the guarantee moves from policy to silicon. Trusted Execution Environments (TEEs) are isolated regions built into modern processors and GPUs. Memory inside the enclave is encrypted, so the operator who owns the server can't read what's being processed. The enclave also produces an attestation report: a signed statement from the chip maker's hardware that says what code is running and that it hasn't been tampered with. Today's stack pairs an Intel TDX or AMD SEV-SNP confidential VM with an NVIDIA GPU in confidential computing mode, which NVIDIA introduced with the H100. #### Venice TEE and E2EE modes Venice launched TEE and E2EE modes on 18 March 2026. Inference in both runs inside enclaves operated by [NEAR](https://ownyourmind.ai/projects/near/) AI Cloud and [Phala Network](https://ownyourmind.ai/projects/phala/). In E2EE mode your prompt is encrypted in the browser, stays encrypted through Venice's proxy, and is only decrypted inside the verified enclave. Each E2EE response carries a verification icon that links to the attestation report. The trade-offs: - **E2EE switches off features.** Web search, memory, file uploads and function calling are unavailable, because each would need your data decrypted outside the enclave. TEE mode keeps them. - **Paid tiers only.** Both modes sit behind Venice's Pro plan and above. - **A subset of models.** Only the models Venice tags `tee-` or `e2ee-` run this way. Both modes also work through the API. Developers pick models by prefix, and E2EE needs client-side ECDH key exchange (secp256k1, HKDF-SHA256) with AES-256-GCM encryption, per Venice's developer docs. Venice wasn't the first to put hardware-enforced privacy in front of consumers. Apple announced Private Cloud Compute in June 2024 and opened a research environment for inspecting its builds that October. Apple's version is bound to Apple devices and Apple's verification tooling. Venice runs in any browser on inference operated by independent providers, and the attestation report is yours to check. #### NEAR AI Cloud [NEAR](https://ownyourmind.ai/projects/near/) AI Cloud sells private inference directly, and it's also one of Venice's two enclave operators. Its docs describe Intel TDX confidential VMs paired with NVIDIA GPU isolation, with each environment producing cryptographic proofs of its integrity and configuration. NEAR publishes open-source verifier code, so developers can check a deployment's evidence before sending a prompt and check a response's signature afterwards. Read the model list carefully. NEAR tags open-weight models such as DeepSeek, Kimi and Qwen as "Private TEE". The Anthropic, OpenAI and Google models on the same page carry an "Incognito" label instead. Those companies don't release their weights, so NEAR can't load them into its own enclaves. #### Phala Network [Phala Network](https://ownyourmind.ai/projects/phala/) runs GPU TEE inference and cloud hosting on NVIDIA H100, H200 and B300 hardware paired with Intel TDX, and holds SOC 2 Type I and HIPAA compliance, per its own site. Its open-source dstack SDK packages standard Docker containers as confidential VMs. Bittensor's [Chutes](https://ownyourmind.ai/tokenomics/chutes-bittensor-revenue-machine/) subnet (SN64) is heading the same way. It says it runs a TEE mode on Intel TDX and NVIDIA confidential computing, and says prompts on eligible TEE-backed instances are encrypted on the client and only decrypted inside a verified enclave. #### Morpheus with a TEE provider Morpheus shipped attestation in two phases. v6.0.0 (18 March 2026) attests the provider's proxy-router: your node checks that the provider is running the published, signed build inside Intel TDX, with chat logging switched off, before opening a session. v7.0.0 (23 April 2026, with SCRT Labs' SecretVM) extends the chain to the GPU, binding the CPU enclave to NVIDIA's GPU attestation on every prompt, according to its release notes. v7.2.0 (2 June 2026) added AMD SEV-SNP hosts alongside Intel TDX, though the v7.15.0 release notes say that SEV check failed open until 14 October: an empty measurement was skipped, so any SEV host passed. So Morpheus reaches Level 2 end to end against a TEE provider. Against a standard provider it stays at Level 1, because Morpheus is a marketplace and runs no GPUs of its own. The privacy level is whatever the provider you route to delivers, and Morpheus's contribution is letting your node verify it. #### What it costs, and where it breaks The speed penalty is small. A 2024 benchmark of confidential computing on NVIDIA Hopper GPUs found overhead below 7% for most typical LLM queries, falling to nearly zero for larger models and longer sequences. Most of that cost is moving data between CPU and GPU, not the computation itself. The trust model is the weak point. You're trusting Intel, AMD and NVIDIA to have built the enclave correctly, and physical attacks keep landing: - **WireTap (September 2025).** Researchers pulled SGX attestation keys out of DDR4 servers with under $1,000 of equipment. Phala stopped admitting SGX workers the same day and moved its roadmap entirely onto TDX and NVIDIA confidential computing. - **TEE.fail (October 2025, IEEE S&P 2026).** A DDR5 memory interposer, also under $1,000, extracted attestation keys from Intel TDX and broke AMD SEV-SNP. The researchers showed they could "borrow" NVIDIA attestations from GPUs they didn't own, so a user could be convinced their chat was running inside a GPU TEE while it ran in the clear. Phala's dstack was one of the systems they demonstrated against. Both attacks need physical access to the server. That turns "a rogue operator reads your prompt" into "a rogue operator with a soldering iron, a budget and intent forges your proof." That's a far higher bar than Level 1, but it isn't zero, and TEE.fail shows attestation proves the hardware is authentic. It doesn't on its own prove your prompt ran on that hardware. **Practical guidance:** for healthcare, legal work and proprietary data, Level 2 is now the sensible default when you need a frontier-scale open model you can't run locally. Prefer services whose attestation binds to your specific connection, and keep the most sensitive material for Level 4. ### Level 3: Cryptographic privacy. FHE, MPC, ZK Level 3 is the mathematical guarantee. There's no trust in hardware vendors or policies, only the cryptography. #### Fully Homomorphic Encryption (FHE) Fully Homomorphic Encryption lets a server compute on data that stays encrypted the whole time. Your prompt is encrypted on your machine, the GPU works out the answer without ever decrypting it, and only your key can read the result. Without that key the provider can't see anything. It's slow. Zama's own Concrete ML demo runs a single attention block from GPT-2, a 2019 model, under FHE in about 14 minutes on a 196-core server. A modern LLM has dozens of those blocks per token, so FHE is nowhere near real-time chat today. #### Multi-Party Computation (MPC) Multi-Party Computation splits your data into shares held by several parties. Each computes on its share, none sees the whole, and the result is reconstructed at the end. For LLMs the coordination cost is brutal. The PUMA research system needed around five minutes to generate a single token from LLaMA-7B under MPC in 2023. MPC suits smaller, well-defined jobs: private statistics, key management, and encrypted storage. Arcium is the most visible MPC network in crypto. It acquired Inpher's core team and technology, including the Manticore MPC protocol, in November 2024, launched Mainnet Alpha on Solana in February 2026, and has since launched its ARX token. Arcium is a member of NVIDIA's Inception programme. [Nillion](https://ownyourmind.ai/projects/nillion/) uses MPC for its nilDB storage, but its nilAI LLM service runs inside TEEs on NVIDIA confidential computing, which makes Nillion's AI inference Level 2, not Level 3. #### Zero-Knowledge Proofs (ZKPs) Zero-knowledge proofs prove a computation was done correctly without revealing its inputs. You could prove "this model produced this output from a valid input" without showing the input. ZK proves a result is correct, and on its own it leaves the data exposed during computation. For inference privacy it works alongside TEE or MPC: one keeps the data hidden during processing, ZK proves the result afterwards. ### Level 4: Local and on-premises inference The most private inference never touches a network. Run an open-weight model on hardware you own and there's no provider, no policy and no attestation to check. The prompt never leaves the machine. What that takes in 2026: - **Hardware.** Memory is the constraint. Ollama lists Llama 3.3 70B at 43GB in its default 4-bit build, so a Mac Studio with 64GB or more of unified memory, or a workstation with enough GPU memory, runs it. Smaller models run on an ordinary laptop. - **Software.** Ollama, llama.cpp, LM Studio and Apple's MLX are free and run the major open-weight families. Our [open-weight model comparison](https://ownyourmind.ai/resources/open-weight-model-comparison/) compares the families by size and licence. - **On-premises for teams.** Businesses that can't send data out are buying turnkey inference servers from vendors such as Zanus AI, or building their own racks, and serving models behind their own firewall. The limits: you're restricted to open-weight models (no Claude, no GPT), capable hardware is a capital cost, and speed depends on your own specs. Our [case for self-hosted AI](https://ownyourmind.ai/why-deai/why-self-host-your-ai/) tests where that trade-off pays on a Mac Studio. For legal documents, medical analysis and proprietary code, local inference is still the gold standard. ## Where each project sits ## How to check a private inference claim A privacy claim is only as good as what you can check yourself. We run these six questions against any service that calls itself private: 1. **Who holds plaintext at each hop?** List every component between your keyboard and the GPU: app, proxy, gateway, provider. Any one that sees plaintext sets your privacy level. 2. **Can you verify the attestation yourself?** A green tick in the vendor's own interface is a claim. An attestation report you can check with open-source tooling is evidence. 3. **Is the attestation bound to your connection?** Look for a TLS certificate fingerprint or a fresh nonce inside the signed report. Without that binding, a valid report can be replayed from a different machine, which is exactly what TEE.fail demonstrated. 4. **Does it cover the GPU?** A CPU enclave around the router says nothing about the model server unless GPU attestation is chained to it. 5. **What sits outside the enclave?** Your IP address, account, payment details and timing are metadata, and enclaves don't hide them. 6. **Who can change the code?** An enclave running a build that one company can swap overnight is only as trustworthy as that company's release process. Signed, reproducible builds narrow that. Here's the method applied to Morpheus. Our own refresher reads every TEE-tagged model on chain and checks the provider's Intel TDX quote against the TLS certificate the host presents. At the last read: - **TEE-tagged models on chain:** 5 (figures as of 2026-10-23) - **Provider nodes serving them:** 1 (figures as of 2026-10-23) - **Host quotes our check verified:** 1 (figures as of 2026-10-23) - **Passing GPU attestation, per the provider's own health endpoint:** 2 (figures as of 2026-10-23) - **Flagged unverified, per the same endpoint:** 0 (a model whose backend is down, rather than failing attestation, counts in neither) (figures as of 2026-10-23) That answers questions 2 to 4 for Morpheus with evidence rather than trust. Question 1 depends on how you connect. Run your own node (from v7.9.0, since older routers fail TEE sessions) and the chain is attested from your machine to the GPU. Use the api.mor.org gateway and its own node verifies the provider for you, but the gateway hop itself is private only by policy, per Morpheus's TEE page, and it knows your identity from sign-up. ## What shipped in 2026, and what comes next The first nine months of 2026 moved hardware-attested inference from pilots into consumer products: - **Venice TEE and E2EE (18 March).** Per-response attestation in a consumer chat app, on hardware run by independent operators. - **Morpheus Phase 1 and Phase 2 (18 March and 23 April).** Attestation from the consumer's node to the GPU, with AMD SEV-SNP hosts added on 2 June (a check that failed open until 14 October). - **Arcium Mainnet Alpha (February).** Arcium's MPC network went live on Solana, though MPC is still far too slow for LLM chat. What's still open: Morpheus has a single TEE partner, with a second on its roadmap and gateway-side attestation at the research stage, per its TEE page. Its release notes list native RA-TLS, which embeds attestation in the TLS handshake itself, as future work. On the hardware side, the TEE.fail class of physical attacks needs a response from Intel, AMD and NVIDIA, since both Intel and AMD treat physical memory attacks as out of scope for their current designs. FHE is the long game. We think fully encrypted inference for interactive chat is still years away. Dedicated FHE accelerator hardware is the development that would change that. ## A practical decision framework Pick the level from the use case. The best level on paper is rarely the one you need. **Level 1 is enough when:** - Your prompts are everyday knowledge work (writing, coding, analysis) - The content wouldn't cause harm if read without your name attached - Your threat model is "I don't want a company logging and training on my data" **Level 2 is required when:** - Your prompts contain personal information about other people (healthcare, legal) - Regulation expects verifiable protection during processing (HIPAA, GDPR processing obligations) - Your threat model includes a compromised infrastructure provider - You need to show an auditor the data was protected during processing **Level 4 is required when:** - Maximum assurance is non-negotiable (trade secrets, privileged legal material) - You can't trust any outside infrastructure, including hardware-attested enclaves - You have the hardware and accept the model and speed limits Most people overestimate the level they need and underestimate the friction of the higher ones. If you're not handling data covered by HIPAA, securities rules or legal privilege, Level 1 is probably enough. The bigger problem is Level 0: sending everything to a provider that keeps it. ## The infrastructure battle Three forces are pushing private inference from niche to default. **Regulation.** GDPR, the EU AI Act, HIPAA, CCPA and a growing list of US state privacy laws all push toward the same requirement: show that data is protected while it's processed. "We have a policy" won't satisfy auditors for long. **Enterprise demand.** Every corporate legal team restricting ChatGPT is a potential customer for private inference. Banks and healthcare providers face the same wall Samsung hit in 2023, with client and patient data in place of source code. **Cost.** TEE overhead is now in single digits for most LLM work, so privacy no longer carries a large performance tax. Decentralised compute is pushing the base cost down too: Akash, for one, claims savings of up to 85% against AWS, Azure and Google Cloud. Venice has shown there's a mass market for Level 1, reporting 3.5 million registered users when it announced its Series A on 1 July 2026. Its TEE and E2EE modes, running on NEAR and Phala hardware, show a consumer app can offer Level 2 without asking users to understand the cryptography. The question is which projects make private inference invisible enough that people choose it without thinking, the way HTTPS became the default for the web. ## Frequently asked questions ### What is private AI inference? Running an AI model on your prompt so that nobody except you can read the prompt or the response during processing. It's delivered by local models, hardware enclaves (TEEs) or encrypted computation. Services that only promise not to log your data offer policy-based privacy, a weaker level. ### Is TEE inference really private? Against the operator of the server, yes, as long as the enclave hardware holds. TEEs encrypt memory so the operator can't read your prompt, and attestation lets you check the enclave is genuine. The residual risk is physical attacks such as WireTap and TEE.fail, which need hands-on access to the machine. ### What's the difference between private inference and confidential computing? Confidential computing is the hardware technology: CPU and GPU enclaves that keep data encrypted while in use. Private inference is one application of it, alongside local models and cryptographic methods such as FHE and MPC. ### Is Venice AI private? It depends on the mode. Venice's default Private mode hides your identity but the GPU operator can read the prompt (Level 1). Its TEE and E2EE modes run inside attested hardware enclaves operated by NEAR AI Cloud and Phala, so the operator can't read it (Level 2). Our [Venice review](https://ownyourmind.ai/projects/venice/) covers the detail. ### Can I run private AI inference at home? Yes. Open-weight models run locally with free tools such as Ollama, llama.cpp, LM Studio or MLX. Small models run on a laptop. A 70B model in its 4-bit build needs roughly 43GB of memory, so a machine with 64GB or more.