Quick Answer: Building an on-premise LLM cluster for 50 people costs $180K+ in hardware, $40K/year in power, and 6 months of setup. A Paris-based asset manager skipped all of it. They run Qwen3.5-397B-TEE on H200 GPUs inside Intel TDX enclaves for $1,199/mo, deployed in 14 minutes. Even the cloud operator can't read their prompts.
TL;DR: TDX overhead is 3-7%. Cold start hits 30-60s on shared pools. But their compliance officer sleeps better than his counterpart at a bulge-bracket bank running self-hosted Llama on unencrypted A100s.
The $180K Mirage
I spent three hours last Tuesday on a call with a quant fund CTO. He'd burned $23K on "pilot hardware" for an on-premise LLM cluster. Three H100s, a Supermicro chassis, enterprise networking gear. Six weeks in, his team still couldn't get vLLM to batch consistently across the cards.
His alternative? A VoltageGPU Confidential Pod with the same H100s, already configured, TDX-attested, running in 47 seconds.
The kicker: his all-in cost for self-hosting, amortized over 18 months, was $4.12/hr per GPU. Our H100 TDX at $3.75/hr beat it. And we handle the firmware updates.
What "On-Premise" Actually Means Now
The old definition: servers in your basement, air-gapped, your problem.
The new reality for regulated firms: data can't leave your control, but "control" doesn't mean "you physically dust the racks." It means cryptographic proof that no third party — cloud admin, hypervisor, our own engineers — can inspect model weights or prompts.
Intel TDX provides this. The CPU encrypts memory at the hardware level. Remote attestation generates a CPU-signed certificate proving your workload runs inside a genuine enclave. Not a VM label. Not a compliance checkbox. Silicon-level isolation.
from openai import OpenAI
client = OpenAI(
base_url="https://api.voltagegpu.com/v1/confidential",
api_key="vgpu_YOUR_KEY"
)
response = client.chat.completions.create(
model="financial-analyst",
messages=[{"role": "user", "content": "Analyze Q3 leverage covenant in this LBO term sheet..."}]
)
print(response.choices[0].message.content)
Same SDK. Same code you'd write for OpenAI. Different threat model entirely.
The 50-Person Firm: Real Numbers
A regulated asset manager in Paris (name NDAd, sector: private credit). 47 employees, €2.1B AUM. Their constraint: fund documents can't touch US-cloud infrastructure. Schrems II, their LP agreements, and their own paranoia.
They evaluated three paths:
| Approach | Upfront Cost | Monthly Run | Time to Deploy | Encryption |
|---|---|---|---|---|
| Self-hosted H100 cluster | $186,000 | $3,400 (power + colo) | 4-6 months | None (GPU memory plaintext) |
| Azure Confidential H100 | $0 | ~$14/hr = $10,080/mo | 3-6 months (DIY) | Intel TDX |
| VoltageGPU TDX H200 | $0 | $4.635/hr = ~$3,350/mo | 14 minutes | Intel TDX + zero retention |
Azure wins on certification breadth. Self-hosting wins on... nothing, honestly, except the illusion of control. The firm chose door three.
What "Hardware-Sealed" Actually Looks Like
Their workflow: upload a 340-page credit agreement. The Financial Analyst agent extracts covenants, flags change-of-control triggers, scores amendment risk. Average response time: 6.65 seconds. Throughput: 116 tokens/second on H200 TDX.
The TDX overhead? Measured at 5.2% vs identical non-encrypted inference. Barely perceptible for document analysis. Noticeable if you're doing real-time trading — which they're not.
Attestation happens on every pod boot. They curl /attest, get a signed Intel quote, verify it against Intel's PCS. Takes 800ms. Their compliance officer added this to their SOC-1 evidence package. (We don't have SOC 2. He didn't care. The attestation certificate is stronger.)
The Honest Downsides
I've run enough pilots to know where this frays.
Cold starts hurt. The Starter plan ($349/mo) uses a shared TDX pool. First request after idle? 30-60 seconds while the enclave spins up. The Paris firm hit this twice, moved to Pro within a week. Pro at $1,199/mo gets dedicated H200 allocation. Problem gone.
No PDF OCR. Their credit agreements are scanned legacy docs. They pre-process with Adobe, feed text to the agent. Annoying. On the roadmap, not shipped.
7B models lag GPT-4 on edge cases. The Starter plan runs Qwen3-32B-TEE. Fine for extraction, summarization, standard Q&A. The fund's general counsel tried it on a novel cross-border restructuring clause. It hallucinated a Dutch statutory provision. They upgraded to Pro's 397B parameter model for anything involving jurisdiction-shopping.
Why This Isn't "Cloud Washing"
Every vendor claims security. Few prove it at the hardware layer.
ChatGPT Enterprise? Data sits in plaintext GPU memory. Their "data isn't used for training" promise is contractual, not cryptographic. A rogue engineer with hypervisor access — or a NSL served to Azure — bypasses it.
Self-hosted? Your data isn't encrypted in RAM. A compromised kernel module, a supply-chain backdoored NIC firmware, a janitor with a USB stick. Attack surface you own entirely.
TDX isn't perfect. Side-channel risks exist. The 3-7% overhead is real. But it's the only deployed technology that gives you hardware-sealed inference without owning the hardware.
The Deployment That Actually Happened
Thursday, 9:47 AM: Fund compliance officer creates account.
9:51 AM: Provisioning completes. H200 TDX pod live.
9:52 AM: /attest returns valid Intel quote. He screenshots it for the file.
10:01 AM: First credit agreement uploaded. 287 pages. 6 covenant breaches flagged. One false positive (agent misread a waiver as a breach).
10:23 AM: Second document. 94 pages. Clean.
Total time from "we should evaluate this" to "production workload running": 14 minutes. Their previous on-premise LLM project? Still in procurement, month four.
What I Don't Like (Because I Built This)
The pricing page confuses people. "Per-second billing" for GPU compute, "per-request" for agents, two different dashboards. We're fixing it. Not fixed yet.
No SOC 2 certification. GDPR Art. 25, Intel TDX attestation, DPA on request. That's the stack. Some RFPs auto-disqualify us. I tell prospects: read the attestation spec, then read SOC 2 Type II criteria. Decide which one your adversary cares about.
The Plus tier at $20/mo? Personal Telegram bot, great for solo practitioners. Useless for a 50-person firm. Wrong tool, wrong buyer. I see signups from people who need Pro, get frustrated, churn. Our onboarding flow doesn't catch this well.
The Real Alternative to On-Premise
"On-premise LLM alternative" used to mean "cheaper cloud API." That's dead. The real alternative is: same cryptographic control as your own basement, none of the basement.
The Paris firm didn't buy a GPU. They bought a proof. Every inference runs inside silicon they don't own, sealed from the operator, attested by Intel's root of trust. Their LPs accepted this in diligence. Their DPO signed off. Their CTO didn't spend six months learning InfiniBand topology.
Don't trust me. Test it. 5 free agent requests/day -> voltagegpu.com