VoltageGPU, Confidential VM, 8x H100 node in NVIDIA multi-GPU Protected PCIe mode Captured on 4 November 2026, 22:28 to 22:31 UTC, from inside a tenant VM (h100-xlarge), over SSH. Same method as the 8x H200 capture of the same evening: ../ppcie-8x-h200-2026-11-04/README.txt First 8x H100 capture, 10 September 2026: ../ppcie-8x-h100/README.txt Guide: https://docs.voltagegpu.com/pods/confidential-vm#gpu-attestation Every nonce and challenge below was drawn with a CSPRNG on our own machine, outside the VM, and passed in on the command line. The node was created for this capture and deleted right after it: 9.8 minutes from the creation request to the deletion confirmed. What this proves ---------------- 1. NVIDIA GPU attestation, 8 of 8 GPUs (attestation-run.txt, nras_token.json). NVIDIA's remote attestation service (NRAS v3) returned "Attestation Successful" for all eight GPUs of the node, from evidence generated inside the VM by nv-attestation-sdk 2.7.3 with get_evidence(options={"ppcie_mode": False}), on the nonce 4f5e935ed40c485266dd09f5fe3f05ee6ba283ccd73f2a966d641c0907a8c8c4 Per GPU: nonce match, report signature, certificate chain, driver RIM and VBIOS RIM fetched with measurements available, measres success, hwmodel GH100, secure boot on, debug disabled, driver 595.71.05, VBIOS 96.00.CF.00.01. raw-tokens-check.txt checks every ES384 signature against the keys NVIDIA publishes, outside the VM. 2. The Intel TDX quote and the NVIDIA tokens, bound to one manifest (manifest.json, bundle.json, verify.txt, verify-offline.txt). A second round, with voltage-verify 0.2.3 in multi-gpu-ppcie mode, on the manifest challenge c985e0e6b85c349f724e2fa735e0d821f1f55e6b8d26777f910a0363d130ed4e The TDX quote (version 4, 4,936 bytes) carries report_data = SHA-512 of the manifest, and the NRAS tokens carry eat_nonce = SHA-256 of the manifest. Verified outside the VM, online and offline: quote chain up to the pinned Intel SGX Root CA, not a debug TD, platform TCB, QE and TDX module UpToDate, PCK certificate not revoked, 9 NVIDIA-signed tokens, every GPU measurements ok: 15 of 15 checks pass. The same bundle under another challenge is rejected (verify-wrong-challenge.txt), and the selftest rejects nine kinds of tampering (verify-selftest.txt). 3. NVSwitch on this node, with one package added in the guest (nvswitch-diag.txt, nras_switch_token.json). A. As on 10 September, with the image as delivered: no NSCQ library in the guest ("undefined symbol: nscq_session_create"), no switch evidence, and NRAS answers INVALID_EVIDENCE. B. After installing libnvidia-nscq 595.71.05-1ubuntu1 from NVIDIA's network repository (the version that matches the driver, as NVIDIA advised on https://github.com/NVIDIA/nvtrust/issues/153), the SDK collected evidence from the four NVSwitch (LS10) and NRAS returned "Attestation Successful", on the nonce c9642d143015ab4af07c6690388265860834eecee9abe4524b3d3aedceb4ba06 Per switch: nonce match, report signature, certificate chain, BIOS RIM measurements available, measres success, secure boot on, debug disabled, switch BIOS 96.10.6D.00.01. raw-tokens-check.txt checks these tokens too. This is the first time the switches of this 8x H100 node pass: on 10 September they could not be attested (no NSCQ library), and on 19 September the same library let nvattest collect switch evidence, but its local verification failed on the OCSP check from the guest. 4. TLS from the guest to NVIDIA's OCSP responder (tls-ocsp-from-the-guest.txt). Twenty curl attempts to https://ocsp.ndis.nvidia.com/, 2.5 s maximum each: 13 completed the TLS handshake and got an HTTP answer (405, as expected for a bare GET), 4 failed with "Connection reset by peer" (curl exit 35) and 3 with "SSL connection timeout" (curl exit 28). The TCP connection itself opened every time, in about 15 ms (once in 1 s). This is the intermittent failure on the network path documented on 27 October, and it is still there: https://github.com/VoltageGPU/confidential-gpu-attestation-evidence/tree/main/nvattest-2.0.0-single-gpu-2026-10-27 The switch result in 3B does not depend on it, because NRAS ran the certificate and revocation checks on NVIDIA's side. What it does not prove ---------------------- - The NVSwitch result (3B) is a single run on this node. It needed a package the image does not ship, it is a separate call with its own nonce, and it is neither in the voltage-verify bundle nor bound to the TDX quote. Local verification inside the guest (nvattest) was not re-run here, and section 4 shows why it can still fail at random from this guest. - Switch attestation says the switches run firmware NVIDIA measured, with secure boot on. Protected PCIe mode does not encrypt traffic between GPUs over NVLink, and these tokens do not change that. - Nothing about the software inside the VM, nor about any VM other than this one. A fresh run on your own VM, on your own nonce, is the proof that counts. - It applies to the Confidential VM tier (h100-xlarge). The same hardware sold as bare metal has no TDX trust domain and no attestation. How the state reads on a multi-GPU node --------------------------------------- CC State : OFF Multi-GPU Mode : Protected PCIe CPU CC Capabilities : INTEL TDX GPU CC Capabilities : CC Capable CC GPUs Ready State : Not Ready, then Ready after the command below "CC State OFF" is how Protected PCIe reports; the Multi-GPU Mode line is the one to read. Ready State read Not Ready at boot and was set with "sudo nvidia-smi conf-compute -srs 1". GPU memory, per GPU: 82,542,720 KiB protected and 0 KiB unprotected (nvidia-smi conf-compute -gm). Verify it yourself ------------------ pip install "voltage-verify>=0.2.3" voltage-verify verify bundle.json --offline --hwmodel GH100 --gpus 8 \ --challenge c985e0e6b85c349f724e2fa735e0d821f1f55e6b8d26777f910a0363d130ed4e voltage-verify selftest bundle.json pip install "PyJWT[crypto]>=2.7" python check_raw_tokens.py \ nras_token.json 4f5e935ed40c485266dd09f5fe3f05ee6ba283ccd73f2a966d641c0907a8c8c4 \ nras_switch_token.json c9642d143015ab4af07c6690388265860834eecee9abe4524b3d3aedceb4ba06 Files ----- node-state.txt nvidia-smi conf-compute -q (before and after -srs 1) and -gm, per-GPU name, UUID, driver, VBIOS, bus id, memory, NVLink topology, CPU, memory, kernel, /dev/tdx_guest, configfs TSM, "tdx: Guest detected", lspci (8 GPUs and 4 NVSwitch). tooling.txt Versions installed in the VM: voltage-verify 0.2.3, nv-attestation-sdk 2.7.3, nv-local-gpu-verifier 2.7.3. attestation-run.txt Console output of round 1 (GPU attestation, SDK), with the decoded claims of every GPU. nras_token.json 17,522 bytes. JSON array: element 0 is the SDK's local envelope (HS256, not NVIDIA-signed), element 1 holds REMOTE_GPU_CLAIMS: NVIDIA's summary JWT and one NVIDIA-signed JWT per GPU (GPU-0 to GPU-7), iss https://nras.attestation.nvidia.com. tdx-and-nras-in-the-vm.txt Console output of round 2 in the VM (manifest, attest, the VM's own check, which is informational only). manifest.json, bundle.json Round 2: the manifest and the voltage-verify bundle (TDX quote, NRAS tokens, NVIDIA JWKS, Intel collateral, environment). verify.txt voltage-verify 0.2.3 on our machine, online, after the VM was deleted. verify-offline.txt Same, offline, collateral from the bundle. verify-wrong-challenge.txt Same bundle, all-zero challenge: NOT VERIFIED. verify-selftest.txt Nine tampered copies of the bundle, all rejected. nvswitch-diag.txt NVSwitch round: lspci, device nodes, fabric manager, attempt A (image as delivered) and attempt B (driver-matched NSCQ library). nras_switch_token.json 7,410 bytes. NRAS answer to attempt B: summary JWT and one NVIDIA-signed JWT per switch (SWITCH-0 to SWITCH-3). tls-ocsp-from-the-guest.txt Round 6: twenty TLS attempts from the guest to NVIDIA's OCSP responder, with the curl exit code of each. raw-tokens-check.txt Output of check_raw_tokens.py on both raw token files. check_raw_tokens.py The checker: ES384 signatures against NVIDIA's JWKS, issuer, nonces, per-device claims. scripts/ The six scripts piped into the VM (ssh ... 'bash -s'), as run. Their comments are in French. SHA256SUMS Checksums of every other file in this folder. Host: [redacted] (SSH port 2222), 8x NVIDIA H100 SXM5 80GB [10de:2330], 4x NVSwitch [10de:22a3], driver 595.71.05, VBIOS 96.00.CF.00.01, 96 vCPU, 1.3 TB RAM, kernel 6.8.0-110-generic, image Ubuntu 24.04. The VM was deleted after the capture.