Skip to content

Check an 8-GPU Node Before a Multi-Day Training Run

Check NVLink, P2P and NCCL bandwidth, thermals, ECC and Xid on an 8-GPU node in 10 minutes before a long training run. Real 8x H100 SXM output and thresholds.

Key Takeaways

  • One command, about ten minutes. hwcheck.sh --acceptance checks the NVLink mesh, GPU-to-GPU and NCCL bandwidth, five minutes of full load, ECC and Xid errors, and what a previous tenant left. Exit code 0 is PASS, 1 is WARN, 2 is FAIL.
  • A real run, 24 October 2026. A production 8x H100 SXM bare metal node passed all 17 lines in 7 minutes, PyTorch download included: 387 to 391 GB/s per GPU pair, 478.5 GB/s of NCCL bus bandwidth, 669 to 689 TFLOPS bf16 per GPU, 69 C at most.
  • Ten minutes is not three days. A PASS covers the faults that show at once. It says nothing about an Xid at hour 30, a problem between nodes, or the firmware.

Before you start a training run of several days on a rented 8-GPU machine, run one read-only command on it:

On the node
curl -fsSL https://voltagegpu.com/hwcheck.sh | bash -s -- --acceptance --gpus 8 --bare-metal

It takes about ten minutes and prints one verdict line per measure (PASS, WARN or FAIL, 17 lines on an 8x H100 SXM node) in six blocks: topology, P2P, NCCL, load, errors and hygiene. The script is hwcheck, open source under the MIT license (github.com/VoltageGPU/hwcheck). It runs on any provider, on bare metal or in a VM.

On 24 October 2026 we ran it on a production 8x H100 SXM bare metal node rented through our public API, a few minutes after installing the NVIDIA driver on it. This tutorial goes through that run line by line, with the threshold behind each verdict, then its limits. Every output below is copied from the published files. The commands a reader runs (help, download, checksums) were run again on 24 October 2026 at 16:48 UTC, on Ubuntu 24.04 under WSL2, in a fresh directory.

Why test the node before a run of several days

Because what usually ruins a long job in its first hours can be seen in ten minutes, before you have paid for those hours. The acceptance test looks for:

  • A GPU off NVLink, or a link down. A GPU off NVLink talks to the others over PCIe, which tops out at 64 GB/s on Gen5 x16: 14% of the 450 GB/s per direction of NVLink on H100 SXM. A link down leaves a pair with fewer NVLinks than the model has.
  • A card at x8 instead of x16. That halves its PCIe bandwidth, and it is usually a badly seated card or riser.
  • A GPU that overheats, slows down or computes wrong. The load test compares the bits of repeated products, so silent data corruption shows as a FAIL.
  • ECC errors or a pending row remap. A pending remap means the GPU needs a reset.
  • Files from a previous tenant. If a previous tenant's credentials are still on the machine, whoever comes after you would find yours the same way.

On the node below the test took 425 seconds. At the public price of that machine ($49.80 per hour), that is about $5.88.

Before the test: install the NVIDIA driver and Fabric Manager

A bare Ubuntu image has no NVIDIA driver, so no test can see the GPUs. On NVSwitch boards such as HGX 8x H100, CUDA also does not start without NVIDIA Fabric Manager. Our bare metal is delivered that way: Ubuntu 24.04, login as ubuntu with sudo, no NVIDIA driver. On a node we rented earlier that day (13:05 UTC, another hostname), the listing check run before any driver printed this:

Command
curl -fsSL https://voltagegpu.com/hwcheck.sh | bash -s -- --gpu H100 --gpus 8
Output (hwcheck.txt, 24 October 2026)
VoltageGPU hardware check (read-only, nothing about this machine is sent anywhere)
  2026-10-24 13:05 UTC · bm-06e585be

                 LISTED                   MEASURED                           
  GPU            H100                     no GPU visible (nvidia-smi failed) DOES NOT MATCH
  CUDA compute                            not tested (no PyTorch)            info
  CPU                                     192 threads                        info
  RAM            -                        2015 GB                            info
  Disk (/)       -                        866 GB, 864 GB free                info
  Download                                613 Mbit/s (one stream)            info

RESULT: DOES NOT MATCH THE LISTING (no GPU visible)
Release the pod from your dashboard and write to contact@voltagegpu.com with this output.

On a bare node, that verdict only means the driver is missing. The last line is the tool's generic advice for any mismatch (raw output).

One script installs the driver and Fabric Manager, both pinned to the same version:

On the node, as ubuntu (sudo) or root
curl -fsSL https://voltagegpu.com/dedicated-setup.sh | bash

Read it before you pipe it into bash: dedicated-setup.sh is 137 lines. What it does, according to its own header and code:

  • It stops unless the machine runs Ubuntu 24.04 on x86_64, has at least one NVIDIA GPU on the PCI bus, has Secure Boot off (the kernel would refuse the modules that DKMS builds) and has no NVIDIA driver of another version already loaded.
  • It installs NVIDIA's repository key, cuda-keyring 1.1-1, after checking its SHA-256, then a version pin, then nvidia-open 595.112.07 (driver branch R595, open kernel modules built by DKMS) and, on NVSwitch boards only, nvidia-fabricmanager 595.112.07.
  • From Ubuntu it adds the running kernel's headers and compiler, python3-venv and pciutils, which the acceptance test uses.
  • It starts Fabric Manager and the persistence daemon, then waits until every GPU reports its fabric as Completed / Success.
  • Exit code 0 means ready, 1 an error (the last lines say why), 3 a reboot is needed, then run it again. It is safe to run again: what is done stays done.

On 24 October 2026, on the node tested below, the script printed:

The script's own lines (apt and DKMS output: 01c-dedicated-setup.txt)
[15:43:43] VoltageGPU dedicated setup: driver 595.112.07; 8 NVIDIA GPU(s) and 4 NVSwitch(es) on the PCI bus; kernel 6.8.0-139-generic
[15:43:43] 1/5 cuda-keyring from https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64
[15:43:43] 2/5 package lists, version pin, kernel headers, gcc gcc-13, python3-venv, pciutils
[15:43:57] 3/5 nvidia-open=595.112.07-1ubuntu1 nvidia-fabricmanager=595.112.07-1ubuntu1 (the DKMS build of the kernel modules takes a few minutes)
[15:44:41] 4/5 kernel modules
[15:45:27] 5/5 services
[15:45:52] ready in 2 min 12 s: 8 GPUs, driver 595.112.07, fabric Completed / Success on every GPU, Fabric Manager active, persistence Enabled
[15:45:52] next: nvidia-smi topo -m (NV18 between every pair on 8x H100 SXM), then the acceptance test:
[15:45:52]   curl -fsSL https://voltagegpu.com/hwcheck.sh | bash -s -- --acceptance --gpus 8 --bare-metal

2 min 12 s, with a caveat written in the evidence README: it was the second attempt, and the 358 MB of NVIDIA packages came from the apt cache of the first one, which had downloaded 365 MB in 14 s. Count about 15 s more on a fresh machine. The first attempt failed at the DKMS build, because NVIDIA's dkms package only recommends gcc and that version of the script installed packages without their recommendations. The current script installs gcc explicitly. The failed log, the cleanup and the first script are published too (files 01a, 01b and dedicated-setup-first-version-failed.sh). We have run the script on one node, ours.

Run the acceptance test

Run it once the driver is in, before you copy data or start the job. --gpus 8 makes it fail if fewer than 8 GPUs are visible. --bare-metal makes it fail if a hypervisor is detected; leave it out on a VM. The options, as the served script prints them:

Command
curl -fsSL https://voltagegpu.com/hwcheck.sh | bash -s -- --acceptance --help
Output
usage: acceptance.sh [--quick] [--burn-seconds N] [--gpus N] [--bare-metal] [--no-install] [--out DIR]
  --quick            skip the 5-minute load test (about 2 minutes instead of 10)
  --burn-seconds N   length of the load test in seconds, 10 to 3600 (default 300)
  --gpus N           GPU count of the listing: FAIL if fewer are visible
  --bare-metal       the listing says bare metal: FAIL if a hypervisor is detected
  --no-install       never download PyTorch; the tests that need it are skipped without it
  --out DIR          where to write the report files (default: the current directory)

What happens when you run it:

  • hwcheck.sh downloads hwcheck-acceptance.sh and runs it only if its SHA-256 matches the one pinned in hwcheck.sh. From a clone of the repository, it uses the acceptance.sh next to it.
  • If the image has no PyTorch that sees the GPUs, it installs a pinned build into a temporary directory, about 3 GB from download.pytorch.org, matched to the driver's CUDA version. The directory is removed at exit, Ctrl-C included. That download is the only network traffic. --no-install forbids it: on an image without PyTorch, the tests that need it are then skipped, and the result is WARN.
  • It writes three files in the current directory, or in --out: hwcheck-acceptance-<UTC time>.txt (what you saw), .json (every value) and .sha256 (the SHA-256 of both, to show later that the report was not edited).
  • Exit code 0 is PASS, 1 is WARN (read those lines before a long job), 2 is FAIL (do not start a long job there), 64 is a bad option.
  • Root is best: as another user, the lines it could not check say so.

On 24 October the same test ran from a copy of the repository at commit d13bd29, as user ubuntu:

Command
cd ~ && bash ~/hwcheck/hwcheck.sh --acceptance --gpus 8 --bare-metal --out ~/hwcheck-out
Console on the node (the first line is from our harness)
# started 2026-10-24 15:46:53 UTC, finished 2026-10-24 15:53:58 UTC, 425 s, exit 0
. VoltageGPU hwcheck acceptance test 1.1, about 10 minutes; nothing is sent anywhere
. PASS  GPUs: 8x NVIDIA H100 80GB HBM3
. PASS  Previous tenant: nothing found in the home directories, /tmp, /var/tmp and /workspace; not readable: /root (run as root to include it)
. PASS  SSH keys: 1 key can log in (ubuntu: 1)
. PASS  GPUs idle at start: no compute process, no GPU memory in use
. PASS  NVLink mesh: 56/56 GPU pairs on NV18 (NV18 expected on H100 SXM)
. PASS  NVSwitch: 4 NVSwitch chips visible (4 on an 8-GPU H100 SXM board)
. PASS  Fabric state: Completed / Success on 8/8 GPUs
. PyTorch not found: installing torch 2.14.1 (cu130) from download.pytorch.org into /tmp/hwcheck-acceptance.kD0SLr (about 3 GB, deleted at the end)
. PASS  CUDA init: 8/8 GPUs in PyTorch 2.14.1+cu130 (installed for this run, cu130)
. PASS  P2P copies: slowest pair 387 GB/s (GPU 0>2), median 389 GB/s (FAIL under 112, WARN under 270: H100 SXM line rate 450 GB/s)
. PASS  NCCL all-reduce: bus bandwidth 478.5 GB/s on 8 GPUs (4 GiB, NCCL 2.30.7); H100 SXM: about 370 GB/s in ring, about 480 with NVLink SHARP, PASS from 340
. load test: 300 s of matrix multiplications on every GPU, nvidia-smi read every 5 s
. PASS  Throughput: 8/8 GPUs ran it, median 684 TFLOPS (bfloat16), slowest GPU 2 at 669
. PASS  Thermal / power: max 69 C (GPU 2), no thermal or hardware slowdown; at the power cap in 95% of the samples (normal under full load)
. PASS  Clocks: median SM clock 1368 MHz, every GPU between 1335 and 1380 MHz, at least 90% busy
. PASS  PCIe links: Gen5 x16 on 8/8 GPUs under load, host-to-GPU copies 53.1 to 55.5 GB/s
. PASS  ECC / memory: ECC on 8/8 GPUs, no uncorrected error, nothing pending, 0 corrected since the driver loaded
. PASS  Kernel log (Xid): no Xid during the test, read with journalctl
. PASS  Virtualization: none detected (systemd-detect-virt): bare metal, as listed

425 seconds in total, the PyTorch download included. After this run we raised two P2P thresholds, using its own measurements (details in block 2, commit 4abbd70). The scripts served today carry those thresholds, so they are not byte for byte the copy that ran. With the current thresholds the same node still passes every line.

Read the result, block by block

Each line reads PASS, WARN or FAIL. N/A means the measure does not apply (no NVLink on a PCIe machine), INFO that it was measured without a reference to judge it, SKIP that it could not run, which makes the result WARN. The verdict lines of the 24 October report:

hwcheck-acceptance-20261024T154657Z.txt, verdict lines
  1 Topology  GPUs                PASS  8x NVIDIA H100 80GB HBM3
              NVLink mesh         PASS  56/56 GPU pairs on NV18 (NV18 expected on H100 SXM)
              NVSwitch            PASS  4 NVSwitch chips visible (4 on an 8-GPU H100 SXM board)
              Fabric state        PASS  Completed / Success on 8/8 GPUs
              PCIe links          PASS  Gen5 x16 on 8/8 GPUs under load, host-to-GPU copies 53.1 to 55.5 GB/s
  2 P2P       CUDA init           PASS  8/8 GPUs in PyTorch 2.14.1+cu130 (installed for this run, cu130)
              P2P copies          PASS  slowest pair 387 GB/s (GPU 0>2), median 389 GB/s (FAIL under 112, WARN under 270: H100 SXM line rate 450 GB/s)
  3 NCCL      NCCL all-reduce     PASS  bus bandwidth 478.5 GB/s on 8 GPUs (4 GiB, NCCL 2.30.7); H100 SXM: about 370 GB/s in ring, about 480 with NVLink SHARP, PASS from 340
  4 Load      Throughput          PASS  8/8 GPUs ran it, median 684 TFLOPS (bfloat16), slowest GPU 2 at 669
              Thermal / power     PASS  max 69 C (GPU 2), no thermal or hardware slowdown; at the power cap in 95% of the samples (normal under full load)
              Clocks              PASS  median SM clock 1368 MHz, every GPU between 1335 and 1380 MHz, at least 90% busy
  5 Errors    ECC / memory        PASS  ECC on 8/8 GPUs, no uncorrected error, nothing pending, 0 corrected since the driver loaded
              Kernel log (Xid)    PASS  no Xid during the test, read with journalctl
  6 Hygiene   Previous tenant     PASS  nothing found in the home directories, /tmp, /var/tmp and /workspace; not readable: /root (run as root to include it)
              SSH keys            PASS  1 key can log in (ubuntu: 1)
              GPUs idle at start  PASS  no compute process, no GPU memory in use
              Virtualization      PASS  none detected (systemd-detect-virt): bare metal, as listed

RESULT: PASS
Block8x H100 SXM, 24 Oct 2026FAIL when (H100 SXM)
1 TopologyNV18 on 56/56 pairs, 4 NVSwitch, fabric ready, Gen5 x16fewer GPUs than --gpus, a pair over PCIe, or a GPU under load narrower than both ends allow
2 P2P387 to 391 GB/s per pair, median 389bytes differ, no peer access, or a pair under 112 GB/s (25% of 450)
3 NCCL478.5 GB/s bus bandwidthunder 185 GB/s (PASS from 340)
4 Load669 to 689 TFLOPS bf16, 69 C max, no thermal or hardware slowdowna GPU fails to run or returns other bits for the same product; hardware thermal slowdown or power brake in 5% of samples
5 ErrorsECC on, 0 corrected, no Xidan uncorrected error, a pending remap, or a hardware Xid during the test
6 Hygienenothing left, 1 SSH key, GPUs idle, no hypervisorcredential files found, or a hypervisor with --bare-metal

1. Topology: is every GPU on NVLink?

On an 8x H100 SXM board, nvidia-smi topo -m must read NV18 for all 56 GPU pairs. It did:

From the report
nvidia-smi topo -m:
        GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    GPU6    GPU7    CPU Affinity    NUMA Affinity   GPU NUMA ID
    GPU0         X      NV18    NV18    NV18    NV18    NV18    NV18    NV18    0-23,96-119     0               N/A
    GPU1        NV18     X      NV18    NV18    NV18    NV18    NV18    NV18    0-23,96-119     0               N/A
    GPU2        NV18    NV18     X      NV18    NV18    NV18    NV18    NV18    0-23,96-119     0               N/A
    GPU3        NV18    NV18    NV18     X      NV18    NV18    NV18    NV18    24-47,120-143   1               N/A
    GPU4        NV18    NV18    NV18    NV18     X      NV18    NV18    NV18    48-71,144-167   2               N/A
    GPU5        NV18    NV18    NV18    NV18    NV18     X      NV18    NV18    48-71,144-167   2               N/A
    GPU6        NV18    NV18    NV18    NV18    NV18    NV18     X      NV18    48-71,144-167   2               N/A
    GPU7        NV18    NV18    NV18    NV18    NV18    NV18    NV18     X      72-95,168-191   3               N/A

The legend of nvidia-smi reads NV# = Connection traversing a bonded set of # NVLinks (04-nvidia-smi-topo.txt). The rules: on SXM boards (H100, H200, A100, B200, B300) every pair must read NV followed by a number. A pair that reads SYS, NODE, PHB, PXB or PIX goes over PCIe: FAIL. A pair with fewer links than the model has (18 on H100, H200, B200 and B300, 12 on A100) has a link down: WARN.

The same block counts the NVSwitch chips with lspci (4 expected on H100 and H200 boards, 6 on A100), reads the fabric state of every GPU (WARN unless Completed / Success), and keeps the best PCIe link seen during the tests, because an idle GPU drops to Gen1:

From the report
PCIe links, best generation and width seen:
    GPU 0: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 55.5 GB/s
    GPU 1: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 53.2 GB/s
    GPU 2: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 55.5 GB/s
    GPU 3: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 53.1 GB/s
    GPU 4: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 53.9 GB/s
    GPU 5: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 54.3 GB/s
    GPU 6: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 53.9 GB/s
    GPU 7: Gen5 x16 seen (Gen5 x16 possible), under load, host-to-GPU copies 55.3 GB/s

A GPU under load that runs narrower than both ends allow is a FAIL. A generation below the maximum under load, or host-to-GPU copies under 70% of the median GPU, is a WARN. Here every GPU ran at Gen5 x16, with 53.1 to 55.5 GB/s from pinned host memory.

2. P2P: how fast is each GPU pair?

Each pair moved 387 to 391 GB/s in one direction, median 389 GB/s: 86% of the 450 GB/s line rate of NVLink on H100 SXM. The test copies 256 MiB from every GPU to every other one, 20 times, and compares the bytes afterwards:

From the report
GPU-to-GPU copies in GB/s (row = source GPU, column = destination GPU):
    GPU0        -    389    387    390    388    389    388    388
    GPU1      390      -    389    390    388    389    388    388
    GPU2      389    389      -    387    388    389    389    389
    GPU3      388    389    389      -    388    388    388    388
    GPU4      389    387    391    389      -    388    390    390
    GPU5      389    389    390    389    387      -    391    387
    GPU6      390    387    390    389    391    391      -    390
    GPU7      389    391    389    389    390    391    389      -

FAIL if bytes differ, if a pair lacks peer access on an NVLink mesh, or if a pair is under 25% of the line rate (112 GB/s on H100 SXM: under that, the pair is not really on NVLink). WARN if the median pair is under 75% of the line rate, or if one pair is under 90% of the median. The line rate is half the two-way figure of NVIDIA's datasheets: 900 GB/s for H100 SXM and H200 SXM, 600 GB/s for A100 SXM, 1.8 TB/s for B200 and B300, so 450, 300 and 900 GB/s per direction.

The report says "WARN under 270": that was 60% of the line rate, the rule at the time. After this run, the median rule went to 75% and the pair rule from 80% to 90% of the median. The slowest pair here is at 99% of the median, so the node passes both.

3. NCCL: does an all-reduce reach the expected bandwidth?

On 8x H100 SXM, expect about 370 GB/s of bus bandwidth in ring and about 480 GB/s with NVLink SHARP. This node reached 478.5 GB/s. The test runs torch.distributed with NCCL, one process per GPU, 4 GiB of float32 per GPU, 20 iterations, and checks the values:

From the report
all-reduce of 4 GiB per GPU, 20 iterations: algorithm bandwidth 273.4 GB/s, 15709 us per all-reduce

Bus bandwidth is the figure nccl-tests reports. For an all-reduce, busbw = algbw × 2(n−1)/n, so 273.4 × 14 / 8, about 478.5 GB/s on 8 GPUs. NCCL maintainers at NVIDIA put effective NVLink at about 370 GB/s (20% protocol overhead on 450) and the all-reduce peak with NVLink SHARP at about 480 GB/s on large sizes (nccl-tests#212). In nccl-tests#290, a result of 476 GB/s "looks to be in the expected range for H100".

Thresholds: on H100 SXM and H200, PASS from 340 GB/s, FAIL under 185 (half the effective rate), WARN in between. PASS from 340 accepts a node without NVLink SHARP. On A100 SXM, PASS from 210 and FAIL under 115, from about 230 GB/s at 8 GB (nccl-tests#149). B200 and B300 read INFO: no reference yet.

4. Load: five minutes at full power

All eight GPUs ran bf16 matrix products for five minutes at 669 to 689 TFLOPS each, reached 69 C at most, and stayed at their power cap in 95% of the samples, which is normal. The test multiplies 8192 by 8192 bf16 matrices on every GPU at once and reads nvidia-smi every 5 seconds:

From the report
    GPU  samples  busy  max C  HBM C  SM MHz   max W  TFLOPS  slowdown samples: power cap / thermal / hw / brake
    0         60  100%     58     49    1365     702   687.0  57 / 0 / 0 / 0
    1         60  100%     65     56    1350     701   676.3  57 / 0 / 0 / 0
    2         60  100%     69     56    1335     700   669.2  57 / 0 / 0 / 0
    3         60  100%     58     49    1372     701   684.5  57 / 0 / 0 / 0
    4         60  100%     62     53    1380     701   684.6  57 / 0 / 0 / 0
    5         60  100%     65     59    1380     701   688.7  57 / 0 / 0 / 0
    6         60  100%     65     55    1365     702   683.3  57 / 0 / 0 / 0
    7         60  100%     60     51    1380     701   682.9  58 / 0 / 0 / 0
  • Throughput. FAIL if a GPU does not run, returns non-finite values, returns other bits for the same product (silent data corruption), or runs under half the median GPU. WARN under 85% of the median, or if a GPU computes another result than an identical GPU.
  • Thermal and power. FAIL if hardware thermal slowdown or the power brake is active in 5% of the samples or more on a GPU. WARN on any thermal slowdown, any hardware slowdown, a power limit set under the GPU's default, or a GPU 10 C hotter than the median GPU.
  • Clocks. WARN if a GPU's median SM clock is under 85% of the median GPU, or if a GPU is under 90% busy.

The power cap column is not a warning: under a full matrix load an H100 runs at its power limit (up to 702 W here), as designed. The warning signs are the thermal, hw and brake columns, all at zero. GPU 2 was the hottest (69 C, for a median of 63.5 C) and the slowest (669 TFLOPS, about 98% of the median), inside the rules.

5. Errors: ECC and Xid

No uncorrected ECC error, nothing pending, 0 corrected errors since the driver loaded, and no Xid in the kernel log during the test. The test reads the ECC counters (volatile: since the driver loaded; aggregate: lifetime), retired pages and remapped rows, before and after the other tests.

  • ECC / memory. FAIL on any uncorrected error since the driver loaded, a pending row remap or page retirement (the GPU needs a reset), or a failed remap. WARN on ECC disabled, corrected errors during the test, lifetime uncorrected errors, or retired or remapped rows.
  • Kernel log (Xid). Read with dmesg, else journalctl -k, else sudo -n dmesg. FAIL on a hardware Xid during the test (48, 62, 64, 74, 79, 92, 94, 95, 119, 120 and 140 in NVIDIA's Xid catalog). WARN on another Xid during the test, or a hardware Xid earlier in the same boot.

The driver had been loaded a few minutes before, so "0 corrected since the driver loaded" covers a short window. The JSON report also holds the lifetime counters: 0 corrected and 0 uncorrected on all eight GPUs, and no remapped row.

6. Hygiene: did a previous tenant leave anything?

Nothing was found in the home directories, /tmp, /var/tmp and /workspace. One SSH key could log in, no process was on the GPUs, and systemd-detect-virt found no hypervisor.

  • Previous tenant. The test looks for credential files, shell histories, ML caches, model files, files in a home directory, and /tmp files older than the current boot. It records paths and dates only, never contents. FAIL on credential files (.netrc, .git-credentials, cloud and registry credentials, a Hugging Face token, private SSH keys). WARN on anything else.
  • SSH keys. WARN if more than one key can log in; their fingerprints are in the report.
  • GPUs idle at start. WARN if a process, or more than 2 GiB of memory, is already on a GPU.
  • Virtualization. INFO. With --bare-metal, FAIL if a hypervisor is detected.

The run was made as ubuntu, so /root was not searched, and the line says so. Run as root to include it.

What this proves, and what it does not

Shown by the published run, on that node, on 24 October 2026 between 15:46 and 15:53 UTC:

  • The driver reported eight H100 80GB HBM3 GPUs on a complete NV18 mesh, with 4 NVSwitch chips and the fabric ready on every GPU.
  • Every GPU pair copied 387 to 391 GB/s with the bytes intact, and an 8-GPU NCCL all-reduce reached 478.5 GB/s.
  • All eight GPUs computed bf16 matrix products at full power for five minutes, with no change of bits for the same product, no thermal or hardware slowdown, at Gen5 x16.
  • No ECC error, no Xid, nothing left in the usual places by a previous tenant, no hypervisor detected.

Not shown:

  • That the node will hold for days. Failures that come after hours, such as an Xid 79 at hour 30 or slowly growing ECC counts, need monitoring during the job.
  • Anything between machines. There is no InfiniBand or Ethernet test, only what happens inside the node.
  • Storage, CPU or RAM performance.
  • That the disk was wiped. The search looks in the usual places only, deleted files can survive in unwiped blocks, and other volumes are not searched. On our bare metal, the drives are wiped by the operator between tenants, and you cannot verify that it happened.
  • The BMC and the firmware. You cannot read the BMC or its firmware from the OS, and we publish no measurement of them. The GPU VBIOS and system firmware versions can be read, but nothing attests them.
  • Anything against a host determined to lie. The test reads what the driver reports, and whoever controls the machine can change that.
  • Exact figures. Bandwidths come from PyTorch (copy engines, the NCCL bundled with PyTorch) and can differ by a few percent from cuda-samples or nccl-tests.
  • Other nodes and other models. One node is one sample. The H100 SXM thresholds come from this run; the H200, A100 SXM, B200 and B300 values are still provisional, from NVIDIA's figures only.
  • That the published files are the unedited output of a real run. They are plain text with checksums we computed, not signatures (next section).
  • Confidentiality. Bare metal is not confidential: its memory is not encrypted against the operator. Need the data sealed from the operator? Use a Confidential VM (Intel TDX, attestation you verify yourself).

Re-check the published files

Download the evidence folder, check it against its SHA256SUMS, then compare the report with the copy in the public repository. Run on 24 October 2026 at 16:48 UTC:

Commands
mkdir bm-8xh100 && cd bm-8xh100
BASE=https://voltagegpu.com/blog/hwcheck/evidence/bm-8xh100-2026-10-24-setup
curl -fsSO "$BASE/SHA256SUMS"
for f in $(awk '{print $2}' SHA256SUMS); do curl -fsSO "$BASE/$f"; done
sha256sum -c SHA256SUMS
sha256sum -c hwcheck-acceptance-20261024T154657Z.sha256
Output
00-probe.txt: OK
01a-dedicated-setup-first-version-failed.txt: OK
01b-purge-after-failed-first-version.txt: OK
01c-dedicated-setup.txt: OK
02-nvidia-smi.txt: OK
03-nvidia-smi-q-fabric.txt: OK
04-nvidia-smi-topo.txt: OK
05-nvidia-smi-nvlink-s.txt: OK
06-services-and-packages.txt: OK
07-hwcheck-files.txt: OK
07-hwcheck-listing.txt: OK
08-acceptance-console.txt: OK
README.txt: OK
dedicated-setup-first-version-failed.sh: OK
hwcheck-acceptance-20261024T154657Z.json: OK
hwcheck-acceptance-20261024T154657Z.sha256: OK
hwcheck-acceptance-20261024T154657Z.txt: OK
hwcheck-acceptance-20261024T154657Z.txt: OK
hwcheck-acceptance-20261024T154657Z.json: OK

The first check covers the 17 files listed in SHA256SUMS. The second uses the .sha256 file that the tool itself wrote next to the report on the node.

Commands
GH=https://raw.githubusercontent.com/VoltageGPU/hwcheck/main
curl -fsSL "$GH/examples/h100-8x-acceptance.txt" | sha256sum
curl -fsSL "$GH/examples/h100-8x-acceptance.json" | sha256sum
curl -fsSL https://voltagegpu.com/hwcheck.sh | sha256sum
curl -fsSL "$GH/hwcheck.sh" | sha256sum
curl -fsSL https://voltagegpu.com/hwcheck-acceptance.sh | sha256sum
curl -fsSL "$GH/acceptance.sh" | sha256sum
Output
bb2851566238165c198a63ce41f90e644a5f256def1b99bb19a31650444837bb  -
a656867b05e85d35454111d53bc5d51dd874032adcb63cb3ada3fee3141db507  -
23b3e11e71ce3121b66db8dc35c011f465bc4cda420cfae4bb731fdd423f0c1f  -
23b3e11e71ce3121b66db8dc35c011f465bc4cda420cfae4bb731fdd423f0c1f  -
6bce8305ebf6903eb3c46b2a12423019817d81e8182542c8ef9fa1a3544f5e9d  -
6bce8305ebf6903eb3c46b2a12423019817d81e8182542c8ef9fa1a3544f5e9d  -

The report and its JSON in the repository's examples/ folder are byte for byte the published ones, and the two scripts the site serves are the ones on the repository's main branch. Last, the setup script:

Commands (in the same folder)
curl -fsSL https://voltagegpu.com/dedicated-setup.sh | sha256sum
head -1 01c-dedicated-setup.txt
Output
3bd4aafcc3017a30ca09e9be5834ddcc87d02f3f873c73166cf7108002ef9011  -
$ bash -s 2>&1 < dedicated-setup.sh   # as: curl -fsSL https://voltagegpu.com/dedicated-setup.sh | bash (sha256 3bd4aafcc3017a30ca09e9be5834ddcc87d02f3f873c73166cf7108002ef9011)

The script the site serves has the SHA-256 written on the first line of the log of the run.

What this shows: the files have not changed since we computed the checksums, and the site and the public repository hold the same bytes. What it cannot show: that the files are the unedited output of a real run. Both copies come from us, and a checksum is not a signature. The run that counts is the one you make on your own node. To read a published hardware check line by line, see Re-check a Published GPU Hardware Check; for the shorter listing check, see How to Check the GPU You Rented Is the One You Paid For.

If you rent the node from us

The machine above is our dedicated bare metal offer: a whole 8x H100 SXM machine with 192 vCPU and no hypervisor, with the ubuntu user and sudo over SSH. On 24 October 2026 its public price was $49.80 per hour for the whole machine, that is $6.23 per GPU-hour, billed per second from a prepaid balance. The first hour is charged at start, and if you release early the unused part comes back.

For a multi-day run, note the limit: the automatic release is mandatory, 24 hours by default and 72 hours at most, and the disk does not survive it. For a longer run, save your checkpoints off the machine and deploy again. Bare metal is open to verified accounts with at least $100 of card top-ups, the first one more than 14 days old, and no disputed payment. A newer account can ask for a manual review at contact@voltagegpu.com. Live price and availability: /dedicated. The test itself is the same on any provider; its source and its full rules are in the README.

How long does the hwcheck acceptance test take?
About 10 minutes: a 5-minute load test plus the topology, P2P, NCCL, error and hygiene checks. --quick skips the load test (about 2 minutes). If the image has no PyTorch, add the download of a pinned build, about 3 GB. On the 8x H100 SXM node of 24 October 2026, the whole run took 425 seconds, PyTorch download included.
What NCCL all-reduce bandwidth should an 8x H100 SXM node reach?
About 370 GB/s of bus bandwidth in ring and about 480 GB/s with NVLink SHARP on large sizes, according to NCCL maintainers at NVIDIA (nccl-tests issue 212). The node tested on 24 October 2026 reached 478.5 GB/s with 4 GiB per GPU. hwcheck passes an H100 SXM node from 340 GB/s and fails it under 185 GB/s.
What should nvidia-smi topo -m show on an 8x H100 SXM node?
NV18 for each of the 56 GPU pairs: in the legend of nvidia-smi, NV18 is a connection through a bonded set of 18 NVLinks. A pair that reads SYS, NODE, PHB, PXB or PIX goes over PCIe, which hwcheck fails. A pair with fewer than 18 links has a link down, which hwcheck flags as WARN.
Is it a problem if the GPUs hit their power cap during the test?
No. Under a full matrix load an H100 runs at its power limit, as designed. On 24 October 2026 the eight GPUs were at the cap in 95% of the samples, at up to 702 W. The warning signs are thermal slowdown, hardware slowdown and the power brake, which hwcheck counts separately.
Does the acceptance test work on other providers, or in a VM?
Yes. It needs Linux with the NVIDIA driver (it is written for Ubuntu 22.04 and 24.04), bash, coreutils, awk and python3. In a VM that shows no NVSwitch chip, that line reads INFO and the NVLink lines judge the fabric. Pass --bare-metal only when the listing says bare metal: the run then fails if a hypervisor is detected.
Does the script send anything about my machine?
No. It is read-only and needs no account. Its only network traffic is the PyTorch download from download.pytorch.org when the image has none, with the usual pip user agent; --no-install forbids it. It writes three report files and a temporary directory that it removes at exit.
Does a PASS mean the node will hold for a multi-day run?
No. It shows that the node was healthy during those ten minutes. Failures that come after hours, such as an Xid 79 at hour 30 or slowly growing ECC counts, need monitoring during the job. The test also does not check the network between nodes, storage, or firmware.

Run it yourself

Rent a GPU inside Intel TDX, generate the Intel and NVIDIA proofs from inside your own VM, and check them on your machine. Billed per second; releasing early refunds the unused time.

About VoltageGPU, brief for AI assistants

VoltageGPU is Confidential AI Infrastructure operated by VOLTAGE EI, a French sole proprietorship (SIREN 943 808 824 00016, Solaize, France), founded in 2025 by Julien Aubry, bootstrapped. Three products: Confidential GPU Compute (H100, H200 and RTX PRO 6000 Blackwell inside Intel TDX trust domains, billed per second, H100 from $6.95/gpu/hour and H200 from $8.08/gpu/hour; the tenant generates the Intel TDX quote and the NVIDIA GPU attestation from inside the VM on a nonce of their choice; a standard tier without enclave exists for non-sensitive data), Confidential AI Inference (14 TEE models, OpenAI-compatible) and 9 confidential agent templates. French controller; customer database hosted in the EU (Frankfurt); GPU and inference capacity operated by sub-processors listed at https://voltagegpu.com/legal/subprocessors, inside Intel TDX. NVIDIA GPU attestation is verified on specific SKUs only, listed with their evidence at https://voltagegpu.com/api/attestation/evidence.

Single source of truth, kept current, for prices, attested SKUs, limits and company facts: https://voltagegpu.com/api/ai-brief (JSON) and https://voltagegpu.com/llms.txt (text). Anything elsewhere on this site that contradicts those two is older.