There is a
second rack now.

On measured MLPerf Inference 6.0 results the two are close enough that workload matters more than badge. AMD's MI355X delivered 100,282 tokens per second on single-node Llama 2 70B in FP4, which is 97% of the NVIDIA B200 result and 93% of the B300. On GPT-OSS-120B at multi-node scale the ordering reverses: MI355X reached 1,031,070 tokens per second offline and 900,054 in server mode, or 111% and 115% of B200 respectively. The MI355X also carries 288 GB of HBM3E against the B200's 192 GB — 50% more memory per GPU, which matters when a model must fit in as few accelerators as possible.

Helios is AMD's rack-scale answer to NVIDIA's NVL72: 72 Instinct MI455X accelerators paired with 6th-generation EPYC Venice CPUs in a single rack, with volume deployments expected in the second half of 2026. AMD publishes 432 GB of HBM4 per GPU for 31 TB per rack and 19.6 TB/s of bandwidth per GPU, against 13.4 TB of HBM3E per rack for GB200 NVL72. Scale-up bandwidth is 260 TB/s on Helios versus 130 TB/s of NVLink on NVL72, and Helios adds 43 TB/s of scale-out through AMD Pensando Vulcano 800G NICs. The compute headline figures — 2.9 exaFLOPS FP4 for Helios against 1,440 PFLOPS sparse FP4 for NVL72 — are not directly comparable, because the two vendors do not state sparsity conventions the same way.

Because the published numbers use different conventions. NVIDIA labels GB200 NVL72 as 1,440 PFLOPS FP4 sparse and 720 PFLOPS FP8 dense, making the convention explicit. AMD's Helios figures of 2.9 exaFLOPS FP4 and 1.4 exaFLOPS FP8 do not state whether structured sparsity is included. A 2:1 sparse figure is exactly double its dense equivalent, so a reader comparing one vendor's sparse number to another's dense number can be off by 100% without noticing. This is why MLPerf submissions — run on identical models and datasets under audited rules — are the only apples-to-apples comparison available, and why peak FLOPS should be treated as a marketing ceiling rather than a procurement input.

For mainstream inference serving, largely yes. AMD's ROCm stack now supports PyTorch, Hugging Face, vLLM and SGLang, which covers the majority of production LLM serving deployments, and AMD's MLPerf Inference 6.0 submissions were produced on that stack rather than on custom research code. The remaining gap is breadth rather than capability: CUDA has a decade-plus lead in long-tail kernels, profiling tooling, and third-party library support, so bespoke or research workloads are more likely to hit friction on ROCm than a standard vLLM deployment is.

It changes how many accelerators a model needs, which changes the economics. A model and its KV cache must fit in aggregate GPU memory, so 288 GB per GPU against 192 GB means roughly a third fewer GPUs to hold the same working set, and at the MI455X's 432 GB it is a factor of more than two against a 192 GB B200. Fewer GPUs per replica means less tensor-parallel communication, better utilization, and lower cost per served token — which is why memory capacity per accelerator, not peak FLOPS, is frequently the binding constraint in large-model inference.