AMD is on stage at Hot Chips 2026 detailing the AMD Instinct MI400 series GPU architecture, the compute that will drive the Helios rack-scale systems the company has been showing around the industry. AMD’s talk covers the MI455X silicon, the cache, memory, and compute upgrades over the MI355X, and the ROCm software stack AMD is pushing to stay competitive with CUDA. If you have not seen it yet, Ryan did an awesome in-depth MI400 deep-dive that goes into tons of detail.
This one is running live, so please excuse any typos while AMD presents.
AMD MI400 GPU at Hot Chips 2026
AMD opens with the argument that AI work is broadening from single-model training into a mix of frontier training, enterprise fine-tuning, and always-on inference. This scale curve runs from the roughly 65 million parameter Transformer in 2017, through GPT-4 class models around a trillion parameters in 2023, and on to the 10 trillion-plus agentic and reasoning models AMD expects this year, with the takeaway that infrastructure has to move data as fast as models scale.
AMD frames the MI400 family around Helios, the rack-scale AI infrastructure it expects to ship. Headline numbers are a 2.9 exaflop rack with 31 TB of HBM4 memory and 1.7 PB/s of HBM4 bandwidth across 72 GPUs, along with 260 TB/s of scale-up and 43 TB/s of scale-out bandwidth per rack. We have covered AMD’s double-wide Helios racks before, and this fills in the silicon behind them.
This basic building block is a compute tray that combines compute, host CPU, memory, and networking. Each tray holds four AMD Instinct MI455X EAMs fed by a single-socket AMD EPYC 9006 SP7 server CPU over Infinity Fabric, with UALoE links carrying scale-up traffic at 1.8 TB/s per direction per GPU and up to three AMD Pensando Vulcano 800 AI NICs per EAM handling scale-out.
Here you can see the Helios node from Advancing AI 2026:
At the center of the tray is the AMD Instinct MI455X, an enhanced modular chiplet design built from eight accelerator complex dies on N2 flanked by fabric and cache dies plus I/O dies on N3P. It packages 256 total active work group processors with 192 MB of global L2 and 12x HBM4 stacks running 432GB at 23.3 TB/s, and it connects through PCIe Gen 6 as well as 72 UALoE lanes pushing 3.6 TB/s. We have a full AMD Instinct MI455X and CDNA 5 deep dive for more on the GPU.
This packaging split is notable because each die type moves to the node that best fits its job. Compute dies on N2 sit under 3D hybrid-bonded XCDs for higher density per watt, while the N3P fabric, cache, and I/O dies, plus CoWoS-L packaging, tie the whole package around the twelve HBM4 stacks.
AMD boils the MI400 changes into three buckets: bigger memory and cache, faster compute, and less data movement.
AMD has its conceptual block diagram. It is useful for keeping the XCDs, fabric and cache dies, I/O dies, and HBM4 stacks straight as AMD walks through how scale-up and scale-out attach to the package. One of the challenges with this approach is ensuring you can get power to all of the ties as well as just cooling them.
The new MI455X substantially expands the on-chip memory hierarchy compared to the MI355X. AMD doubles per-SIMD vector registers and per-WGP local data share, and a 4 MB L2 broadcast arbitrator can amplify bandwidth by up to 4x. Main memory jumps from 288 GB of HBM3E to 432 GB of HBM4, a 1.5x capacity increase, while AMD lists roughly 2.9x the total HBM bandwidth. The L2/ Infinity Cache changes are a big one that AMD has focused on previously since that is an area that was re-architected in the SoC.
On the compute side, the headline is a peak MXFP4 figure of 40.26 petaflops, up to 4x the MI355X, with MXFP6 and MXFP8 each rated at 20.13 petaflops. Vector FP16 and matrix or vector FP32 both reach 315 TF, up to 2x, on a 256 WGP architecture. Native Wave32 execution and a new transcendental engine target lower dispatch latency and faster attention math.
AMD walks through the MX data formats it is adopting across the stack. This table lays out exponent and mantissa layouts from float64 down through the MXFP family, with shared-scale blocks that can now span 16 or 32 elements and a new fractional scale for MXFP4. Four-bit tensor LUT instructions let data stay in 4-bit memory and convert to 4, 6, or 8-bit compute formats.
This efficiency push is about keeping work and data in place. A Tensor Data Mover copies data asynchronously into local LDS without staging through registers, work group clusters, and L2 multicast cuts redundant traffic for operators like GEMM and Flash Attention, and a reworked command processor lowers dispatch latency so short kernels stay fed.
AMD moves data movement into dedicated DMA engines so transfers run in parallel with AI kernels instead of stealing compute cycles. Topology-aware HBM DMA automatically affinitizes traffic to UALoE links, which keeps software oblivious to data placement and spreads load across the 72-GPU fabric.
Software is where AMD is trying to narrow the ROCm gap. It positions the ROCm core framework as the integration point and is shipping AI Skills so popular agents such as Claude, Codex, Cursor, and Gemini become ROCm users, plus a Hyperloom tool for end-to-end workload optimization.
AMD closes the architecture section with ROCm.AI performance comparisons of the MI455X against the MI355X. It reports 20 TB/s of measured MLA decode bandwidth in FP8 for 3.8x higher performance, 20 petaflops of measured FP4 compute for 3.3x higher AI compute, 3.2 TB/s of scale-up bandwidth for 3.5x higher bandwidth, and 190 GB/s of scale-out bandwidth for 2x higher bandwidth. AMD also estimates a 2.4x gain in AI energy efficiency on the way to its 2030 goal of a 20x improvement in rack-scale efficiency.
AMD wraps up with the six threads it says carry the future of AI, from advanced packaging with SOIC, HBM4, and CoWoS-L, through fault-tolerant memory and confidential computing, to the 72-GPU rack-scale subsystem and a modular architecture built for frontier training and high-volume inference.
It is a dense presentation, but the through-line is clear. AMD pairs a substantially larger memory and compute envelope in the MI455X with the Helios rack fabric and a software stack designed to ease the CUDA migration.
Final Words
Since we have Ryan’s deep-dive, that is probably still the best resource, but it was good to see at Hot Chips 2026 as well. A lot of folks in the room here are different than they were in San Francisco a few weeks ago. AMD has a big GPU for this generation in the Instinct MI455X.
