Of all the product announcements made at this year’s AMD Advancing AI event, by far the most high-profile was the semi-formal launch of AMD’s Helios rack. The pride and joy of the company, Helios is a culmination of multiple AMD technologies, CPU, GPU, and DPU/NIC, along with its ZT Systems engineering, into a single product in order to build a computing appliance that can operate at a scale beyond what a single server can do. Or, as it has since been coined, rackscale systems.
Increasingly serving as the backbone of datacenter AI hardware, rackscale systems have come to the fore as AI workloads are too large for a single server, both in terms of processing power and memory capacity. The next step beyond a single server, then, is that rackscale systems are all about clustering multiple servers, and multiple types of servers, to function as a single coherent machine. It is a computing paradigm that, in part, has helped put AMD rival NVIDIA in GPU computing space.
The technology principles behind a rackscale system are reasonably straightforward, and simpler supercomputer clusters have existed long before them. Still, the execution needed to make a rackscale system work is another matter. A rackscale AI system not only needs CPUs and GPUs to orchestrate and process data, but they need a high-performance fabric to link those systems together. In that respect, the biggest advances in making rackscale systems a reality have been adding those kinds of fabrics to GPUs and then designing networking hardware to extend (scale up) those fabrics to connect to the other nodes within a rack.
After acquiring Pensando in 2022, four years later, all of the pieces are finally in place for AMD to strike out in the world with its first true rackscale system, Helios. The combination of AMD EPYC 9006 “Venice” CPUs, Instinct MI455X GPUs, and Pensando NICs/DPUs, Helios is the flagship technological achievement for AMD. The resulting significance of Helios to AMD on both a technical and a business level is difficult to overstate, and fittingly, the company has been promoting it for over a year now.
At long last, Helios and its constituent parts are now in production. And with that, AMD is giving Helios its day in the sun at Advancing AI with the deepest dive yet on Helios’s scale-up architecture. So let us jump in and see what makes Helios fly.
AMD Helios By the Numbers
We will start with a high-level overview of Helios’s hardware specifications.
Looking at the state of Helios a year after AMD first announced the rack, the shipping specifications for the system ended up being remarkably close to AMD’s original projections. The 72 GPU systems will offer 2.9 EFLOPS of peak MXFP4 compute performance. With those GPUs paired up with 432GB of HBM4 memory offering 23.3TB/second of memory bandwidth each, the cumulative rackscale system offers 31TB of HBM4 memory with a total memory bandwidth of 1.7PB/second.
That latter specification is also the starkest difference from AMD’s original projections: the bandwidth per MI455X GPU (and thus the whole rack) is about 21% higher than AMD originally planned. With memory bandwidth remaining one of the major bottlenecks in AI inference, any improvement will be most welcome by AMD and its customers alike.
Meanwhile, on the networking side, the rackscale system features a series of network connections for both scale-up networking within a Helios rack and scale-out networking for clustering multiple Helios racks if needed. The scale-up domain offers a massive 260TB/second of cumulative network bandwidth between the nodes, while the scaling-out domain offers a more modest 43TB/second of bandwidth beyond a Helios rack. In a historical context, those are huge numbers from a single rack.
