AMD Helios Scale-Up Topology
Now that we have had a chance to look at the physical hardware making up a Helios rack, let us take a look at the system’s logical topology.
The use of switch trays within Helios means that each GPU is, at most, one hop away from any other GPU. In other words, never having to go through more than a single switch. AMD’s intention with this design was to not only offer the ability for every GPU to talk to every other GPU, but to keep latency to a minimum while doing so.
This topology also means that the bandwidth between GPUs is uniform, including even GPUs within the same node. Locality simply is not a thing for Helios. Going from one GPU to another will always require making a trip through a switch, ensuring that each GPU is equidistant from the next.
By the numbers, each MI455X GPU provides for 3 UALoE links to each of the 12 switch ASICs, which is where the 36 links-per-GPU figure for the scale-up fabric comes from. In total, this means each compute tray features 144 links for connecting to the switch nodes.
Ultimately, the purpose of this design is to allow for the entire rack to function as a single pod. While even the speedy UALoE links are not nearly as fast as each GPU’s local HBM4 memory, at 3.6TB/second in cumulative bandwidth, they are fast enough for remote memory access to be viable.
AMD Helios Fault Tolerance
Besides ensuring high performance for a Helios rack, the system’s architecture is also designed to give it a high degree of fault tolerance. With so much hardware packed into a single rack, it is statistically an inevitability that something will eventually fail. To that end, AMD has designed Helios to ensure that failures are bypassed or contained as much as is reasonably possible.
The highest degree of fault tolerance is in the scale-up networking layer itself. Since each GPU is attached to every switch ASIC in the rack, the system can fully compensate for any single failure in the scale-up network. In practice, that means routing traffic around failed links using the other 5 switch trays. This ensures that the rack as a whole stays online and that work continues uninterrupted, though the loss of a switch tray will reduce the rack’s aggregate bandwidth.
The entire failover process is automated, of course. As is the recovery process for transient networking errors. On this matter, the big thing emphasized by AMD is that the goal is to avoid losing a job altogether and requiring reloading it from a checkpoint, as even a temporary pause is preferable to restoring a checkpoint.
As for failures that are not in the networking layer, this is where failure isolation comes into play.
Internally, each Helios rack can be partitioned into multiple virtual pods (vPods). Each vPod, in turn, can be comprised of a single node or multiple nodes. In this partitioned configuration, each vPod is its own system that is fully isolated from the others in terms of hardware resources.
This virtual isolation, in turn, allows Helios to fail more gracefully if a compute tray failure (or a similar critical error) occurs. In that case, the failure is contained to only the vPod housing the failed hardware, allowing the other vPods to remain up and running without impact. This is still a fatal error for the failed vPod, and in that case, the workload will need to restart from a checkpoint after the failed hardware is repaired/replaced, but it allows for a contained failure that does not require taking down the whole rack. (And this is where the ease of hardware maintenance and the use of busbars also shine.)
All of this is backed by copious hardware health monitoring as well. Helios and its management software not only monitor fabric and tray failures but also keep tabs on the performance of the power shelves, the liquid-cooling system, and the thermal sensors placed throughout the rack. You tend to see many management devices in the rack just to provide all of this telemetry data.
While AMD did not go into detail about this during its deep dive, we are told that Helios also has provisions for leak detection. Usually, this is detection with the ability to quickly stop flow to avoid unmitigated leaks. That capability we have shown in other AI servers.
Final Words
Wrapping up our look at the AMD Helios architecture, if there is only one thing to take away from the system’s design, it is probably the impact of networking hardware on the larger rack-scale system. Not only is it a core component of building a rackscale system, but it is also the single greatest differentiator between Helios and earlier clusters based on MI3xx. Networking hardware is integrated throughout the system, from discrete NICs and DPUs to massive switch trays to UALoE links baked into the MI455X GPUs themselves. To be sure, Helios is a testament to many different AMD teams, but it is the integration of networking that has enabled AMD to finally scale up to this level – and it will be networking that will let them scale up further in successive generations, as well.
On which note, besides AMD’s ambitions for Helios in the immediate future, the rackscale system is also slated to be the first in an ongoing series of rackscale systems from the company. With AMD operating on a yearly cadence for its datacenter GPUs (and to a lesser extent with its DC CPU), the company is also putting Helios on an annual cadence.
Later next year, AMD will be releasing its first follow-up rackscale system, tentatively titled Helios 500. The 500 will combine AMD’s EPYC “Verano” CPUs (Zen 6 with LPDDR) with AMD’s next-generation Instinct MI500 GPUs. It will also incorporate new Pensando NICs and DPUs, which are currently going under the codenames “Como” and “Monza”. Unlike last year’s Helios preview, this year AMD is not disclosing any significant specifications about the next-generation rackscale system this far in advance, and for now, the bulk of AMD’s energy and attention is on Helios.
On that note, with all of Helios’s constituent components now in production, so are the full Helios racks. According to AMD, the first production Helios racks will ship to customers in Q3 of this year, with AMD continuing to ramp up production through the second half of next year. So while Helios’s production kicking off is an important moment for AMD, it is just the next step in bringing the rackscale system to market. Nonetheless, exciting times are ahead for AMD.
