One major advantage that NVIDIA has over the rest of the industry is its NVLink and NVSwitch technology. While many often focus on the GPUs or AI accelerators themselves, NVIDIA has been pioneering a way to fit up to 16 GPUs in an AI server for years and then scale. AMD knows it needs to close this gap, so it is opening up its XGMI/ Infinity Fabric at a time when PCIe links are going to take on a role of higher importance in systems. The idea is that AFL or Accelerated Fabric Link will work over PCIe Gen7 links and allow AMD to scale up its accelerated computing architectures.
AFL on Broadcom Switches for AMD AI
As part of the recent disclosures around PCIe switches and retimers, we spoke to Broadcom about its PCIe switch plans. Broadcom told us that it plans to start sampling its 144 lane PCIe Gen6 switches in the second half of this year for 2025 use in servers. Those large switches will provide more connectivity for accelerators to have direct device-to-device communication. Broadcom also told us that it will support CXL in its family. We have covered some early CXL switches like the XConn XC50256 CXL 2.0 Switch Chip.
Perhaps the most exciting part of the discussion and materials centered around Broadcom expanding AMD Infinity Fabric over its future PCIe switches. In the next-generation Broadcom Atlas 4 line, AMD Infinity Fabric / XGMI will extend over a switched architecture using AFL. If you saw our Next-Gen Broadcom PCIe Switches to Support AMD Infinity Fabric XGMI to Counter NVIDIA NVLink piece, this is the implementation of that promise.
It appears as though in the PCIe Gen7 era, we will get AFL to support accelerator fabrics. Here is the Broadcom PCIe switch and retimer roadmap for where Atlas 4 switches fit.
If you have been sleeping on PCIe since the Gen3/ Gen4 era, we are getting into a cadence of much faster interconnect speed transitions to support all of the new compute and networking technologies coming.
Final Words
Now, the hard part is going to be the wait. We know that this is an AMD and Broadcom roadmap collaboration, but we still have many quarters to get our hands on these AFL systems. In the meantime, the Atlas 3 family is coming with PCIe Gen6 and CXL 3.1 support. Hopefully, we will be reviewing products with that family next year in 2025.
HurryUpAndWait( 4TheFuture ); // Standard IT conundrum
But way more exiting them my Groundhog Day(ish) decade in a data center: “Oh look, another pallet of PCIe 3 / 6-Gbit SATA servers to install”.
Too slow too late. It won’t make a dent in Nvidia NVSwitch business. By the time, AMD/Broadcom will use PCIe Gen7, green team will be on 1.6T photonics Serdes. sad…