ASRock Rack 4U16X-GNR2 ZC
We have covered ZutaCore’s 2-phase liquid cooling solution several times on STH. Instead of just warming water in a liquid-cooling loop, ZutaCore exploits a phase change in its coolant to provide even cooling to the hottest components in servers. This
Just taking a look at the ZutaCore solution, you can see the blue chambers. One other thing to look for is the different sizes of the cooler and warmer lines. This is because of the phase change occurring in the cold plates.
Taking a look at the GPU tray, you can see that there are different-sized inlet and outlet hoses, and also different hoses than the DLC version. That is another good hint that there is a lot going on inside.
When I saw Ada’s photos of the 8-GPU tray, the full coverage of the HGX B300 8-GPU platform looks impressive.
Here is an overhead view that really highlights the size delta between the different phase lines.
You can see that the different components have their own blocks, including the NVIDIA Blackwell Ultra GPUs, NICs, and NVLink Switches.
In this generation, the OSFP cages are cabled, so cables run over the GPU tray, but there is a lot of room to route them.
This is a gratuitous photo just because of how different this is.
ZutaCore’s two-phase cooling is getting another look because it is waterless, which is useful in a world where concerns (albeit often misguided) about water use in data centers persist. Hopefully, you enjoyed these photos just as something a bit different.
STH Server Spider: ASRock Rack 4U16X-GNR2
In the second half of 2018, we introduced the STH Server Spider as a quick reference to where a server system’s aptitude lies. For this system, networking and form factor are major parts of the story.
Overall, this is a high-density GPU server with plenty of networking to ensure that the GPUs can communicate with other GPUs and also that storage can be securely delivered to the platform. With the onboard NVLink interconnect, there is actually more networking on this type of system than you would see in a 4U or 5U PCIe GPU server. At the same time, this is also decidedly not a storage or general-purpose CPU computing platform.
Final Words
Overall, this is a really neat server. It houses a massive NVIDIA HGX B300 8-GPU platform with two Intel Xeon 6 processors and 32 DIMMs. Beyond the onboard low-speed NICs and the rear north-south PCIe NICs, there are eight 800Gbps NICs on the system, making no mistake that it is a big AI server. At the same time, it was extra exciting to see the DLC and ZutaCore versions side-by-side, since many folks want to look at different cooling solutions today.
One of the neat parts of this server is that ASRock Rack managed to keep some of its hallmark nice-to-have features, like the option to use either front or rear management and onboard networking in the same system via internal Ethernet cables. That is always so nice to see in these systems. Hopefully, you liked this look at a big AI server node.
