Gigabyte has a new duo of AMD EPYC powered PCIe Gen4 GPU compute servers on the market. The Gigabyte G292-Z44 and G292-Z24 offer not just the PCIe Gen4 connectivity from the AMD EPYC 7002 Series Rome CPUs, but also come equipped with Broadcom PCIe Gen4 switches. Previously, Gigabyte had PCIe Gen3 variants of the platform but with GPUs from AMD (Radeon Instinct MI50) and NVIDIA (A100) we are seeing a shift to PCIe Gen4 architectures.
Gigabyte G292-Z44 and G292-Z24
The primary differences between the G292-Z44 and G292-Z24 are in their socket configuration. The -Z44 is a dual AMD EPYC system while the -24 is a single socket solution. Each provides 8 DIMMs per CPU with the motherboard in the center of the chassis and eight PCIe GPUs flanking the sides.
Startin with the Gigabyte G292-Z24 we get four sets of Broadcom PCIe Gen4 switches each with 48 lanes being used (x16 to the CPU and 2×16 to the GPUs.) That means the CPU has a total of 64x PCIe Gen4 lanes to the GPUs in a 2:1 oversubscription model. Additionally there are two PCIe Gen4 x16 slots for high-speed networking such as InfiniBand or Ethernet at 100Gbps/ 200Gbps speeds.
The Gigabyte G292-Z44 is a dual-socket solution. The immediate changes are that the PCIe slots for GPUs and networking are split to either CPU. There are other subtle changes as well. For example, the block diagram for the -Z24 shows a Mellanox/ NVIDIA ConnectX-4 Lx onboard NIC. We are surprised this is not dual 25GbE instead of dual 10GbE given the CX4 Lx and PCIe x8 link. The -Z44 has 10GbE via an Intel X550-AT2 and half of the PCIe lanes going to that dual 10GbE solution. Another change is the front panel drive bays go from 6x SATA and 2x U.2 NVMe on the -Z24 to 4x SATA and 4x U.2 NVMe on the -Z44.
Another interesting point is that we lose the M.2 x4 and x2 slots on the dual-socket solution. Given both are 2U form factors, the choice between the two systems is likely going to be driven more by the CPU and RAM per GPU ratio that organizations like to have in their GPU compute solutions.
Final Words
Until Intel Ice Lake-SP Xeon generation platforms get into the market, AMD has a commanding lead for those who want to deploy PCIe Gen4 on x86 today. This is a subject we discussed in our The 2021 Intel Ice Pickle How 2021 Will be Crunch Time article. Gigabyte is capitalizing on this market dynamic with these new high-density GPU servers. For some, the question will be whether the PCIe form factor is still the right form factor over the custom SXM form factors that use NVLink instead of PCIe Gen4 for inter-GPU communication.
Not to be too pedan’ic, but your third paragraph begins with “Startin”.