Google has New NVIDIA RTX Pro 6000 8-GPU Instances with AMD EPYC 9005 CPUs

Google Cloud announced today the preview availability of its G4 VMs. These VMs are neat for a few reasons, but perhaps the biggest is due to the hardware. As GPU instances, these new machines combine two high-end AMD CPUs, eight NVIDIA GPUs, fast networking, and plenty of memory.

Google has New NVIDIA RTX Pro 6000 8-GPU Instances with AMD EPYC 9005 CPUs

Unfortunately, Google did not provide pictures of the hardware. Luckily, this is STH, so we have seen or used a lot of the components. First, on the spec sheet, is using eight NVIDIA RTX Pro 6000 GPUs. These GPUs have 96GB of memory each (for 768GB total) but use GDDR7 instead of HBM3E, making them much less expensive than the NVIDIA H200 NVL or B200 GPUs. They also support features like RT cores, even on the server edition parts that Google is using, so they have pipelines available for graphics, rendering, and VDI workloads.

The GPUs are also significant insofar as this generation now supports NVIDIA MIG or multi- instance GPUs. For folks who want to partition the larger 96GB GPUs into up to four smaller chunks. This means in an 8 GPU system, one can have up to 32 GPU partitions with MIG.

Next is something we might not have expected. Google is using two AMD EPYC 9005 Turin CPUs per system to provide 384 vCPUs.

Along with these CPUs there is 1.4TB of system memory for approximately a 2:1 CPU to GPU memory ratio.

For networking, these machines are getting 400Gbps of network bandwidth with Trainium offloads.

Final Words

For some context, this is an interesting announcement on many levels. One of them is that the network bandwidth of 400Gbps is lower than we see many OEM systems configured with these days as 4x 400GbE links for East-West GPU traffic plus an additional BlueField-3 DPU for North-South traffic is common. Also, Google using AMD EPYC Turin is something that we know many customers are moving towards AMD EPYC CPUs even as NVIDIA has been pushing Intel Xeon in its DGX platforms.

Still, the 8x PCIe GPU systems have been around for ages, and have been popular for their lower cost and flexibility. Hopefully these are popular instances too.