For our power testing, we used AIDA64 to stress the NVIDIA RTX 3090 NVLink, then HWiNFO to monitor power use and temperatures.
After the stress test has ramped up both GeForce RTX 3090 GPUs, we see it tops out at 738 Watts under full load and 59 Watts at idle. Keep in mind that this is for the GPUs alone, adding the rest of the system like the Threadripper platform we closed in on 1,500 Watts from the wall.
Just for comparison purposes, this is in-line with what we see from complete servers, at the wall, with minimal storage, and with ~200-225W TDP CPUs.
A key reason that we started this series was to answer the cooling question. Blower-style coolers have different capabilities than some of the large dual and triple fan gaming cards.
Temperatures for the two GeForce RTX 3090 GPUs ran at 74C under full loads, and again this is for a single GPU. The bigger challenge is going to be getting enough airflow into a chassis to keep these GPUs cool. We would not suggest trying to squeeze this setup into a small case with poor airflow.
There are effectively two large buckets of performance we are seeing here. First, there are applications that are not designed to use multiple GPUs. These tend not to be workloads like we see in the deep learning and scientific fields. We can understand why NVIDIA would start a SLI phase-out. In the rare instances, we saw slightly worse performance with two cards than one card. Our sense is that one will purchase two GPUs because it meets the needs of their most demanding use case. If you have a workload that will spend 80-95% of the time being 80-95% faster, and another workload that will be 5-20% of the time but be 5-20% slower, that is still a net win.
For those applications such as our deep learning training and inferencing benchmarks, we get great results. This is directly due to the domain areas focusing on multi-GPU support and NVIDIA creating NVLink to address that market. Since these are GeForce cards, double precision math is not great. We also cannot use scale-out features such as GPUDirect RDMA. Still, the impact of moving from a GeForce RTX 2080 Ti to dual GeForce RTX 3090’s can yield a 2-4x performance gain in many instances which can absolutely be career-changing and yield better end work product if limited by deadlines.
Still, the key challenge remains availability. Indeed, that is part of the reason we have two cards that are different sizes. We wanted to highlight that just getting two cards, let alone matching cards to recreate our experience is difficult, many months after these cards hit the market. We validated that even if one has to get different cards from different brands that one can get a performance benefit, even if it creates physical connection challenges.