Recently we had a Dell EMC PowerEdge R640 1U server and a Dell EMC PowerEdge R740xd server in the lab. Each had a relatively similar configuration so that provided a unique opportunity to test a long-held assumption in the server world: that a 2U server is more efficient than a 1U server. The theory behind this assumption is that with fewer larger fans less power is spent on cooling making two otherwise identical systems diverge in power consumption. We hear this stated as fact on a weekly basis so it seems like there is a large segment of IT that sees this as conventional wisdom.
Test Configurations
This is going to seem redundant, but that is indeed important for the exercise. You can read more about the 1U system in our Dell EMC PowerEdge R640 review.
The 1U test configuration:
- Server: Dell EMC PowerEdge R640 1U 10x 2.5″ Chassis
- CPUs: 2x Intel Xeon Platinum 8180
- RAM: 384GB in 12x 32 DDR4-2666MHz RDIMMs
- Storage Controller: PERC H740P
- SAS3 Storage: 5x Samsung PM1635a 400GB Mixed Use SAS3 12gbps SSDs
- NVMe Storage: 2x Samsung PM1725 1.6TB U.2 NVMe SSDs
- OS Storage: BOSS controller card with 2x 120GB m.2 sticks
- Networking: Mellanox ConnectX-4 Lx dual-port 25GbE
- Power Supplies: 2x Dell 1100W
The 2U test configuration:
- Server: Dell EMC PowerEdge R740xd 2U 24x 2.5″ Chassis
- CPUs: 2x Intel Xeon Platinum 8180
- RAM: 384GB in 12x 32 DDR4-2666MHz RDIMMs
- Storage Controller: PERC H740P
- SAS3 Storage: 5x Samsung PM1635a 400GB Mixed Use SAS3 12gbps SSDs
- NVMe Storage: 2x Samsung PM1725 1.6TB U.2 NVMe SSDs
- OS Storage: BOSS controller card with 2x 120GB m.2 sticks
- Networking: Mellanox ConnectX-4 Lx dual-port 25GbE
- Power Supplies: 2x Dell 1100W
As you can see, these are very similar configurations. The caveat here is that the Dell EMC PowerEdge R740xd has the greater front I/O capacity with a 24-bay solution versus 10-bay on the 1U R640. That adds circuitry and therefore adds power. At the same time, there were the same numbers of drives connected to each system.
Testing 1U Versus 2U Power Consumption
We conducted a simple test. We ran a computational workload on the dual Intel Xeon Platinum 8180 CPUs for a 24-hour period to ensure we were at full heat soak. We measured total work done over the final 23 hours and saw nearly identical performance (0.006% delta) so we were confident that the systems were cooling the CPUs well to run at similar performance. Here is a quick ~2-minute video with our findings.
From that video, here is the most important screen grab.
This is showing the same workload running on the Intel Xeon Platinum 8180 CPUs. The R640 is on the top while the R740xd is on the bottom. The key here is that with the same workload and same installed hardware, the 1U server is 5-6% more efficient.
Final Words
Testing conventional wisdom is often prudent. We hear that 2U servers are more efficient on a weekly basis so admittedly these were surprising results. At the same time, there is one factor which is likely leading to the difference. The Dell EMC PowerEdge R740xd has more SAS3 and NVMe backplane capacity. In turn, that means additional components that add to that platform’s power consumption. The Dell EMC PowerEdge R740xd also has the additional expansion capabilities that make it suitable for a wider range of applications.
Still, through this testing, we have a result that likely defies conventional wisdom that those in the IT industry have heard for years. Next time someone says 2U servers have bigger fans and thus have lower power consumption, send them this article.
Quick Question – was the same amount of work being done?
sure, these systems were heat soaked – but were they running the CPU’s at the same clock speed?
Was the 2U better at dissipating the heat and running the CPU at higher frequencies (which draw more power)?
the 2U might still be more efficient if it is doing more work
“The Dell EMC PowerEdge R740xd has more SAS3 and NVMe backplane capacity.” — which also means that its fans needs to be more powerful in comparison with R640. That may be the reason behind more Watts pulled from the wall…
@jason – same workload pegging each physical core to 100%. Over the testing period, 0.006% net delta in work done.
@KarelG – modern servers are able to adjust airflow based on cooling needs. Dell EMC PowerEdge 14th generation servers have a feature called “Multi Vector Cooling” which is the next evolutionary step here. Same drives each server was cooling. You can read more about PowerEdge 14th gen here.
This isn’t really a good test, there’s multiple variables including potentially different vendors for the fans. You need to isolate and reduce the number of variables for it to mean much.
That said, I think the power consumption thing is a rounding error for most use cases. You do get better density with 1U’s given the example above obviously. However fan noise on a 1U is notably higher.
Robert – the fans are different models, of course, because they are different sizes in 1U v. 2U. That will have more impact than vendors. Either way, both are Dell EMC configured options for the CPUs including high-performance fans for the 1U cooling 2x 205W TDP Xeons.
We also ensured a consistent inlet temperature to isolate as many variables as possible. Both servers had the same temps and humidity as they were run at the same time.
You are right that vendor to vendor the results may be different. Indeed, we have another vendor’s offerings inbound for AMD EPYC to test as well.
For this to have any meaning you have to test 1 rack of 1U servers vs 2 racks of 2U servers na get the power consumptions per rack.
Ok, but what about overall system noise? My IT CW has always been that 2u servers make less noise than 1u servers because 2u servers have larger, quieter, fans than the 1u models with itty bitty fans screaming their heads off. And 2u servers are better at surviving harsh (higher temp) environments better because the cooling is better when it really matters. Let me know.
@patrick thanks for confirming that the work done is tiny (insomuch it makes no difference).
really interesting article too!
Thanks for the article Patrick, very interesting. Keep up the good work!