AMD EPYC Genoa Power Consumption
On the power consumption side, Genoa is formidable. We generally saw between 1kW-1.2kW testing dual socket AMD EPYC 9654 systems. That may sound like a lot, but we will note that previous generation top-end Intel and AMD top-end systems were often in the 800W-1kW range.
We also tried bumping TDP from 360W to 400W. Performance was a single-digit percentage better, but we saw 100-120W more at the wall using that mode in dual-socket configurations. With modern air-cooled systems, adding more CPU TDP often comes with a 15-20% “fan tax” for additional power consumption.
On a performance-per-watt basis, this is phenomenal. AMD increasing performance often by 2-3x while increasing power consumption by 20-40% is a trade-off most organizations will make any day.
We are just going to quickly note here that this is not an area where looking at PSU/ PDU power consumption is very important. Increasing power density also increases air cooling requirements in a non-linear fashion. Further, a lot of AMD’s performance comes from the 12-channel DDR5. If we take just package power, we would look at the CPU’s increase in power for performance, but not the additional power being consumed by the memory to achieve that level of performance. In a 24x DDR5 server, it is not uncommon to see memory using well over 100W, or ~10% of system power. The difference between Genoa and Sapphire Rapids will also come at a system power cost of those additional memory channels.
Still, performance per watt has increased greatly with Genoa.
AMD EPYC Genoa Dual Channel DDR5 and More
One point that we learned is that AMD will not be supporting DDR5-4800 2DPC configurations in servers like ours at launch.
This is a symptom of the new DDR5 as well as having more memory channels. It is also why we received many OEM employees sending us notes so intrigued about our Microsoft OCP show Genoa pictures.
AMD’s platform maturity, generally, is not at the same level as Intel’s platform which is launching in 2023. That feels strange to say, but Intel’s Sapphire Rapids has been so delayed, and the Sapphire Rapids ES/QS chips being so prevalent, that vendors have systems that are ready, and are just waiting on chips.
So AMD has something that is more like a minimum viable product in terms of a platform, while Intel has something very mature, but it needs to produce more chips. AMD’s minimum viable product is still good enough (or much better than that) for the majority of the market. Also, fitting 48x DIMMs into a dual-socket server is not an easy feat itself, so there are practical physical constraints that will keep many servers at 1DPC only, like the QCT system we tested.
Market Impact 2023: Big Versus Small Servers
One of the more interesting impacts of Genoa is not at the high end. Instead, at the lower end of the market, Genoa is more challenging. The new EPYC CPUs require PCIe Gen5, with better materials for motherboards. Also, AMD’s new chips have 12 memory channels. For 96-core machines, 12 memory channels feel like the right choice. For a 16-core machine, it feels imbalanced.
Many servers reside outside of hyper-scale data centers. Many customers are not adopting DPUs in 2023. Others are not even adopting PCIe Gen5 NVMe SSDs or other devices throughout fleets. As a result, the new platforms are going to simply be too expensive.
Just for some sense taking a server with 64GB DIMMs and a 32 core CPU. In the AMD EPYC 7003 generation, that is 64GB DDR4 x 8. In the AMD EPYC 9004 generation, that is 64GB DDR5 x 12 to fill memory channels. Current spot pricing for DDR5 is down to around a 50% premium over DDR4. Adding 50% more modules at 50% higher prices is a reason we are seeing things like Non-Binary DDR5 capacities.
AMD’s bet is likely twofold. First, many customers are going to stay on AMD EPYC 7003 Milan in 2023. If an organization has under 512GB of memory per socket demands, does not need DDR5 bandwidth, and is using only 100GbE NICs and a few SSDs, then Milan is going to be fine and perhaps the better option. The second is really the upcoming AMD EPYC Siena line. Many applications in the world transition slower than technology advances. For many single socket 16-32 core servers, as an example, Genoa is too much and Siena will target a class of those use cases.
Whereas we have seen AMD transition for years to new platforms, with Milan, we saw lower-end Rome occupy lower-cost market segments for some time. Now with Genoa getting larger, AMD needs something to address the lower end.