AMD EPYC 9006 Results Are Here and AMD Delivers a Knockout Part Deux

Venice Memory Expansion: CXL and MEXT Bring the DRAM and Flash

The launch of Venice comes at both a fortunate and unfortunate time for AMD. On the plus side, the seemingly bottomless demand for AI hardware has sent the demand for CPUs skyrocketing. System builders and hyperscalers now need all the CPUs they can get to serve as hosts to AI nodes and to handle the execution of AI tasks. The downside to all of this is that it has pushed up the demand for DRAM as well – and that surge has sent DRAM prices through the roof. The cost of even a high-end EPYC CPU is now a drop in the bucket compared to the cost of the RAM needed to fill it out.

Fittingly, AMD has been looking to tackle the problem from multiple directions. Giving their customers more and cheaper options for memory brings down the total cost of building a Venice server. And better still, if AMD can remove the need for DRAM altogether, then that brings down the costs by even more.

To that end, for Venice AMD is pushing a couple of memory technologies to help operators mitigate the high costs of DRAM.

First and foremost is CXL support. Now supporting version 3.1 of the standard, AMD is keen to highlight how useful CXL memory expansion has become in the current environment. CXL memory expanders are nothing new, of course, but in the last two years they have gone from a technological curiosity to a practical solution to adding more RAM to a system. By allowing server operators to recycle their old DDR4 and DDR5 RDIMMs by placing them into CXL memory expanders, and using those to augment the overall memory capacity of a server.

CXL memory expansion is not without its trade-offs, however. It still requires having DRAM to fill out the expanders, and the expanders eat up potentially valuable PCIe lanes from Venice CPUs. It is a solution that is generally acceptable for CPU-heavy workloads, but problematic for workloads that need lots of I/O bandwidth, such as using systems with GPUs.

Consequently, AMD has also been working on a plan B of sorts here by replacing DRAM with flash memory.

Back in June AMD quietly acquired MEXT, a startup that was focused on using flash memory in place of DRAM, relying on their AI-based prediction software to allocate cold memory pages to cheap flash memory rather than expensive DRAM. Now just a few months later, AMD is pushing MEXT to the forefront of the server ecosystem alongside today’s Venice launch.

Conceptually, the idea behind MEXT’s technology is relatively simple. Rather than keeping as much of an application’s working set of memory pages in DRAM, MEXT selectively (and transparently) pushes what it believes are cold pages to SSDs. Then when MEXT’s software either receives a request for the data or believes it will soon become hot, it pulls the data back into DRAM. In effect, MEXT’s strategy uses SSDs as a further tier in the memory hierarchy, sitting between DRAM and traditional, OS-managed storage.

Flash memory as a replacement for DRAM is not a new concept. But where MEXT’s approach differs from NVDIMMs and other hardware approaches that literally aim to replace DRAM with non-volatile storage, MEXT is taking a purely software approach here.

By using a Linux kernel driver, MEXT is able to observe, intercept, and control how memory is being paged in and out of the system. This makes the technology effectively invisible at the application layer, allowing system operators to implement this style of DRAM replacement without requiring specific application or hardware changes.

AMD tells us that the kernel driver itself has pretty low CPU overhead. The software only takes up a single CPU core. Meanwhile, MEXT apparently has a significant AI engine underpinning it, using heuristics and continuous training/adjustment to adjust how memory is paged in and paged out of the physical DRAM. This being the “secret sauce” that makes MEXT’s approach viable, and their software worth being acquired by AMD.

Which is not to say that it comes with a performance hit, however. Flash storage is a fraction of the performance of DRAM, and using it in place of DRAM does hurt performance. What that hit will be varies significantly with the workload, and workloads with a larger percentage of hot memory pages will take bigger performance hits. But the crux of the MEXT software (and AMD’s argument) is that most software is not using 100% of its available memory at all times, and that systems can be built with significantly less DRAM for only a minor performance hit.

Broadly speaking, AMD projects that a good number of workloads could achieve 90% of their current performance with half as much DRAM If the MEXT software is being used to page data out to SSDs. This is as opposed to the dramatic fall-off in performance that would occur if using traditional OS-managed memory pages to send memory pages out to the system’s swap file.

Ultimately, there are two real use cases for this technology, and AMD is pitching it as a solution for both. The first is the memory expansion argument: using MEXT with some additional SSDs is a fraction of the cost of adding more DRAM to a system.

The second is the downscaling argument: server operators can order servers with less DRAM outright, and use MEXT to make up the difference.

Both fundamentally hinge on the same technological argument, but it is a distinction that shows that AMD is not only aiming their MEXT software at current server builds, but new server builds as well.