If you are reading STH, there is a great chance that you have seen DDR5 pricing in the consumer market has spiked. For many segments, DDR5 pricing is going up as AI data centers are putting significant demand pressure on the DRAM supply. As a result, hyper-scalers are turning to the tens of Zettabytes of DDR4 they have purchased over the years to lower the impact. We went to Marvell to see its Structera X CXL Expansion in action. While we were there, we also got to see the Marvell Structera A that combines 16 Arm cores with the DDR5 CXL controller, so system builders can scale both compute and memory. There is a narrative that CXL is not suitable for AI, but we have a great example of how CXL memory can increase performance.
Note, we flew to Marvell’s labs to film this video, and got special access so we have to say this is sponsored.
Marvell Structera X DDR4 and DDR5 Controllers
While we were at Marvell, we got to see all three Structera chips. That includes the Structera X, which are memory expansion controllers, and then Structera A, which adds memory expansion, plus sixteen Arm Neoverse V2 cores. These are the high-performance Arm cores, and we always wondered how they worked. Not only did we see that, but we saw some examples of how “slow” CXL devices actually can speed up systems.
One huge feature in these is that the CXL devices also support LZ4 compression at line rate of the memory. Marvell told us that it has been seeing 1.8x to 2x compression ratio. Let that sink in. Not only can you add memory, but if you add it on the Structera controller, your effective cost per GB is roughly half versus if you were to install it elsewhere.
Starting with the Structera X 2404, this is a DDR4 expander with 4-channels.
For hyper-scalers, this is something that is extremely attractive because you get not only four DDR4 channels, but you can do three DIMMs per channel or 3DPC. In other words, you get twelve DDR4 DIMMs per Structera X controller.
If a hyper-scaler pulls 128GB DDR4 DIMMs from decommissioned servers, effectively making them “free” it can then have 1.5TB of memory on a single Structera X 2404 controller. With compresison that is roughly 2.75-3TB of effective capacity.
Another benefit to this model is that the hyper-scaler gets to recycle the memory meaning that to deploy this the only manufacturing that needs to happen is the cable, board, and the controller, but not the DRAM. For those with environmental impact targets (let us forget AI for a moment) the impact of this goes beyond just the cost savings.
The DDR4 version is more of a cost play, but one pays for that cost in performance since you have much lower operating speeds. For those who want CXL memory epxanders with higher performance, there is a DDR5 4-channel version called the Structera X 2504.
This also offers LZ4 compression, meaning that not only can you add more memory to a system, but the memory you add to this is effectively less costly than if you add it to a CPU’s DDR5 DIMM slots because you get more capacity per dollar.
The other benefit is that you get more bandwidth since CXL goes over the controllers and wires for PCIe not the DDR5 controllers and wires. As a result, this net increases both the memory capacity, but also the available memory bandwidth in a system.
If you were wondering how Marvell is doing this, think of a CXL controller using this block diagram. One side connectx to the CXL 2.0 / PCIe Gen5 lanes. The controller has the compression and decompression/ encryption decryption IP. It then has the memory controllers. These memory controllers in the DDR5 version are DDR5, and in the DDR4 version they can be DDR4. In theory, with CXL you can use different types of memory.
All of this is great, but it was time to look at the hot rod Structera A with 16 Arm Neoverse V2 cores built-in, and then get them running.
Without third-party testing of latency, bandwidth and latency while being bandwidth bottlenecked it’s difficult to know whether this is useful or not.