The d-Matrix team is tackling AI inference with a key innovation. The company is 3D stacking DRAM and logic to provide a solution to lower costs and increase performance over HBM4. In the AI data center, HBM tends to have a huge influence on cost. It is also a technology that is a challenge for many, as NVIDIA can afford to get the highest-bin HBM, while others looking to save on costs get lower-speed parts. HBM cost and supply is a major AI industry challenge.
d-Matrix Pavehawk Brings 3DIMC to Challenge HBM4 for AI Inference
The d-Matrix Corsair is designed with 256GB of LPDDR5 and 2GB of SRAM as a chiplet-based inference accelerator. Unlike others, d-Matrix is focused on the AI inference side, not the training side. Its big idea is that instead of using expensive HBM, co-packaging the AI acceleration engines and DRAM can be a winning combination.
Today, the d-Matrix Pavehawk uses a top TSMC N5 logic die and then 3D DRAM stacked to tightly couple the DRAM and logic.
Just taking a look at the technology roadmap, we showed Corsair above, but the vision is to 3D stack multiple DRAM dies atop of each other to give more capacity and bandwidth.
d-Matrix claims that it will not only provide an order of magnitude better capacity and performance, but also at a lower power to transfer data.
Final Words
Given that this is a multi-year journey for d-Matrix, it seems like this was a team in the industry that started working on the memory wall problem fairly early. A good indication that large AI shops are looking at more tightly coupled memory and compute is that we keep seeing examples of this high-level approach such as CXL accelerators with built-in memory controllers and compute. Of course, d-Matrix is taking the next step and building custom silicon solutions to build a high-performance and high-capacity line.
