I wanted to just sit and discuss a perspective on the next-generation AMD EPYC “Venice” CPUs in the era of Agentic AI. We have been covering this topic on our Substack and did a big piece on AMD EPYC 9006 Venice last week, but I wanted to apply it to the next generation of AMD EPYC CPUs because I think AMD’s strategy on server CPUs is becoming much more expansive.
A Perspective on AMD EPYC 9006 CPUs and Agentic AI
A few weeks ago in the aforementioned Substack piece, we explored what it even means to be a data center CPU in the era of Agentic AI, because data centers run a lot of processors. We will spare you the details here, but imagine a 500W CPU with triple-digit core counts, huge amounts of PCIe and memory attached, and then a sub-6W dual-quad-core low-performance Arm processor with maybe 2GB or 4GB of memory attached, and you can argue many processors are in the data center. We have been highlighting this on STH for years, but the ASPEED BMC in most servers runs its own firmware, has its own memory, storage, and network connectivity, runs its own web server and APIs, and so forth, and sits inside a server. If you say that is a data center CPU, then does the same processor sitting in a rear door heat exchanger monitoring temps, pumps, and flow also count as a data center CPU? What about the one sitting in a power shelf? The list goes on, but we had to find a more relevant population, and this is what we came up with.
From there, we identified the types of nodes commonly found in those locations. We got some feedback on the Substack framework, so here is a bit more expanded view for folks to consider:
Since this is such a large topic these days, I wanted to put together a perspective on Venice mapped to some of these. People who sell CPUs often say that their chips are the best for “everything”, but usually that “everything” is within certain swimlanes. AMD’s own product lineup mirrors this since we have so many versions of the EPYC 9006 family.
First, we have the AMD EPYC SP7. In previous generations, this was the flagship or “high-end” configuration. Now, it is one of many that can be considered a flagship. Key stats are that you can get up to 256 cores and 512 threads (in a 600W part) or up to 96 cores and 192 threads at up to 5GHz (in a 500W part). While there is a 64-core 300W part, five of the nine public SKUs are 500W, and two more are 600W. The 1.6TB/s of memory bandwidth looks like 16 channels of 12800MT/s MRDIMMs.
The other side of Venice is the SP8 socket version that scales from 130W to 400W and from 8 to 128 cores.
At this point, if we are mapping to Intel Xeon, you can think of this like how there are the Intel Xeon 6700P and Xeon 6900P series, just with newer cores, newer memory standards, PCIe standards, and so forth. I am not sure how a 288-core Xeon 6900E series fares when there is an option in the market for a 256-core/512-thread version with much more memory bandwidth and I/O. I bring up that Intel Xeon 6 comparison because Intel’s product planning saw the need to bifurcate the market into a higher-end socket and a lower-end socket years in advance. AMD is now taking a similar approach.
I think this is a critical point, especially when everyone wants an agentic AI CPU. Part of the workload is in the AI Factory, but much of the load will also fall on other data centers. Having a big socket to chase maximum performance and density, and then a smaller socket to fit more traditional racks, seems like the way to go.
Then there is the chip that should make folks excited. With 1.1GB of L3 cache or 12MB L3 cache/ core, plus the 1.6TB/s of memory bandwidth and higher frequencies, the EPYC 9006X is also a SP7 CPU but is designed for more HPC applications. It succeeds the 2022 AMD Milan-X and 2023 AMD Genoa-X, with more modern cores and newer everything else.
The interesting thing is that the EPYC 9006X will come out a bit later, but it should be a monster. Let us put it another way, there are many in the industry that look at Meta Muse and its 2 vCPU environments (one core with SMT=2 on) as meaning that the goal is stuffing as many vCPUs into a socket. Others believe the right form is sub-100 cores per socket, maximizing memory bandwidth and per-core performance. The EPYC 9006X is what that second camp would want in an ideal world.
Then there is the AI host processor, the AMD EPYC 9006 LP, which brings LPDDR5X to the AMD EPYC line. To us, this feels like a direct competitor to an NVIDIA Grace or Vera CPU designed to go into a Helios Rack.
We have seen a number of 8x GPU AMD Instinct servers with the 192-core Turin Dense parts over the past year or so. Other systems we have tested have had 64-core or 128-core Turin parts. It seems like the answer for these rack-scale designs is hitting a power envelope at the expense of having the most cores in the racks. Technically, this is a Verano, not a Venice CPU, so we changed the title of this piece from EPYC 9006 from Venice to make it easier to slot the series into the data center CPU framework.
With that overview, it is time to map. We have a number of EPYC 9006 series processors and a chart, so now it is time to map.
I love these