Intel Xeon 6 Platforms
Now, the challenge is that delivering four different high-level P/E-core/socket combinations (at least) means that there is a lot of variability in specs. You will see a lot of “Up to” here and that is because Intel is differentiating on things like features depending on the core type and SKU.
A great example of this is that the Xeon 6700E has 88 PCIe Gen5 lanes, but the slide says 96 lanes because it does not just cover the 6700E series. The P-cores, however, can get up to 136 PCIe Gen5 lanes for single-socket designs in the R1S configuration that we covered. Intel also has 12-channel memory with P-cores MCR DIMM support and more UPI links in the 6900 series for more inter-socket bandwidth.
At STH we have been talking about CXL a lot for the past several years. Intel Xeon 6 supports CXL 2.0 Type 1, Type 2, and Type 3 devices on the 64 lanes that support CXL.
Of course, there are also caveats to things like the CXL Type 3 devices as the P-core variants get interleaved memory options in both sockets, but the E-core variants do not. We have seen the CXL Heterogeneous Interleaved mode and it is really cool. Imagine having ~8 additional channels of memory bandwidth and running DIMMs and CXL memory modules both in a giant pool of memory with bandwidth striped across all of them. Sierra Forest does not get that. Instead, CXL memory can either be its own NUMA node or a hardware-assisted flat memory mode.
What we know in this generation is that the Type-3 devices for memory expansion are going to be the big driver for CXL adoption.
Running in flat memory mode, Intel can get more memory capacity, and potentially at a lower cost.
Just to make this a bit easier, using something like an Astera Labs Aurora A1000 card, we can simply put that in one of the PCIe Gen5 x16 riser slots.
We can fill the card with 4x 64GB DDR5 RDIMMs and get another 256GB of memory capacity at about the same latency as adjacent socket memory (e.g. the memory pool connected to the opposite CPU in a 2-socket server.) We can then do this four times and get an extra TB of memory in the server and roughly 8 channels of DDR5 worth of bandwidth.
CXL has another trick, however. Suppose you are a hyper-scale customer with loads of working DDR4 modules. In that case, you can use a compatible CXL DDR4 controller card and plug memory modules into your DDR5 server via a similar methodology to the above. We have a ton of DDR4 memory and cannot wait until we can get a CXL DDR4 memory shelf that we can cable (using retimers) and use with these systems.
We recently saw the Marvell Structera that can handle up to twelve DDR4 DIMMs so they can be used as a CXL 2.0 memory pool for DDR5 servers.
For hyper-scalers who have lots of DDR4 being decommissioned, this can be a big source of cost savings even if only 1/3 of memory is recycled from previous generations.
Next, let us get to the performance.
Wow can’t even hide Patrick’s love affair with Intel anymore can we ? Intel has not even properly launched this but yet it’s 128c Intel vs 96c Genoa, but AMD will have same 128c in 2 weeks time……just be honest finally and call it servingintel.com ;-)
Yawn… Still low on PCIe lanes for a server footprint when GPUs and NVME storage is coming fast and furious. Intel needs to be sold so someone can finally innovate.
Whether love or not, the numbers are looking good. For many an important question will be yield rates and pricing.
I wonder why Epyc is missing from the povray speed comparison.
One thing I’d like to see is a 4-core VM running Geekbench 6 while everything else is idle. After that Geekbench for an 8-core VM, 16-core, 32-core and so forth under similar circumstances. This sort of scaling analysis would help determine how well balanced the MCRDIMM memory subsystem is to the high-core-count processors–just the kind of investigative journalism needed right now.
As an asside, I had to work over eight captchas for this post.
The keyword would be availability. I checked just now, and these newer parts don’t have 1k Tray Pricing published yet. So not sure when would they be available. It felt painful to restrict the On-Premise Server procurement specification at 64 cores to get competitive bidding across vendors. Hope for the best.
It is hard not to get excited about competition, Intel has finally done it, they launched something faster than AMDs previous generation… Intel’s focus on AMX accelerations seems to have paid off, I guess we shall see when Turin launches in a few weeks.
@Patrick how do you manage to call ~53GB/S “closing in on” ~330GB/S? Even dual GR is slower by a factor of three.
Well there’s 90 minutes of my life well spent. I’d like to thank Patrick and the STH krew on this one.
Rodigas I didn’t get that sense at all. Intel’s the first to launch a 500W TDP part on a modern process and they’ve got cores and memory bandwidth so they’re winning for now. In Q4 when Turin is out we’ll say he loves AMD. It’s shaping up like Intel will at least show it’s competitive with Turin. That’s great for everyone.
Eric O – I’d like to see GB5 not 6. You’ve hit it before, GB6 is useless even for workstation class multicore.
Ram is right, these aren’t really available yet. Paper launch, or HyperScale only launch.
Emerth do you mean that 543391MB/S is much more than 330GB/S? The screenshots in the memory section show Intel’s far ahead. With MCRDIMMs adding 38% more bandwidth they’re getting close to 750GB/s on one CPU. So maybe they meant to say the Grace dual chip is almost up to a GR MCRDIMM single chip?
Intel’s doing a fine job. I can’t wait for 18A chips.
@RamShanker & francis:
– ASUS has a webpage up; search for ASUS “RS920Q-E12”, not quite for sale yet, but there’s a PDF.
– NextPlatform has published a guesstimate of 6980P U$24,980 and 6979P U$24,590; with lower prices for trays. Prices are fairly meaningless ATM with the competitor’s launch imminent.
> Intel is differentiating on things like features depending on the core type and SKU
Glad for the competition, but really wish they’d simplify the stack of SKUs. Is Intel On Demand gone?
Did Intel say why all the black ink on that Clearwater Forest chip?
That seems to be a very risky chip … gaa/bspd/hybrid bonding,18A all being introduced in the same chip. Did they actually demo one?
Never thought I’d see a vulture capitalist group (Apollo Global Management) investing in Intel. I thought Gelsinger was supposed to be Intel’s savior?
As others have pointed out, these seems a bit bias on the Intel side of things.
Yes, we’re all glad to see them finally getting their house in order and competing, but do better on containing your fanfare.
Wow. So many haters claiming bias. Go back and re-read the linked epic Rome review from 2019.
When I compare that to this one and all I see is that good products get good reviews (this one) and great products get great reviews (Rome). I also noticed how thankful Patrick is to have intel be competitive in the top of the line, which it is with this latest launch and how awesome it was back in 2019 to have AMD jump out and surpass intel just a few years after they were nearly bankrupt.
For detailed benchmarks I refer all to Phoronix – but a very nice piece by ServeTheHome.
many thanks, L
So, different kinds of “leadership” …
According to Micheal at Phoronix this year’s Intel 6980P is 12% faster than last year’s AMD 9684X.
But, the 6980P has 700 TDP and the 9684X has 400 TDP (while remembering that comparisons of their TDPs isn’t exactly equal) and AMD costs U$10K less. So, 75% more TDP and 5x more $ (unfairly comparing guestimated MSRP vs discount pricing). With the new Turin (coming RSN) offering moar Coors and a big bump over AMDs last generation; in the same socket.
Making a tortoise and hare comparison would be confusing as to who is who and who is ahead at a particular point in time.
We appreciate the effort it takes to put together these articles and enjoy reading them; except for the shameful core latency mushy pea soup image, while other sites has tack sharp puny numbers and a reasonable sized image file nonetheless.
I need Intel to go up so I can give Grandma her retirement back…
We all know Turin is coming. At least AMD now needs to push really hard instead of just coasting because Intel’s been so far behind. Let Intel have its weeks at the top.
On the plus Epyc now has some competition coming. The one big pain point will be software licensing where it’s licensed per Core.
What is up with that lscpu output for the SNC3 configuration? It reports:
Node 0: 43 threads
Node 1: 43 threads
Node 2: 73 threads
Node 3: 86 threads
Node 4: 86 threads
Node 5: 84 threads
And then threads 256-352 are completely unaccounted.
@emerth: I see 0.5TB/s in stream on 128 cores while NVLD seems to go to 0.6 TB/s — so I’d agree with “closing” here.
Intel fanboys forgot AMD Turin with 192 cores? That is always the case, Intel concentrated for quarter year profits instead of keeping R&D on good shape. Now better to concentrate selling factories to someones that need “old school” stuff. Game over. There could be some ligth if they could boost soon out 256 c, which is very unlikely. AMD will do it soon anyway, most likely minor change to just add 20 % more cores. But fanboys are fanboys and always forgotting the truth.