AMD Ryzen AI Halo Agentic AI CPU Performance
We have seen the AMD Ryzen AI Max+ 395 quite a few times now. Still, we have the new AgentSTH V7 test bench rolling out, so we wanted to at least show where the AMD Ryzen AI Halo stacks up. Specifically, we wanted to see if the little box with big cooling has out-sized performance because it is from AMD itself. We are using the GMKtec EVO-X2 as one comparison point since that was the first Strix Halo system we tested. The next planned AMD Strix Halo review after this one will be the Minisforum N5 Max, a 64GB version, the first 64GB model we tested, and so we wanted to add that as well, roughly a year separating it from the EVO-X2 review. We will swap that Minisforum for the company’s Minisforum MS-S1 Max for the GPU/ AI testing, just because having half the memory means that we cannot run the same models/ context lengths with the 64GB system.
AMD Ryzen AI Halo LM Bench
Kicking off, we have the AMD Ryzen AI Max+ 395 LM Bench results, looking at the working set size versus latency.
A question we had was whether this was substantially different than other AMD Ryzen AI Max+ 395 systems, so here is a look at the 64B results.
You can see that comparing this to the GMKtec EVO-X2 and the Minisforum N5 Max, we see fairly similar results.
The N5 Max is a 64GB machine and is on the newer side, while the 128GB EVO-X2 is the first Strix Halo system we tested. All three look similar here.
AMD Ryzen AI Halo Core-to-Core Latency
We also wanted to do a quick core-to-core latency test. Here we have all three systems:
It makes sense that all three are relatively close, in terms of latency, but it was something we wanted to look at to see if AMD was doing anything special with the Ryzen AI Halo.
Estimated STH SPEC CPU2026 Performance
SPEC CPU2026 is the company’s new benchmark suite. Its predecessor, CPU2017, was a de facto industry standard in the server industry. We are running this using open-source compilers at -O3 optimization levels, so our results should not be directly compared to the official numbers on the SPEC website. Instead, these are benchmarks at lower compiler optimization levels and should only be compared to STH results.
We start with the estimated STH SPEC CPU2026 int rate results:
Here you can see that all of the systems are relatively close in terms of performance, but as we would expect, Minisforum has the best single-core performance from the bunch since the company’s systems are focused on performance, despite being a NAS system. Here are the subtest scores:
Moving on to the STH estimated fp rate side, we see a similar pattern:
Here are the subtests:
Overall, you can see the performance of the entire set of loaded chips fairly well here. Next, however, we wanted to get to our AgentSTH V7 testing.
Does AMD really even have a >10Gb NIC that they can put into these at a reasonable price point, even if they had enough free lanes? I see that they make a AMD Pensando Pollara 400, but they’re not really available on the open market so I can’t tell if it makes sense to put one into a $4k computer. The only one that I can see for sale anywhere is $3,500 on eBay. The bulk of the >100G NICs that get announced these days are DPUs, which really don’t make much sense in this context, and I just can’t see AMD shipping a Mellanox or Intel NIC in this sort of thing.
I really wish they’d included a PCIe slot or, better yet, an OCP slot for BYONetworking. That would let them enable clusters without having to force the connectivity cost onto every customer that doesn’t use it. Does Strix Halo have spare PCIe lanes or is it so narrowly focused that it only allows for an NVMe or two?
It would be nice to compare benchmarks with medium size (20B-50B) dense LLM on this thing to similarly priced Apple offerings with M4 MAX, M5 Pro and/or M5 MAX, alongside a GB10.
I couldn’t see from the pictures, but is the power USB-C wired up for data? My dream device would be compatible with a dock so I could move it around to different sites and only plug in a single cable. I know there was some AI box (one of those DGX Spark boxes, maybe the HP one) that actually listed a data speed on the power port, but I don’t have one to test it with a high power dock.
I couldn’t see from the pictures, but is the power USB-C wired up for data? My dream device would be compatible with a dock so I could move it around to different sites and only plug in a single cable. I know there was some AI box (one of those DGX Spark boxes, maybe the HP one) that actually listed a data speed on the power port, but I don’t have one to test it with a high power dock.
To reinforce Frank’s comment: What would have made this review really useful is a performance comparison with Apple and nVidia boxes.
I’m going to go a step further than Frank and justsomeguy and say that this review is actually useless without performance comparisons against other hardware.
and when you decide to add comparative llm inference benchmarks, may I suggest that you use MLX on Apple hardware, CUDA on NVIDIA and both ROCm and Vulkan on AMD (the results can sometimes be surprising)?
IMHO, this box is a little bit weird, I guess they’re testing the waters for Gorgon Halo and they can do a respin. But releasing a reference platform a year into production is weird. Just buy a Bosgame, GMKTEC or Minisforum, or wait for Medusa Halo.
@Scott Laird
Broadcom do a respectable line of >25G NICs, although they wouldn’t be cheap, or there’s Marvell DPUs.
Or be really weird and follow the methodology of the Mikrotik CCR2004 which is a switch chip with PCIe.
The other Strix Halo machines had dual NVMe, so there’s upto ~40G of PCIe bandwidth available.
This was an amazing review, the hardware part is fantastic and so brave. I don’t care for RGB but if I messed up that flat cable I would cry! 3999 USD for 128GB of ram pretty much. I don’t see how it’s any better than the mini pcs with the same specs.