First Thoughts
Wrapping things up, NVIDIA’s architectural detail dump today has given us a great deal to think about, both in regards to Vera’s architecture as well as its performance.
On the architectural front, by and large the details disclosed today back and further illustrate the kind of CPU that NVIDIA has designed. The company has said that they went about designing a high IPC CPU core for servers – and particularly for agentic AI workloads – and all of the architectural details presented today back that up. The Olympus CPU core is simply massive, not only in die size but in the width of both the processor front-end and backend. And NVIDIA has employed several (no doubt die-space consuming) features such as graph prefetching that further boost the performance of each CPU core. I do not believe that much of what is being presented today is going to catch regular CPU aficionados by surprise, but Olympus and the Vera CPU built from it appears to be a well-designed, high performance CPU architecture.
But the performance ramifications for this are complex, as there are multiple angles to look at this chip – and all of them valid. Once Vera is shipping and third parties are free to run benchmarks, I would not be the least bit surprised if indeed NVIDIA can best the current field of server chips in single-threaded integer performance. Certainly, Olympus/Vera seems designed to first and foremost win at that, and the workloads that NVIDIA is pitching Vera to reflect that design goal.
However, with just 88 CPU cores, even with a high IPC NVIDIA is not well-positioned to win at high-throughput workloads. In that case, current-generation chips such as AMD’s EPYC 99xx (Turin Dense) and Intel’s Xeon 6+ (Clearwater Falls) can already best Vera.
And therein lies the rub: these are comparisons to current-generation chips. The timeframe for Vera’s release means that NVIDIA is going to have, at best, a short window of opportunity before AMD starts shipping their EPYC “Venice” chips, which are based on their upcoming Zen 6 architecture. Venice is coming to compete with Vera, and with specs like 16-channel DDR5-12800, Venice at 88 cores should be within roughly 10-20% of the memory bandwidth of Vera. All the while Venice will bring its own IPC and single-threaded performance improvements. We also would expect Venice to have more PCIe Gen6 lanes than Vera, and with up to 256 cores per socket, the ability to reach much higher throughput overall.
For now, the best that NVIDIA can show is how Vera compares to the hardware that is already on the market and can be tested by them. For a true next-generation comparison, we will have to wait for the rest of the next-generation competition to become available. Until then, it leaves a lingering question of whether Vera is a big enough improvement over the current generation of server CPUs in order to favorably compete with the next generation.
If nothing else, Vera seems very well positioned to serve as Grace’s successor. Though the outcome of NVIDIA’s competitive ambitions remains up in the air for now, Vera is a major improvement over Grace at every level. All of which should put NVIDIA’s core Vera Rubin ecosystem on very solid footing as production systems begin shipping to customers later this year. In the meantime we look forward to learning more about Vera, Rubin, and just what kind of performance those final systems will bring.
It looks to me like someone set the colour scale on the core to core latency table so the entire table was green. Even so, the chip seems well designed for tightly coupled parallel workloads. Since AI is different than the cloud-style micro-service throughput targeted by existing processors, I think it makes sense for Nvidia to fabricate their own.