The AMD EPYC 7251 is an absolutely fascinating chip. At around $520 street price at the time of this writing (expect this is heading lower), it is also the least expensive AMD EPYC CPU you can purchase to use in a dual socket AMD EPYC platform. The lowest bin SKUs are generally not for the performance minded. Instead, they are generally used to “light the platform” or simply light all of the I/O in a system. We decided to run some benchmarks on the lower-end CPUs just to see what kind of CPU performance we would get in Linux.
Reviews thus far, including content at STH have focused on the high-end AMD EPYC 7601 benchmarks. That chip costs about 8-10x as much (each) so we generally would see that as a lower volume part based on pricing. The AMD EPYC 7251 on the other hand, is priced alongside the heart of the Intel Xeon Silver line. As a result, we see this as a significantly higher volume part. We are going to focus on dual socket performance here. Soon we will have single socket performance figures.
Key stats for the AMD EPYC 7251 CPU: 8 cores / 16 threads, 2.1GHz base and 2.9GHz turbo with 32MB L3 cache. The CPU features a 120W TDP. Here is the AMD spec page. Essentially for around $1000 (for two CPUs) one gets 16 cores/ 32 threads and can address up to 4TB of DDR4-2400 RAM.
Here is what lscpu output looks like in a dual socket configuration:
One item we wanted to address is that the EPYC 7251 differs from many of the higher-end SKUs in one key area: memory speed. The EPYC 7251 official spec says DDR4-2400 is the highest memory speed supported.
We confirmed this by trying the chips with DDR4-2666 DDR4 RDIMM modules that run at full speed with the EPYC 7601. They clocked down to DDR4-2400 as we would expect. With the AMD EPYC architecture, DDR4-2400 v. DDR4-2666 has a direct impact on the Infinity Fabric speed.
For our tests we have been using a Supermicro Ultra platform configured as follows:
- System: Supermicro 2U Ultra EPYC Server (AS-2023US)
- CPUs: 2x AMD EPYC 7251 8-core/ 16-thread CPUs
- RAM: 256GB (16x16GB DDR4-2666 Samsung RDIMMs at 2400MHz)
- OS SSD: Intel DC S3710 400GB
- OS: Ubuntu 17.04 “Zesty” Server 64-bit
- NIC: Mellanox ConnectX-3 Pro 40GbE
The Supermicro 2U Ultra platform we have been using a number of readers have taken delivery of. We believe it is one of the first commercially available dual socket EPYC systems and have heard that readers have been taking delivery for a few weeks already. We expect other vendors to follow suit soon.
We are using Ubuntu Ubuntu 17.04 “Zesty” but have found Ubuntu 16.04.3 with the HWE kernel to work as well. Alternatively, we suggest upgrading the kernel to 4.10 or later if you must use 14.04 LTS. Our general policy is to use standard Ubuntu LTS and CentOS releases but AMD EPYC is getting more performance from newer software ecosystems. This is normal and we expect EPYC to get performance gains as software optimizes for the new CPU architecture.
We normally only publish power consumption figures for system reviews. Just to give you an idea of power consumption, we are seeing around a 110W power draw at idle and 300-305W maximum power consumption on our 208V 30A circuit. With power consumption figures and servers, you need to take into account environmental factors as they have a major impact on power consumption. These measurements are taken at 17.7C and 73% RH.
Dual AMD EPYC 7251 Linux Benchmarks
For this exercise, we are using our legacy Linux-Bench scripts which help us see cross platform “least common denominator” results. We do have a full set of expanded benchmarks from our next-gen test suite (Linux-Bench2) that are finishing regressions. We are starting to add some of those results into this review, and expect to see us continue to expand results soon.
Python Linux 4.4.2 Kernel Compile Benchmark
This is one of the most requested benchmarks for STH over the past few years. The task was simple, we have a standard configuration file, the Linux 4.4.2 kernel from kernel.org, and make the standard auto-generated configuration utilizing every thread in the system. We are expressing results in terms of compiles per hour to make the results easier to read.
Overall this is good performance. We do want to point out that there are a few competitive vectors for the AMD EPYC. First, is Intel Xeon Scalable systems. Second, are the single socket AMD EPYC systems as well as the dual socket EPYC 7281 platforms. The lowest end “light the platform” SKUs generally have a significant step up in performance by spending slightly more. See our Intel Xeon Silver 4108 benchmarks for an eample.
c-ray 1.1 Performance
We have been using c-ray for our performance testing for years now. It is a ray tracing benchmark that is extremely popular to show differences in processors under multi-threaded workloads.
AMD Zen architectures do extremely well in this type of workload. Here we see that AMD obliterates Intel Xeon Scalable. That is why AMD focuses on this type of workload for its marketing.
7-zip Compression Performance
7-zip is a widely used compression/ decompression program that works cross platform. We started using the program during our early days with Windows testing. It is now part of Linux-Bench.
Here the AMD EPYC 7251 is performs extremely well. For perspective, it falls between the dual Intel Xeon Silver 4110 and 4114. At this point, we did want to pause and mention that the hardest part about recommending the EPYC 7251 for performance purposes is that the AMD EPYC 7281 is slightly more expensive on a total system perspective, but is significantly faster. First-level thinking may uncover that the EPYC 7281 is around $750 or 50% more. The more appropriate view is at the system level. If you have a $5000 server, accessing that additional performance is only a 10% upgrade.
NAMD is a molecular modeling benchmark developed by the Theoretical and Computational Biophysics Group in the Beckman Institute for Advanced Science and Technology at the University of Illinois at Urbana-Champaign. More information on the benchmark can be found here. We are going to augment this with GROMACS in the next-generation Linux-Bench in the near future. With GROMACS we have been working hard to support Intel’s Skylake AVX-512 and AVX2 supporting AMD Zen architecture. Here are the comparison results for the legacy data set:
We have GROMACS results later with AVX2 and AVX-512 results. Here again, we can see a strong performance by the dual AMD EPYC 7251 configuration.
Sysbench CPU test
Sysbench is another one of those widely used Linux benchmarks. We specifically are using the CPU test, not the OLTP test that we use for some storage testing.
In this test the dual AMD EPYC 7251 falls closer to the dual Intel Silver 4108 results.
OpenSSL is widely used to secure communications between servers. This is an important protocol in many server stacks. We first look at our sign tests:
Here are the verify results:
Here the dual AMD EPYC 7251 configuration is close to the dual Intel Xeon Silver 4110. Those CPUs are similar in terms of price.
UnixBench Dhrystone 2 and Whetstone Benchmarks
One longest-running tests are the venerable UnixBench 5.1.3 Dhrystone 2 and Whetstone results. They are certainly aging, however, we constantly get requests for them, and many angry notes when we leave them out. UnixBench is widely used so we are including it in this data set. Here are the Dhrystone 2 results:
On the whetstone side here are the results:
Here perhaps the interesting chart. The AMD EPYC 7251 is fairly close to the performance of a single Intel Xeon Gold 6132. At the same time, it is between the dual Silver 4110 and 4114 results.
GROMACS STH Small AVX2/ AVX-512 Enabled
We have a small GROMACS molecule simulation we previewed in the first AMD EPYC 7601 Linux benchmarks piece. In Linux-Bench2 we are using a “small” test for single and dual socket capable machines. Our medium test is more appropriate for quad socket machines. Our GROMACs test will use the AVX-512 and AVX2 extensions if available.
There is a glaring take away from this result set. Disabling the second AVX-512 unit on the Xeon Silver line, as well as the lower clock speeds, mean that AVX performance is hurt in the Xeon Bronze and Silver lines. The castration of AVX-512 performance on Xeon Silver allow the AMD EPYC 7251 to perform very well in its price bracket. At the same time, the Intel Xeon Gold and Platinum lines have a huge performance boost. Intel is pushing AVX-512 for reasons illustrated here but leaving it out of the Xeon Silver line was a competitive mistake versus the AMD EPYC 7251.
Chess is an interesting use case since it has almost unlimited complexity. Over the years, we have received a number of requests to bring back chess benchmarking. We have been profiling systems and are ready to start sharing results:
Here we see results that are about in-line with what Intel is offering here with the dual Silver 4110 in the same price range.
Some folks may disagree, however, we do not see the AMD EPYC 7251 as the dual socket SKU that is going to have Intel most concerned. The single socket picture may be a different story. At the same time, performance is completely acceptable. One has to take into account that these are just over $500 apiece right now. With $1000 worth of processors, one could serve up to 4TB of RAM. To put that into perspective, to get to that ram level, one would need a four socket Intel Xeon Scalable system and to be paying for M SKUs where each processor has a $3000 premium over its base price. If you are buying over $100,000 worth of RAM, we would likely suggest moving up the EPYC SKU stack as AMD has much more performant parts.
Likewise, in terms of I/O, Intel does not have OPA on Bronze and Silver SKUs so it has no way to add up lanes and get 128 PCIe 3.0 lanes in a dual socket Xeon Bronze or Silver server. AMD simply out scales Intel at the platform level, a difference that will be exaggerated when we look at the single socket solution.
At $500 Intel essentially has two SKUs, the Intel Xeon Silver 4110 and Intel Xeon Silver 4112. In most cases, the Intel Xeon Silver 4112 is completely outclassed by the AMD EPYC 7251. Conversely, the Intel Xeon Silver 4110 is slightly less expensive which may make it a better value.
Another key takeaway is what AMD is clearly lacking here: a low price SKU. Servers purchased with one or two of the absolute lowest-spec Xeon CPUs, make up a not insignificant number of unit sales at major OEMs. These CPUs were previously the Xeon E5-2603 / 2609, and now the Xeon Bronze series (check out our Intel Xeon Bronze 3104 v E5-2603 V3 and V4 Three Generations Compared piece.) AMD still needs a four die package to get full EPYC I/O and therefore we suspect $200 and $300 are harder price targets to hit in MCM form factors.
The $500 / CPU dual socket market is a bit of an oddball. It is for those buyers who are not looking for the absolute lowest cost CPUs, but instead are looking for a bit more performance while enabling the full platform. The industry has teams of people dedicated to figuring out how to entice potential buyers to move from the $200-350 CPU range up to $500. Likewise from $500 up to $750+. We suspect AMD’s OEM partners are going to have many discussions that start with “I want the EPYC 7251” and end with “the EPYC 7281 is a better fit for us.”
AMD’s other position, given its unique architecture, is that unless RAM capacity is a concern, the single socket only 7000P parts are very attractive. Under 512GB RAM per system, moving from two EPYC 7251’s to a single EPYC 7401P is going to be about similar once platform costs are taken into account but the 7401P will have more performance. Stay tuned for that.
Overall, the AMD EPYC 7251 is not the chip that is going to garner the most headlines. If you are looking for the head turner, see the dual socket AMD EPYC 7601. At the same time, performance is good. If you are budgeting $1000 or so for two CPUs, this is a good option. Unlike looking toward an alternative architecture such as ARM, AMD has a platform that will run just about everything out of the box. We are already seeing signs of the software ecosystem maturing which is great to see.
Now that we have looked at the top and bottom of the AMD EPYC dual socket offerings, we will filling in the picture with additional single and dual socket EPYC and Xeon Scalable configurations over the next few weeks. The key takeaway here is that the AMD EPYC 7251 is extremely competitive in terms of performance in its price bracket.
This is f&@^*ng awesome. The gold here isn’t just the numbers. There’s the first dual Silver and dual 7281 here too.
Thanks for the covert power numbers. I’m impressed with the performance and power
You’re right ——- AMD NEEEDS a cheaper CPU. $500 even $450 is too much per for the entry.
Thanks for the review and pointing out that this is probably the most useless AMD EPYC CPU.
I would have understood a 7251P(single socket) for let’s say $ 350..400.
1 EPYC7351P for $ 750..800 already beats this dual socket system in every way(except the 4 TB memory).
XEON Bronze looks more like a waste product of intel Skylake SP.
Try to demonstrate the real power of 8 and 16 core epycs next time: they still have 16 memory channels across two sockets, which is a godsend for people running memory bound codes. And there are many, especially among engineering R&D folks.
My limited stream testing shows that 16 epyc threads are already able to saturate all 8 channels on a socket so these “cheap” epycs are just perfect.
So 7251 is a 4 die chip competing against Intel’s one die 8 core chip?
Can AMD make any money on that?
One would assume these are binned chips so they are getting revenue instead of $0 for a die.
I agree. These look OK but the epyc 7281’s are giving me a massive stiffy. Where’s my dell system so I can buy a few 2s?
Wow saving 2x$200 to spend it on 16 memory modules, well when you use $60 4GB modules, it makes sense, maybe even with $120 8 GB modules.
Great review STH. I can’t wait to buy these.
Does anyone know if there´s a chance to use 2 different epycs on a dual socket mainboard? i´m planing to buy a epyc 7281 in combination with a dual socket supermicro board and maybe in a year or something a second cpu. What also would be interesting if there are limitations when i use a dual socket mainboard with only one cpu? Thanks in advance. Georg
It makes sense when you work on ~200GB of data, where you need to read all of it, do simple calculation on each element and then store all of it, again and again for each time step. CPU caches don’t help you here, the only thing that works is raw memory bandwidth. So pairing two cheap epycs with 16x16gb dimms is the way to achieve a nicely balanced and effective CFD system.
Also, licensing costs for commercial cfd solvers are per core and much MUCH more expensive than hardware so low core count makes sense in this way as well.
Love that you guys did this with the lower SKU. Just a shame that EPYC can’t take higher clocked RAM. Isn’t JEDEC DDR4 at 3200MHz now? This platform as mentioned would benefit so much from that.
@Georg from a video I saw, sadly, no you can’t. That would have been really interesting.
@CheapMeat I just took a look at the block diagram from “my” mainboard and it answered all my questions. With 1 CPU installed i´m losing only one pcie slot and 2 nvme ports. the rest comes from cpu1 what is great. lan, ipmi, usb everything. so i shouldn´t have any problem with only 1 CPU installed.
This article helped me get the PO approved for two dual EPYC 7281’s. Thanks STH! Just what we’ve been needing.
Do more EPYC
Great work. More EPYC I’m applying a second motion to.
“At around $520 street price at the time of this writing…”
Where is the street on which I can purchase an EPYC? Serious question. All the online stores that I’ve found (eg SabrePC) list it as pre-order.
What does the power draw at load look like for the Epyc 7251 vs Xeon Silver 4110?
well , where is that 2P main boards only what i can find today is SM brand , nothing else ….