Dissecting Intel's EPYC Benchmarks: Performance Through the Lens of Competitive Analysis
by Johan De Gelas & Ian Cutress on November 28, 2017 9:00 AM EST- Posted in
- CPUs
- AMD
- Intel
- Xeon
- Skylake-SP
- Xeon Platinum
- EPYC
- EPYC 7601
Enterprise & Cloud Benchmarks
Below you can find Intel's internal benchmarking numbers. The EPYC 7601 is the reference (performance=1), the 8160 is represented by the light blue bars, the top of the line 8180 numbers are dark blue. On a performance per dollar metric, it is the light blue worth observing.
Java benchmarks are typically unrealistically tuned, so it is a sign on the wall when an experienced benchmark team is not capable to make the Intel 8160 shine: it is highly likely that the AMD 7601 is faster in real life.
The node.js and PHP runtime benchmarks are very different. Both are open source server frameworks to generate for example dynamic page content. Intel uses a client load generator to generate a real workload. In the case of the PHP runtime, MariaDB (MySQL derivative) 10.2.8 is the backend.
In the case of Node.js, mongo db is the database. A node.js server spawns many different single threaded processes, which is rather ideal for the AMD EPYC processor: all data is kept close to a certain core. These benchmarks are much harder to skew towards a certain CPU family. In fact, Intel's benchmarks seem to indicate that the AMD EPYC processors are pretty interesting alternatives. Surely if Intel can only show a 5% advantage with a 10% more expensive processor, chances are that they perform very much alike in the real world. In that case, AMD has a small but tangible performance per dollar advantage.
The DPDK layer 3 Network Packet Forwarding is what most of us know as routing IP packets. This benchmark is based upon Intel own Data Plane Developer Kit, so it is not a valid benchmark to use for an AMD/Intel comparison.
We'll discuss the database HammerDB, NoSQL and Transaction Processing workloads in a moment.
The second largest performance advantage has been recorded by Intel testing the distributed object caching layer memcached. As Intel notes, the benchmark was not a processing-intensive workload, but rather a network-bound workload. As AMD's dual socket system is seen as a virtual 8-socket system, due to the way that AMD has put four dies onto each processor and each die has a sub-set of PCIe lanes linked to it, AMD is likely at a disadvantage.
Intel's example of network bandwidth limitations in a pseudo-socket configuration
Suppose you have two NICs, which is very common. The data of the first NIC will, for example, arrive in NUMA node 1, Socket 1, only to be accessed by NUMA node 4, Socket 1. As a result, there is some additional latency incurred. In Intel's case, you can redirect a NIC to each socket. With AMD, this has to be locally programmed, to ensure that the packets that are sent to each NICs are processed on each virtual node, although this might incur additional slowdown.
The real question is whether you should bother to use a 2S system for Memached. After all, it is distributed cache layer that scales well over many nodes, so we would prefer a more compact 1S system anyway. In fact, AMD might have an advantage as in the real world, Memcached systems are more about RAM capacity than network or CPU bottlenecks. Missing the additional RAM-as-cache is much more dramatic than waiting a bit longer for a cache hit from another server.
The virtualization benchmark is the most impressive for the Intel CPUs: the 8160 shows a 37% performance improvement. We are willing to believe that all the virtualization improvements have found their way inside the ESXi kernel and that Intel's Xeon can deliver more performance. However, in most cases, most virtualization systems run out of DRAM before they run out of CPU processing power. The benchmarking scenario also has a big question mark, as in the footnotes to the slides Intel achieved this victory by placing 58 VMs on the Xeon 8160 setup versus 42 VMs on the EPYC 7601 setup. This is a highly odd approach to this benchmark.
Of course, the fact that the EPYC CPU has no track record is a disadvantage in the more conservative (VMware based) virtualization world anyway.
Facebook
Twitter
RSS
105 Comments
View All Comments
Topweasel - Tuesday, November 28, 2017 - link
Yeah I want to give them the benefit of the doubt and I have no problem with them posting numbers even as analyzation of Intel in regards to EPYC. But a full page "review" of Intel's Epyc benchmarks as a product is kind of schilly. I mean where is their tests to back up the information? Where are the counterpart test where they test something similar that wasn't handpicked by Intel. How can any company assess the validity of a product based solely off of it's competitors testing of the product?bmf614 - Tuesday, November 28, 2017 - link
If you could actually get ahold of Epyc they would probably review the hardware themselves but as of yet it is a paper launch.supdawgwtfd - Wednesday, November 29, 2017 - link
It's not a paper launch dipshit.They can bearly keep up with orders for large companies.
Ryan Smith - Wednesday, November 29, 2017 - link
To throw some context in here, the purpose of this article isn't to publish Intel's benchmarks. Rather, it's commentary on what has been a very unusual situation.Up until now, neither AMD nor Intel have engaged in any serious Skylake Xeon vs. Zen EPYC technical marketing.
"AMD's technical marketing of the new CPU has been surprisingly absent, as the company not published any real server benchmarks. The only benchmarks published were SPEC CPU and Stream, with AMD preferring for its partners and third parties to promote performance"
This despite the fact that AMD and Intel's server products haven't been competitive like this in nearly a decade. Normally you'd expect there to be case studies flying out left and right, which has not been the case. And it's especially surprising since, as the underdog, AMD needs to claw back lost ground.
Consequently, Intel's own efforts are, to date, the first efforts by a server vendor to do a comprehensive set of benchmarks over a range of use cases. And let's be clear here: this is Intel doing this for Intel's own benefit. Which is why we've already previously reviewed the two CPUs, as have other 3rd party groups.
Still, I think it's very interesting to look at what Intel has chosen to represent, and what their numbers show. Intel has more resources than pretty much everyone else when it comes to competitive analysis, after all. So their choices and where they show themselves falling behind AMD says a lot about the current situation.
TL;DR: We thought this stuff was interesting, especially since neither vendor until now has done a Xeon-SP vs. EPYC comparison. And since we've already done our own independent review (https://www.anandtech.com/show/11544/intel-skylake... ), it gives us a set of data to compare to our own conclusions (and to be clear, this isn't a review nor are we trying to call it one)
CajunArson - Tuesday, November 28, 2017 - link
Yeah, you were so doing your righteous complaints when Anandtech did literally the same thing for AMD when AMD went out and misconfigured Intel boxes to pretend that Epyc was better than it actually was.Oh wait, you weren't.
ddriver - Tuesday, November 28, 2017 - link
The problem is the heavily biased towards intel AT coverage you clog. How could anyone complain about the opposite when AT have never displayed pro-amd bias? I have a problem with bias, and I point it out when I see it. You can bet your ass the moment AT shows unfair bais toward amd I will be there to point it out. But I cannot point it out if it doesn't exist.Hurr Durr - Tuesday, November 28, 2017 - link
He was too busy ordering specific platters for his thousands of HDDs with one hand and screaming in threads about hypetane with the other.lkuzmanov - Tuesday, November 28, 2017 - link
I've frequented the site for what must be over 10 years, but I fully agree this is, at the very least, a terrible idea.bmf614 - Tuesday, November 28, 2017 - link
Toms and many other sites also covered this.wumpus - Wednesday, November 29, 2017 - link
If Intel suddenly feels the need to compete with AMD, that 's news (practically "man bites dog" news judging from the last decade or so).The fact that they have to pick carefully contrived benchmarks to appear superior to AMD is even more telling. Totally ignoring power consumption (one of the biggest concerns for datacenters) is even more telling.