The Snapdragon 8 Gen 1 Performance Preview: Sizing Up Cortex-X2
by Dr. Ian Cutress on December 14, 2021 8:00 AM ESTMachine Learning: MLPerf and AI Benchmark 4
Even as a new benchmark in the space, MLPerf has been made available that runs representative workloads on devices and takes advantage of both common ML frameworks such as NNAPI as well as the respective chip libraries for each vendor. Using this benchmark on retail phones to date, Qualcomm has had the lead in almost all the tests, but given that the company is promoting a 4x increase in AI performance, it will be interesting to see if that comes across all of MLPerf’s testing scenarios.
It should be noted that Apple’s CoreML is currently not supported, hence the lack of Apple numbers here.
Across the board in these first four tests Qualcomm is making a sizable lead, going above and beyond what the S888 can do. Here we’re seeing up to a 2.2x result, making an average +75% gain. It’s not quite the 4x that Qualcomm promoted in its materials, but there’s a sizable gap with the other high-end silicon we’ve tested to date.
The only non-lead is with the language processing, where Google’s Tensor SoC is almost 2x what the S8g1 scores. This test is based on a mobileBERT model, and either for software or architecture reasons, it fits a lot better into the Google chip than any other. As smartphones increase their ML capabilities, we might see some vendors optimizing for specific workloads over others, like Google has, or offering different accelerator blocks for different models. The ML space is also fast paced, so perhaps optimizing for one type of model might not be a great strategy long-term. We will see.
In AI Benchmark 4, running in pure NNAPI mode, the Qualcomm S8g1 takes a comfortable lead. Andrei noted in previous reviews with this test that the power consumed during this test can be quite high, up to 14 W, and this is where some chips might be able to pull ahead an efficiency advantage. Unfortunately we didn’t record power at the same time as the test, but it would be good to monitor this in the future.
Facebook
Twitter
RSS
169 Comments
View All Comments
eastcoast_pete - Wednesday, December 15, 2021 - link
As of now, and assuming Mediatek doesn't screw up the firmware, the Dimensity 9000 would best QC's "flagship" mobile SoC for 2022. Now, I remember reading that no major smartphone maker plans to sell phones with the Dimensity 9000 in the US, at least not officially. Is that really so, and if yes, does anyone here know why? I gladly do without the mm wave 5G channels (the 9000 doesn't cover those), which are really only available on Verizon, and even then only in some places.TheinsanegamerN - Friday, December 17, 2021 - link
Dimensity has no support for Verizon, which means you lose about 40% of the US market from the get go.Wardrive86 - Wednesday, December 15, 2021 - link
Excellent article Ian! Those geekbench scores are extremely close to my tigerlake i5-1135g7 in fullsize notebook. Extremely impressive, look forward to sustained performance testing and comparisons to mediatekTheinsanegamerN - Friday, December 17, 2021 - link
How many times does it need to be pointed out that geekbench is worthless for comparisons across systems? Unless you are doing ARM android VS ARM android or the like, the scores are not representative of real world performance.Comparing ARM android to x86 windows is like comparing an apple to a medieval sword. Totally different use cases.
Wilco1 - Saturday, December 18, 2021 - link
How many times does it need to be pointed out the scores are directly comparable? It compilers the same benchmark source code using LLVM whenever possible and runs the same datasets. There are differences between OSes of course, but that only affects scores in a minor way.And it's not like Geekbench scores show a completely different picture from SPEC results - modern phones are really as fast as laptops.
Wardrive86 - Sunday, December 19, 2021 - link
Agreed Wilco, it may not stress the memory subsystems to the extent that SPEC does, but it seems to paint roughly the same picture.IUU - Monday, February 7, 2022 - link
"modern phones are really as fast as laptops."No they are not. You will inever beat the physical law. The only reason some Android or iOS devices have reached to the point of being comparable with laptops or desktops is because they have managed for the first time in history to grab some superior lithographies first
If that was not the case , laptops and desktops would be x3 to x15 faster than today's mobile devices. But if laptops nad desktops sold at 700 to 2000 dollars then how would smartphones be sold at today's prices ? The comparison would be abysmal.
So the industry found a solution. Keep desktops and laptops at ancient lithographies while grab for android and iOS the best of the best.
Then con
IUU - Monday, February 7, 2022 - link
then convince the sheeple that they somehow live in a revolution of better and most economical devices while keeping them at the same level of computing capacity for the best part of the decade. Not only that , but also get tons of dollars for selling a dream....IUU - Monday, February 7, 2022 - link
I accept both geekbench AND antutu for making a quick comparison between mobile devices(and yes I know the qualms about Antutu but I think they are hysterical. And geekbench for making quick comparisons between platforms, having in the back of my mind their limitations, always.Before the mobile devolution came we had a very clear picture of the computational capacity of our chips. Then the radicals came being secretive about their chips. So , though we still have a gross impression of the theoretical performance of desktop and laptop chips we are in the dark concerning the "mobile" . As far as I am concerned I take their claims with a huge grain of salt.
Good guys don't hide....
ChrisGX - Wednesday, December 15, 2021 - link
On these numbers the Snapdragon 8 Gen 1 doesn't appear to be living up the 4x AI perf claim. Is there more performance to come? We will know soon enough but even without any further improvement in the AI scores the SD8 Gen 1 still looks pretty good.With the SD8 Gen 1 and the MediaTek Dimensity 9000 posting good AI scores (unconfirmed in the case of the D9000) we can expect Google and even Apple to start raising the stakes on the AI performance of future silicon. Qualcomm, too, is going to have to lift its game with MediaTek looking very competitive in so many areas.