The Snapdragon 8 Gen 1 Performance Preview: Sizing Up Cortex-X2
by Dr. Ian Cutress on December 14, 2021 8:00 AM ESTMachine Learning: MLPerf and AI Benchmark 4
Even as a new benchmark in the space, MLPerf has been made available that runs representative workloads on devices and takes advantage of both common ML frameworks such as NNAPI as well as the respective chip libraries for each vendor. Using this benchmark on retail phones to date, Qualcomm has had the lead in almost all the tests, but given that the company is promoting a 4x increase in AI performance, it will be interesting to see if that comes across all of MLPerf’s testing scenarios.
It should be noted that Apple’s CoreML is currently not supported, hence the lack of Apple numbers here.
Across the board in these first four tests Qualcomm is making a sizable lead, going above and beyond what the S888 can do. Here we’re seeing up to a 2.2x result, making an average +75% gain. It’s not quite the 4x that Qualcomm promoted in its materials, but there’s a sizable gap with the other high-end silicon we’ve tested to date.
The only non-lead is with the language processing, where Google’s Tensor SoC is almost 2x what the S8g1 scores. This test is based on a mobileBERT model, and either for software or architecture reasons, it fits a lot better into the Google chip than any other. As smartphones increase their ML capabilities, we might see some vendors optimizing for specific workloads over others, like Google has, or offering different accelerator blocks for different models. The ML space is also fast paced, so perhaps optimizing for one type of model might not be a great strategy long-term. We will see.
In AI Benchmark 4, running in pure NNAPI mode, the Qualcomm S8g1 takes a comfortable lead. Andrei noted in previous reviews with this test that the power consumed during this test can be quite high, up to 14 W, and this is where some chips might be able to pull ahead an efficiency advantage. Unfortunately we didn’t record power at the same time as the test, but it would be good to monitor this in the future.
Facebook
Twitter
RSS
169 Comments
View All Comments
GeoffreyA - Wednesday, December 15, 2021 - link
Since about Kit Kat or so, the bytecode of an Android app is compiled to native code at install time. But doubtless C++ would still be faster because Java overdoes object-oriented style and dynamic allocation. C/C++ code can work in Android using the NDK, but it appears to be a nightmare getting it latched in. Sometimes I wonder what they were thinking when they picked Java as their language.Meteor2 - Friday, December 17, 2021 - link
They were thinking about the number of developers who could code it...TheinsanegamerN - Friday, December 17, 2021 - link
Developer support, ease of transition, and flexibility.Chaser - Tuesday, December 14, 2021 - link
Before, I used to care about these SOC improvements/benchmarks, etc. But I don't play games on smartphones. All I care about today is battery life improvements, not performance.Alistair - Tuesday, December 14, 2021 - link
That's why I switched to Apple. I get battery life and efficiency improvements, and CPU improvements. The only highlight for Snapdragon is GPU. CPU improvements are almost non-existent this year, again. Same for several years, they get farther and farther behind Apple in CPU.flyingpants265 - Thursday, December 16, 2021 - link
Yeah, but everything else about it sucks.Alistair - Saturday, December 18, 2021 - link
just browsing the web and using apps that are the same on all platforms, and doing it fasterKangal - Sunday, December 19, 2021 - link
Apple has a long history, that whenever their next chipset offers more efficiency, they counteract this by using a new (power hungry) feature.For instance, newer 5G modem, 120Hz screen, higher resolution, or more processing demand in their next software.
And at times, they even reduce the battery capacity. This saves weight off the phone, and they counteract this by adding other components like more cameras, or heavier glass construction.
All in all, the iPhones don't gain more battery life, they follow a general trend, and there are some outliers there (eg iPhone 6S+ and iPhone 13 Max).
TheinsanegamerN - Friday, December 17, 2021 - link
Budget phones like the moto g power annialate apple in real life battery benchmarks for a fraction of the price.Alistair - Tuesday, December 14, 2021 - link
I'll all in on Mediatek then, this was disappointing. Can't wait for Mediatek comparisons!