Massive ISP Upgrades

I’ve been hearing for some time now that 2022 flagships will have massive camera upgrades, and the new features of the next-gen SoCs being described by MediaTek and now also Qualcomm explain why that is.

The new ISP of the Snapdragon 8 Gen 1 falls under a new marketing name – “Snapdragon Sight”, and includes large improvements of the capabilities of the IP blocks within the image processing chain.

The big flagship feature being advertised is the fact that the new ISP is now capable of 18-bits of color depth per channel, up from the previous generation 14-bit ISP. While mobile image sensors nowadays still are barely 12-bit native in terms of their ADCs, the ushering of new HDR techniques such as staggered HDR capture, where exposures are immediately subsequent to each other on the sensor’s readout, means that new phones now are able to capture images a lot faster, recombining them into higher bit-depth results. Particularly here, the new 18-bit ISP pipeline now allows for three exposure HDR stacking off these new sensors.

The increased bit-depth should allow for an increase of 4 stops in dynamic range (or 2^4 = 16x the range), which greatly helps with very contrasting environments and challenging lighting situations. This is quite beyond any other camera solution right now, and being able to have this implemented in such a hardware fashion sort of blurs the line between traditional image capture techniques and the more software-defined computational photography methods of the last few years.

Indeed, the new ISP architecture seems to be very much a way to implement many of the existing computational photography techniques into fixed-function blocks: there’s a new neural-net controlled 3AA (auto-exposure, auto-focus, auto-white-balance) and face detection block, which sounds eerily similar to Google’s HDRnet implementations.

Night mode is also said to be vastly improved through a new multi-frame noise reduction and image stacking block, being able to now stack and align up to 30 images, and achieve also much finer detail this generation. Qualcomm here claims up to 5x better night mode shots.

Further improvements include a new distortion correction block that’s able to now also correct for chromatic aberrations, and a hardware video Bokeh engine, being able to operate at up to 4K video recording. Think of it as the same as the new Cinematic mode on the new A15 iPhones, but not only limited to 1080p.

Qualcomm notes that all the AI/ML/neural network features on the ISP are actually run and accelerated on the ISP itself, meaning that it is not offloaded onto the Hexagon dedicated ML processing blocks or the GPU.

Just as a note- Qualcomm’s 3.2Gigapixel/s throughput metric here seems low compared to the Dimensity 9000’s 9Gpixel/s, it’s possible that the companies are advertising very different metrics, with MediaTek advertising the throughput of lower-bit depth pixels coming from the image sensors per frame, while Qualcomm quoting the full bit depth pixel processing within the ISP itself.

In terms of video encoders and decoders, the new chip allows for 8K HDR recording now, but otherwise is seemingly on par with the Snapdragon 888 media blocks. Unfortunately, this also means no AV1 decoding this year yet again. Qualcomm isn’t part of the Alliance for Open Media consortium and instead is backing VVC/H.266 and EVC, however with AV1 being actively pushed by Google and YouTube, and seeing large adoptions such as by Netflix, it’s becoming questionable for Qualcomm to still not support the format in 2022 devices.

AI Performance - Iterative, but solid

Last year’s Hexagon IP block was a very large change for the Snapdragon 888. At the time, Qualcomm moved on from a more segregated DSP/AI architecture to a single more fused-together block being able to operate on scalar, vector, and tensor operations at the same time. This year’s iteration is an improvement of that larger change. Qualcomm notes that amongst many changes, they’ve doubled up on the shared memory of the block, allowing for greater performance for larger ML models (which are growing at a very fast pace).

Qualcomm didn’t note any TOPS figures this time around, instead stating we’re seeing 2x the tensor throughput performance, and smaller increases for scalar and vector processing. They do quote a day-1 to day-1 performance increase of 4x when compared to the Snapdragon 888, via a combination of both hardware and software improvements, but of course that figure is smaller when comparing both platforms on an equal software footing.

Power efficiency for AI workloads is said to be 70% better this generation, which is actually more significant, and should help with more demanding sustained ML workloads.

X65 Modem Integrated

In terms of connectivity, the Snapdragon 8 Gen 1 is quite straightforward, as it integrates the X65 modem IP that Qualcomm had already announced as a discrete model earlier this year.

The improvements here are the fact that it’s a 3GPP Release 16 compatible modem, including new features such as uplink carrier aggregation. Other improvements are 300MHz of Sub-6 bandwidth on 3 100MHz carriers, and an increase of the mmWave bandwidth from 800 MHz to 1000MHz, allowing a new peak theoretical downlink speeds of 10Gbps.

Introduction & Specifications - CPU & GPU Conclusion & First Impressions
Comments Locked

219 Comments

View All Comments

  • Kangal - Friday, December 3, 2021 - link

    The Cortex-X1 isn't really a performance core from the ground up, it's a "medium core" that's been tweaked/beefed up, so of course it will have disadvantages to something else, like Apple's Firestorm or Nuvia's cores.

    The Cortex-X2 design is mostly recycled from the Cortex-X1/Cortex-A78 (and Cortex-A710). And if you really look at it, it's all part of the same Cortex-A76 family. This is ARM's first attempt to get the ARMv9 protocol out there, that aspect alone takes resources away from development. Their performance and efficiency projections for next year have been a let down. That's the context you need to remember.

    The Cortex-X3 is supposed to be designed by ARM's European team, and they have a stronger track record than their US Team. So I expect good things. On top of that, they're supposed to start on a new platform; eg Cortex-A730. I believe the difference will be a kin to the Cortex-A57 versus Cortex-A72. So it will be worth waiting for.

    With that said, Apple's A15 isn't too far from their A14 and A13 processors. Whilst the X1 and X2 failed to catch up to the Apple A13... I do think the X3 will catch up and potentially surpass it. With the point being that Apple has the new lead with the Apple A16.

    The Nuvia cores have been in development for some time. Obviously it's built with ARMv8 in mind. And they do have silicon pressed and sampled already, ie Working Prototypes. And based on the information we know, their Nuvia Cores are better than the Apple A15 in the labs. But now they will have to go back, tweak it, and covert to ARMv9. Again that will take time of the development, and potentially less optimised design in the interim.

    Software Support is the part which Apple wins. It's all-round better. Not only can they support it longer, but businesses and developers trust that ecosystem, which means it will get priority for development and for optimisation. The other point is that Apple's software is very advanced: Swift and Metal are actually pretty awesome, and their SDK is the Gold Standard in the industry. Sure you have certain limitations, but within the boundary the software and hardware meld much closer. It's the result of throwing billions of dollars, thousands of in-house developers, and years of experience.

    A Qualcomm-Nuvia product has no answer for that. If they try AndroidOS, they will have lots of compromises. Whilst Microsoft is usually better, lately they've done a poor job with W-ART, Windows-on-ARM, and Windows 11. So I don't expect a Q-Nuvia laptop doing as well, so they will lose some performance and efficiency there again to the likes of Apple's iPad/Macbook.
  • tkSteveFOX - Wednesday, December 1, 2021 - link

    It's clear ARM designs have hit a brick wall. The improvements in architecture just aren't enough when compared to Apple's. While Android phones have more mature ISP and modems, Job's lot silicon is at least a generation if not two ahead in performance and efficiency.
    Android phones now regularly costing more or as much as iphones means you as a customer are more likely to choose Apple in the longer run.
  • Wilco1 - Wednesday, December 1, 2021 - link

    A 20% yearly improvement is hardly a brick wall. While the L2/L3 caches are still on the small side (especially when compared to Dimensity 9000), this should narrow the gap. You're not going to notice the difference though. Most phones use far older and slower cores (Samsung just announced a phone with 1.2/1.6GHz Cortex-A55!!!), and there a 10-20% difference will be very noticeable. But at the high-end? Absolutely not.

    As for efficiency, my S21 Ultra has the same battery life as iPhone 13 Pro according to AnandTech. And I paid far less for it. As a consumer that matters to me more than whether or not the iPhone is faster on SPEC or Geekbench.
  • high3r - Wednesday, December 1, 2021 - link

    Wow, 3095 mAh vs 5000 mAh. That's something then. :)
  • Wilco1 - Wednesday, December 1, 2021 - link

    Design capacity of S21 Ultra is 4855 mAh, estimated capacity around 4700 mAh according to Accubattery. It lasts 4 to 5 days on a full charge which is exceptionally good.

    iPhones use a huge SoC with lots of cache on the most advanced process which allows for a smaller battery, while Android phones use a much smaller SoC on a less advanced process and a larger battery instead.
  • TheinsanegamerN - Thursday, December 2, 2021 - link

    It will be something in 2 years when the samsung can still go for more then 4 hours without keeling over. The iphone, OTOH......
  • mode_13h - Wednesday, December 1, 2021 - link

    > While the L2/L3 caches are still on the small side (especially when compared to Dimensity 9000)

    Isn't the 9000 aimed primarily or exclusively at laptops? Why compare it to a phone SoC, then? Phones have smaller batteries and worse cooling.
  • Wilco1 - Wednesday, December 1, 2021 - link

    Eh what?!? This clearly states smartphone repeatedly - in huge letters, so impossible to miss... https://i.mediatek.com/summit-dimensity9000

    For laptops there is Cortex-A78C and Cortex-X1C.
  • mode_13h - Wednesday, December 1, 2021 - link

    Okay, my bad. I thought I remembered something about it being a Chromebook SoC.
  • name99 - Monday, December 6, 2021 - link

    You'd think so ("you won't notice the difference") but you do.

    I'm happy with my iPhone XS (A12) but when I used a friend's iPhone 13 Pro (A15) the extra speed was noticeable. I couldn't say WHERE the difference lies; my phone never stutters or glitches in animation and apps launch basically instantly. But even so you can feel that the iPhone13 Pro is faster, and not just subtly so.
    Maybe it's in the 120Hz? But even so, you need a SoC that can keep that 120Hz fed while never glitching...

Log in

Don't have an account? Sign up now