Qualcomm Snapdragon Tech Summit Live Blog Day Two: All About Mobile

01:24PM EST - After yesterday announcing both the Snapdragon 765 and Snapdragon 865 mobile platforms, which will aim to provide a lot of 5G coverage to 2020 devices, Qualcomm today is going to go deep into what is inside both of these chips. We're here will full live blog coverage - the talk starts at 7pm UTC / 2pm ET.

02:02PM EST - Alex Katouzian to the stage

02:03PM EST - 560k views on the live stream yesterday, more than all of last year combined

02:03PM EST - Yesterday, made another 5G standalone data call with Ericsson

02:03PM EST - Demo areas are open this afternoon with S865 and S765

02:04PM EST - Keith Kressen to the stage, SVP Product Management

02:05PM EST - First Snapdragon summit at this venue was 2017. AI won at Go, Bitcoin was $17k

02:07PM EST - Why 2017? Q4 2017 was when Qualcomm finalized the definition of the S865

02:08PM EST - 865 was prepared for the 2019 world back in 2017

02:08PM EST - 144 Hz displays, 200 MP cameras, ultrasonic fingerprint

02:09PM EST - Christopher Patrick to the stage, SVP Engineering

02:10PM EST - Back in 2013, Snapdragon 800 was launched

02:10PM EST - Lot of industry firsts

02:10PM EST - first ever 4K capture/playback

02:10PM EST - first ever LTE with CA

02:12PM EST - 1st year of development is IP definition

02:13PM EST - Algorithm in hardware definition

02:13PM EST - ML techniques that require DSP adjustments all done in that first year

02:14PM EST - also fundamental things like transistor definition and packaging technology

02:15PM EST - After the SoC, have to consider the device and the other components. Modem, power amplifiers, RF, audio, etc

02:15PM EST - e.g. 8K, need to make sure power and energy at the right parts of the system to also have minimum consumption on the battery

02:16PM EST - also thermal management

02:16PM EST - Packaging, security

02:16PM EST - Then onto handset and ecosystem

02:17PM EST - On Year 3 of development, finalizing the design in hardware, software, and optimization

02:18PM EST - (This means that Qualcomm is likely working on 885 or even 895 today, ready for 2021/2022)

02:19PM EST - 10k engineers to bring S865 to life

02:19PM EST - Keith Kressin back to the stage

02:20PM EST - First up, the X55 modem

02:20PM EST - the modem, the RF and the front end was all developed as a single system

02:20PM EST - 2nd gen 5G modem to antenna solution

02:20PM EST - It's complex, so you need to define it as a system

02:20PM EST - Sub 6 GHz and mmWave support

02:21PM EST - Support for Dynamic Spectrum Sharing

02:21PM EST - Support Carrier Aggregation

02:21PM EST - Premium devices have to be future proofed

02:22PM EST - Wi-Fi 6 through FastConnect 6800

02:22PM EST - 1.8 Gbps with simultaneous 2.4G and 5G band operation

02:22PM EST - First to support 8-way uplink MU-MIMO

02:22PM EST - BT5.1 and Aptex

02:23PM EST - Support two wireless stereo plus for airbuds

02:23PM EST - can connect to two sets of airbuds independently with 2x range

02:23PM EST - S865 built on TSMC 7NP

02:23PM EST - 765 and 765G built on Samsung 7nm EUV

02:24PM EST - Spectra 480 CV-ISP

02:24PM EST - 2 gigapixel/sec

02:24PM EST - redesigned for higher resolution capture and lower power

02:24PM EST - 4K video 64MP pictures at 30 fps

02:24PM EST - 4x pixels per clock

02:24PM EST - EVA new video engine

02:25PM EST - redesigned the pipeline for lower power, not simply upping the freq

02:25PM EST - library for object tracking, 25% lower latency and 50% lower power than S855

02:25PM EST - HEIF support

02:25PM EST - Adreno 650

02:26PM EST - Most efficient graphics in the world

02:26PM EST - Goal to be the best sustained in performance

02:26PM EST - Not to be #1 in peak for benchmarks - but for the best performance after 30 minutes of gaming

02:26PM EST - Performance in Alaska has to be the same in Maui

02:26PM EST - 25% faster rendering, 35% more power efficient

02:27PM EST - Optimized the design point of GPU for 90 fps

02:27PM EST - 35% better power eff at 90 fps

02:27PM EST - Display processing engine has local tone mapping for gaming

02:27PM EST - Also software for graphics

02:27PM EST - CPU time

02:28PM EST - 1x A77 at 2.84 G

02:28PM EST - 3 x A77 at 2.4 GHz

02:28PM EST - 4 x A55 at 1.8 GHz

02:28PM EST - local L2 caches, all shared L3

02:28PM EST - S855 had 2MB L3, this one has 4MB of L3

02:29PM EST - Hexagon to support new tensor accelerator, 4x more perf

02:29PM EST - Also 35% more power efficient

02:29PM EST - 4 threads of scalar processing

02:30PM EST - 15 TOPs when using CPU+GPU+DSP

02:30PM EST - LPDDR5-5500 support (2750 MHz)

02:30PM EST - 3MB System cache

02:30PM EST - Support for LP4 or LP5, up to the OEM based on commodity pricing

02:31PM EST - Sensing hub

02:31PM EST - sub 1mW camera wake up, sub 1mA, scalable framework

02:32PM EST - Offering a separate chip for that sub 1mW camera - if OEMs want an always on camera

02:32PM EST - hands up who wants an always on camera on their phone

02:32PM EST - Smart APIs

02:33PM EST - Multiple wake up words

02:33PM EST - Security

02:34PM EST - Jesse Seed, Senior Director of Security to the stage

02:34PM EST - If you don't think about your data, Qualcomm does

02:35PM EST - Protected pipelines for secure data

02:35PM EST - Cannot copy data out of the secure region

02:35PM EST - even when malware is installed

02:36PM EST - Hardware supported Attestation for banks

02:36PM EST - There's no standard for something like face authentication

02:37PM EST - 'Moving at the speed of snapdragon'

02:37PM EST - isolated secure processing unit for assured processing environments

02:37PM EST - was first on S855

02:38PM EST - used to be only provided by external chips

02:38PM EST - S865 will be first dual SIM dual standby SoC, to replace dual SIMs or dual eSIMs

02:38PM EST - Work with Google on Android Security

02:38PM EST - el goog to the stage

02:39PM EST - research and security. Qualcomm was the first to support the new secure credential API in Android

02:39PM EST - Control flow integrity

02:40PM EST - Enabling hardware security through software as well

02:40PM EST - Security needs to be seemless and unintrusive

02:41PM EST - Qualcomm is suggesting that you can have core ID on your phone. Drivers ID etc

02:41PM EST - (this will not end well)

02:41PM EST - 'Security is glamorous'

02:42PM EST - More on AI, QC VP AI to the stage

02:43PM EST - AI use cases

02:43PM EST - Add more and more intelligence to devices

02:43PM EST - Understand their surroundings

02:43PM EST - More than 1 billion devices enabled that use QC AI tech

02:44PM EST - Different AI use cases - music detection

02:44PM EST - doing it with on-device AI rather than doing it in the cloud

02:44PM EST - always on neural nets at ultra-low power

02:44PM EST - on-device speech to text

02:45PM EST - bypassing the cloud

02:45PM EST - inference happens on the device instead. better experience, lower latency

02:45PM EST - (I thought the low latency of 5G would mean that wouldn't matter? Isn't it more power efficient in the cloud?)

02:46PM EST - 'AI is happening as we speak'

02:46PM EST - Transcription and translation in real time on the device

02:47PM EST - Already running on S865 on stage

02:47PM EST - this is the beast mode for AI

02:47PM EST - Three tools: software, hardware, and dev tools

02:48PM EST - AI is already being used on almost every segment of the mobile platform (all the blue segments)

02:48PM EST - e.g. AI algorithms in the modem to help improve antenna perf

02:48PM EST - 5gh AI Engine

02:48PM EST - Adreno 650 does 2x TOPs, supports FP16 and FP32

02:49PM EST - Hexagon 698 uses deep learning bandwidth compression

02:49PM EST - 50% lossless compression

02:49PM EST - Frees up bandwidth, saves power

02:49PM EST - LPDDR5 has 30% more bandwidth

02:50PM EST - Everything is a tensor

02:50PM EST - Different parts work best are scalar/vector/tensor

02:50PM EST - Others find this difficult to manage - seemless on QC

02:51PM EST - S845 was 3 TOPs, S855 was 7 TOPs, S865 is 15 TOPs

02:51PM EST - Note that's CPU + GPU + DSP. You aren't going to use all three at once, ever.

02:52PM EST - peak perf and peak perf/watt

02:53PM EST - Now sensing hub

02:53PM EST - At low power sub 1mA, device can tell if the audio is the right person

02:53PM EST - Device knows when you are walking, or in the car, all at low power

02:54PM EST - Now AI software

02:55PM EST - It's already in your device, and 865 makes it better

02:55PM EST - Software stack and 25+ partners

02:55PM EST - 25 partners and apps is at the top of the stack

02:56PM EST - partners in the audience

02:56PM EST - Framework support is pytorch, tensorflow, onnx

02:56PM EST - Onnx is the interchange format for other frameworks not listed

02:57PM EST - Qualcomm wants the least amount of friction to enable AI on QC hardware

02:57PM EST - Qualcomm Neural Processing SDK

02:57PM EST - (note I believe you still have to be a preferred partner to get access to the frameworks that do API calls to the DSP)

02:58PM EST - NNAPI support, obviously. It's just a shame that device OEMs can't use the latest drivers to enable it

02:58PM EST - Power, perf, access, features

02:58PM EST - best in class solution

02:59PM EST - QC AI research into things like data-free quantization

02:59PM EST - Accelerate all the key use cases

02:59PM EST - 3-5x speed ups on Hexagon

03:00PM EST - Moving automatic speech recognition with Google, moving it from CPU to hexagon on S865, gives 3x power savings

03:01PM EST - 160+ operator support for AI

03:01PM EST - operator = math functions in AI, not wireless operator

03:01PM EST - User defined operators now on the platform

03:02PM EST - The power of snapdragon powered by you

03:02PM EST - Tensorflow lite

03:03PM EST - Hexagon NN Direct for Tensflow Lite and more

03:04PM EST - Snap Inc to the stage

03:05PM EST - Creative tools on Snapchat using AI

03:06PM EST - only 23 ms per frame - smooth 30 FPS on S865

03:06PM EST - Using hexagon direct

03:07PM EST - back to Qualcomm

03:08PM EST - Qualcomm AI Model Efficiency Tookit - 3x compression with 1% lower accuracy

03:09PM EST - FP32 to INT8

03:13PM EST - Intelligence at all parts of the network: 5G and AI

03:13PM EST - Multiple 320p streams to a single device, upscaled to 720p on the device, saves data on the device

03:14PM EST - tiktok super resolution

03:15PM EST - Loom.ai to the stage. 3D avatars

03:17PM EST - face tracking, avatar application, and background adjustment

03:17PM EST - For meetings etc

03:17PM EST - uses Hexagon

03:17PM EST - avatar to avatar calls at lower data rate

03:18PM EST - QC back to the stage

03:18PM EST - loom.ai uses four different CNNs on the hexagon

03:18PM EST - Also Gaming, XR, Contextual awareness

03:19PM EST - textures in a game adjust based on facial detection of player mood

03:19PM EST - for example

03:20PM EST - You need to offer a complete AI solution: hardware, software, and dev tools

03:20PM EST - Now camera

03:21PM EST - Smartphones enable more camera features than a 'single camera'

03:21PM EST - Computational photography

03:23PM EST - 5 cameras per person in a household, due to smartphone cameras

03:23PM EST - Turning the world into photographers

03:24PM EST - Qualcomm powers the most smartphone cameras on the planet

03:25PM EST - Step forward on Spectra 480 ISP. Biggest update ever

03:26PM EST - 2 gigapixels per second

03:26PM EST - 64MP at 30 fps

03:27PM EST - pixels matter more than raw GB/s

03:27PM EST - 4x pixel processing

03:28PM EST - 4 pixels per clock. This is QC matching the competition

03:29PM EST - 3 pieces to the ISP

03:29PM EST - Capture, video analytics, and HEIF

03:30PM EST - With the margin and the ISP, can use 9x more autofocus points

03:30PM EST - Quad CFA image sensors coming

03:31PM EST - looks like some form of auto pixel fusion but not

03:31PM EST - S865 supports Quad CFA

03:31PM EST - Dedicated cores for noise reduction in the ISP

03:32PM EST - Also local contrast enhancement

03:32PM EST - 40% more pixel processing for noise reduction during video capture

03:33PM EST - S865 enables more simultaneous captures ever

03:33PM EST - Spectra 480 enables full 64MP picture capture during 4K video recording

03:33PM EST - assuming you have the sensors

03:34PM EST - 200MP camera capture support

03:34PM EST - QC has partnered with sensor makers, so 200MP handsets in 2020

03:35PM EST - 16% lower power video capture

03:35PM EST - at 4K60

03:35PM EST - '18% more texture in your photos' - er what? how do you measure that?

03:35PM EST - HEIF

03:36PM EST - Depth map can be stored in the HEIF container

03:36PM EST - Enables post processing

03:37PM EST - (again, it requires the OEM to support capturing the data)

03:38PM EST - Spectra + AI

03:40PM EST - demo time

03:40PM EST - Semantic segmentation with AI

03:41PM EST - Applying algorithms with different strength integrities

03:42PM EST - Using AI to recoup details on normal filters

03:42PM EST - Applying the right algorithm at the right place

03:43PM EST - accelerated by S865

03:44PM EST - Modern cameras can't smooth zoom between cameras - they don't reproduce the same results because of discontinuities

03:45PM EST - *modern smartphones

03:45PM EST - This is a use for AI

03:46PM EST - 8K30 on S865

03:46PM EST - recording

03:46PM EST - 33MP per frame, * 30 fps, = 0.99 gigapixel/sec

03:46PM EST - (if it's 2 gigapixel, why not 8K60 ??)

03:47PM EST - 4K HDR10+ support with S865 with Dolby support

03:48PM EST - Dolby to the stage

03:50PM EST - millions of Dolby devices, over 2500 movies and TV shows

03:52PM EST - QC back to the stage

03:53PM EST - 4K120 supported on S865

03:53PM EST - 120fps playback too with devices that support higher refresh rates

03:53PM EST - Slow mo in 4K

03:53PM EST - 4x 4K slowmo

03:54PM EST - slomo with no limits

03:54PM EST - native capturing at 720p960

03:54PM EST - no limits means no limit on video recording

03:55PM EST - so a 10 minute video recording at 960 fps

03:56PM EST - or a 10 hour video

03:58PM EST - Leilani DeLeon to the stage to talk gaming and XR

03:59PM EST - Snapdragon Elite Gaming

03:59PM EST - Beast Mode again

03:59PM EST - 'gaming is 43% of all smartphone use, 74% of all app store spend

04:00PM EST - Gaming in XR

04:00PM EST - We are now in a mobile first generation

04:01PM EST - perhaps even a 'mobile only' generation

04:01PM EST - 518m eSports viewers in 2020 expected

04:01PM EST - (but mobile gaming eSports? that's super low)

04:04PM EST - some eSports person to the stage

04:06PM EST - 'I want 144 Hz on my phone for fortnite'

04:07PM EST - 80% of gamers in China want specialized gaming phones

04:08PM EST - because winning matters

04:09PM EST - Todd LeMoire to the stage, Principal Engineer, Gaming

04:10PM EST - Fortnite - seemless experience optimized for Qualcomm Elite Gaming

04:10PM EST - S865 pushes beyond the limits

04:11PM EST - Game Smoother

04:11PM EST - Jank Reducer at any refresh rate

04:11PM EST - Adaptive Game Performance engine - working at the sub-frame level

04:12PM EST - 90 FPS Gaming Mode coming to PUBG

04:12PM EST - 120 FPS with some tencent games

04:12PM EST - 1st 144 Hz diplays with S865

04:13PM EST - 10-bit HDR gaming for Android

04:14PM EST - Game Color Plus - enhancing quality on Adreno

04:14PM EST - No changes to the game code are necessary

04:14PM EST - Works on Lineage II Revolution

04:15PM EST - Doesn't add latency

04:15PM EST - Boosts color saturation and contrast. More vivid scenes

04:15PM EST - Adreno HDR Fast Blend

04:17PM EST - Upgradeable mobile drivers !

04:17PM EST - This is important

04:18PM EST - Driver updates through the Play Store

04:18PM EST - Phone agnostic

04:18PM EST - Desktop Forward Rendering

04:19PM EST - Depth of Field through Desktop Forward Rendering

04:20PM EST - Multiple Dynamic Lights

04:20PM EST - Dynamic Shadows

04:22PM EST - One of the key aspects of desktop drivers is day 0 updates for new launches. Qualcomm hasn't said anything about doing this for mobile. Yet

04:23PM EST - S865 summary time

04:25PM EST - All android gaming smartphones today are based on Snapdragon. 2020 will be the same

04:26PM EST - That's a wrap for today. The demo room opens in a couple of hours, will report what's there :)