Qualcomm Details The Snapdragon 888: 3rd Gen 5G & Cortex-X1 on 5nm
by Andrei Frumusanu on December 2, 2020 10:00 AM EST- Posted in
- Mobile
- Qualcomm
- Smartphones
- SoCs
- 5G
- Cortex A78
- Cortex X1
- Snapdragon 888
Triple ISPs: Concurrent Triple-Camera Usage
Interestingly enough, during yesterday’s keynote event, Qualcomm described themselves as a camera company, which is a funny way to see things, but actually somewhat makes sense given the large leaps in smartphone camera capture capabilities over the recent years.
The new Snapdragon 888 pushes the envelope in terms camera abilities by adding a whole new independent third ISP to the SoC, allowing the SoC to now run three independent camera modules concurrently, opening up new use cases for vendors and camera applications.
The new triple-ISP architecture now increases the overall pixel processing throughput by 35% to 2.7Gigapixels/s, allowing for concurrent usage of up to three 28MP sensors with zero shutter lag captures. Alternatively, you can use a combination of 64+25MP sensors with ZSL, or a single 84MP sensor with ZSL. There’s still also support for ultra-high-resolution sensors up to 200MP, but image captures here don’t support ZSL.
Allowing concurrent captures of three sensors now allows for the holy trifecta of ultra-wide-angle, wide-angle and telephoto modules to capture a scene at the same time, allowing for more interesting use-cases such as image stitching and image fusion to happen to seamlessly.
One interesting capability that Qualcomm was advertising is triple-stream 4K HDR video recording. That’s a bit of an odd-ball use-case as I do wonder about the practical benefits, but I do at least hope that the new triple ISP system allows for more seamless switching and zooming in and out between the various camera modules during video recording.
Video recording capabilities this year don’t seem to have changed, compared to the Snapdragon 865. This means 4K120 or 8K30 are still the peak capture modes, supporting also slow-motion of 720p960. Formats are also unchanged, with HEVC encoding in HDR formats such as HDR10+ or Dolby Vision being supported.
AV1 decoding didn’t make the cut this year unfortunately, which means wide-spread adoption in mobile for the codec will be delayed for another year.
While the video encoding formats haven’t changed, the image processing capabilities for HDR capture has. Thanks to the new ISPs and the raw performance throughput, the new Snapdragon 888 will be able to capture 4K HDR footage with the more advanced computational HDR processing being applied on each and every frame of the video.
Qualcomm claims that the new ISP in the Snapdragon 888 is the first to support new next-generation staggered HDR sensors.
Source: OmniVision
These are sensors that can have multiple rolling shutters, meaning sensor line readouts, active at the same time on the sensor. Instead of taking multiple exposures one at a time sequentially by scanning out the sensor matrix from start to finish, the sensor will start another exposure immediately after the completed line read-out, reducing the time in-between exposures greatly. This should allow for significant less motion ghosting between the exposures and a sharper resulting recombined HDR image capture than current generation sensors which only have a single active line readout on the sensor.
While we haven’t actually heard of such sensors from Samsung or Sony yet, Qualcomm is adamant that we’ll be seeing smartphones in 2021 employing this new technology.
A further improvement for still-picture captures is the advancement of the new multi-frame noise reduction engines inside of the ISPs. It’s said that the quality of the noise reduction has been improved this generation, allowing for even better low-light captures with the native capture mode (no computational photography).
Facebook
Twitter
RSS
123 Comments
View All Comments
jaj18 - Thursday, December 3, 2020 - link
It will come with adreno 7××🤔StormyParis - Wednesday, December 2, 2020 - link
"This year although we’re not reporting from Hawaii". Heh heh. I'd feel sorry for you if I wasn't jealous for all the other years ? ;-pKrysto - Wednesday, December 2, 2020 - link
No AV1 decode support in 2021? Really?tuxRoller - Thursday, December 3, 2020 - link
I'm more interested in accelerated encode at this point.We've not had industry wide buy-in of a new lossy codec since jpeg, and hevc haven't quite achieved the ubiquity that h.264 managed after the same time in market.
GeoffreyA - Thursday, December 3, 2020 - link
While hardware AV1 encode would be quite nice to see, there's a possibility it will lose much of software AV1's gains over software HEVC (that is, one might encode quickly but end up with less compression than x265). Also, leaving aside the Slough of Patents for a moment, VVC will have to be taken into account once x266 comes out. If the studies are right, the reference VVC encoder (not x266) already shows better compression and speed than AV1. Hopefully, it won't inherit HEVC's less than pleasing picture too (to my eyes at least).tuxRoller - Friday, December 4, 2020 - link
That's a great point. In my haste to mention the lack of encoding ability I'd forgotten about the actual implementation of such a complicated codec. Which of the 30 or so tools, and their combinations, provide the most bit savings per mm²?Iirc, vvc owes a lot of its gains via integration with ml (there's at least one commercial av1 implementation that does this as well to, supposedly, great effect). IOW, I'm uncertain how much easier vvc will be too implement in hardware. Otoh, EVC looks quite interesting.
GeoffreyA - Friday, December 4, 2020 - link
Oh, yes, it will probably make their heads spin implementing this thing in hardware, and when they do, which they will, they're going to make it a marketing point (even if, in practice, it fell behind x265).Yesterday I was experimenting with libaom-av1 on FFmpeg and discovered a useful parameter: -cpu-used. Controls compression/encoding speed and takes values between 0 and 8. 0 being the slowest, 1 the default, and 8 fastest. To my surprise, 8 brought encoding speed to reasonable levels: about 10x slower than x265, if I remember right, which isn't half bad. I was using a video shrunk to 360p though.
As for VVC, can't wait to give it a go. Hopefully, it'll deliver and be of AVC's calibre. I wasn't familiar with EVC but took a look at it now, and it does appear to be quite an interesting concept.
tuxRoller - Saturday, December 5, 2020 - link
You might be interested in the doom9 forums (https://forum.doom9.org/forumdisplay.php?f=17). In the av1 thread you'll often see people posting updates about the various av1 en/decode implementations, new settings and, in general, some interesting thoughts from folks in the industry.BTW, starting from this post (https://forum.doom9.org/showthread.php?p=1929560#p... there's an interesting discussion regarding qcom & their interest in not pushing av1.
Regarding fast encoders, I'm assuming you've tried svt-av1? That's supposed to have nearly caught up with aom's encoder quality but is still a good deal faster.
Lastly, thanks for the paper. It looks interesting and a quick skim didn't reveal any mention of ml enhanced transform, or even a new entropy code(!); they seem to be continuing to iterate on h.264->h.265. However, only started reading it and realized I'm not getting through that tonight:)
GeoffreyA - Saturday, December 5, 2020 - link
Thanks for those doom9 threads. Looks like a treasure trove of information on AV1 there. As for SVT-AV1, yes, I have tried it. While the speed was good, the picture didn't seem that impressive. Anyhow, I'll have a crack at it again and see how it stacks up against libaom, now that I've got the latter running faster.You're right. I remember getting the impression that this was similar to how HEVC improved over H.264. Mostly, extending techniques already laid down. Yet another reason to tip one's hat to the MP3 of video.
GeoffreyA - Saturday, December 5, 2020 - link
I found this some weeks ago. It goes into some lower-level details of VVC.https://www.cambridge.org/core/services/aop-cambri...