The Imaging Chain

Since we’re talking about a smartphone we must understand the imaging chain, and thus block diagram, and how the blocks work together. There’s a multiplicative effect on quality as we move through the system from left to right. Good execution on the optical system can easily be mitigated away by poor execution on the ISP for example. I put arrows going left to right from some blocks since there’s a closed loop between ISP and the rest of the system.

The video block diagram is much the same, but includes an encoder in the chain as well.

Smartphone Cameras: The Constraints

The constraints for a smartphone camera are pretty unique, and I want to emphasize just how much of a difficult problem this is for OEMs. Industrial design and size constraints are pretty much the number one concern — everyone wants a thin device with no camera bump or protrusion, which often leaves the camera module the thickest part of the device. There’s no getting around physics here unfortunately. There’s also the matter of cost, since in a smartphone the camera is just one of a number of other functions. Material constraints due to the first bullet point and manufacturing (plastic injection molded aspherical shapes) also makes smartphone optics unique. All of this then has to image onto tiny pixels.

Starting with the first set of constraints are material choices. Almost all smartphone camera modules (excluding some exceptions from Nokia) the vast majority of camera optics that go into a tiny module are plastic. Generally there are around 2 to 5 elements in the system, and you’ll see a P afterwards for plastic. There aren’t too many optical plastics around to chose from either, but luckily enough one can form a doublet with PMMA as something of a crown (low dispersion) and Polystyrene as a flint (high dispersion) to cancel chromatic aberration. You almost always see some doublet get formed in these systems. Other features of a smartphone are obvious but worth stating, they almost always are fixed focal length, fixed aperture, with no shutter, sometimes with an ND filter (neutral density) and generally not very low F-number. In addition to keep modules thin, focal length is usually very short, which results in wide angle images with lots of distortion. Ideally I think most users want something between 35 mm or 50 mm in 35mm equivalent numbers.

I give an example lens catalog from a manufacturer, you can order these systems premade and designed to a particular sensor. We can see the different metrics of interest, thickness, chief ray angle, field of view, image circle, thickness, and so on.

During undergrad a typical homework problem for optical design class would include a patent lens, and then verification of claims about performance. Say what you want about the patent system, but it’s great for getting an idea about what’s out there. I picked a system at random which looks like a front facing smartphone camera system, with wide field of view, F/2.0, and four very aspherical elements.

Inside a patent is a prescription for each surface, and the specification here is like almost all others in format. The radius of curvature for each surface, distance between surfaces, index, abbe number (dispersion), and conic constant are supplied. We can see again lots of very aspherical surfaces. Also there’s a doublet right for the first and second element (difference in dispersion and positive followed by negative lens) to correct some chromatic aberrations.

What do these elements look like? Well LG had a nice breakdown of the 5P system used in its Optimus G, and you can see just what the lenses in the system look like.

Smartphone Imaging The Camera Module & CMOS Sensor Trends
Comments Locked

60 Comments

View All Comments

  • mdar - Thursday, February 28, 2013 - link

    You say "This is the standard format for giving a sensor size, but it doesn’t have anything to do with the actual size of the image circle, and rather traces its roots back to the diameter of a vidicon glass tube"
    The above statement, though partially true, is misleading. The dimension DOES give sensor size multiplied by factor of roughly 1.5. For example if some one says 1/1.8" sensor, the sensor diagonal is ~ 1/(1.8*1.5). The 1.5 factor probably comes from vidicon glass tube.

    Infact if some one wants just one parameter for image quality, it should be sensor size. Pixel technologies do improve (like using BSI) but even now a 1/3" sensor size of iphone or samsung or lumia 920 camera can just barely match quality 1/1.8" sensor of 4-year old Nokia N8.
  • frakkel - Thursday, February 28, 2013 - link

    I am currious if you can elaborate a little regarding lens material.

    You say that today most lens elements are made of plastic. Is this both for front and rear facing camera lenses?

    I was under the impression that lens elements in phones still were made of glass but that the industry is looking to change to plastic but this change has not been done yet. Please correct me if I am wrong and a link or two would not hurt :)
  • vlad0 - Friday, March 1, 2013 - link

    I suggest reading this white paper as well:

    http://www.mediafire.com/view/?0o5oo43h8os4ba9

    it deals with a lot of the limitations of a smartphone camera in a very elegant way, and the results are sublime.

    http://sdrv.ms/VQ3eCd

    Nokia solved several important issues the industry has been dealing with for a long time...
  • wally626 - Monday, March 4, 2013 - link

    Although the term Bokeh is commonly used to refer to the effect in pictures of low depth of field techniques it should only be used to refer to the quality of the out-of-focus regions of such photographs. It is much more an aesthetic term than technical. Camera phones usually have such deep depth of focus that little is out of focus in normal use. However, with the newer f/2, f/2.4 phone cameras when doing close focus you can get the out of focus regions from low depth of field.

    http://www.zeiss.com/c12567a8003b8b6f/embedtitelin...$file/cln35_bokeh_en.pdf

    Is a very good discussion of this by Dr. Nasse of Zeiss
  • wally626 - Monday, March 4, 2013 - link

    Someone fixed the Zeiss link to the Nassw article for me awhile back but I forgot the exact fix. In any case a search on the terms Zeiss, Nasse and Bokeh should bring up the article.
  • huanghost - Thursday, March 21, 2013 - link

    admirable
  • mikeb_nz - Sunday, December 22, 2013 - link

    how do i calculate or where do i find the field of view (angle of view) for smartphone and tablet cameras?
    thanks
  • oanta_william - Monday, July 20, 2015 - link

    Your insight in Smartphone Cameras is awesome! Thanks for everything!

    From your experience would it be possible to have only the camera module on a device, with a micro-controller/SoC that has sufficient power ONLY for transmitting the non processed 'RAW' data on another device via Bluetooth - on which the ISP and the rest needed for image processing to be situated.

    I have a homework regarding this. Do you now any reference material/books that could help me?

    Thanks!
  • solarkraft - Monday, January 9, 2017 - link

    What an amazing article! Finally something serious about smartphone imaging (the processor/phone makers don't tell us ****)! Just an updated version might be cool.
  • albertjohn - Tuesday, November 20, 2018 - link

    I like this concept. I visited your blog for the first time and became your fan. Keep posting as I am going to read it everyday.

Log in

Don't have an account? Sign up now