Understanding Camera Optics & Smartphone Camera Trends, A Presentation by Brian Klug
by Brian Klug on February 22, 2013 5:04 PM EST- Posted in
- Smartphones
- camera
- Android
- Mobile
Evaluating Image Quality
How do we evaluate quality from an image? Tomes have been written about this, and really there are many things to look for in a good image. Chief among those is really sharpness, or MTF, the modulation transfer function. That’s a discussion in and of itself, but basically MTF plots show us how much contrast we will see in a square wave at a particular spatial frequency. MTF also tells us about what the highest frequency (spatial resolution) will make it through a system, this is the cutoff frequency. There are other things to look for too, like third order aberrations.
No camera system is perfect, and good design balances one aberration against the other. If we look at field dependency the most difficult part of an image for a designer is the edges, where aberrations increase quickly.
These previous aberrations have been monochromatic, there are also aberrations which exist as a function of wavelength or color. Axial chromatic we can fix with a doublet to some extent or try to minimize. Transverse is what we sometimes see with color fringing, although in most commercial systems purple fringing is often an artifact of ISP.
So what can we look for? Again, distortion is visible quickly since these systems in a smartphone are so wide angle. Chromatic fringing since this is annoying and something easy to notice on silhouetted subjects. Obviously sharpness is a big deal, does the image look blurry. Finally the presence of any residual vignetting and lens color shading, despite lots of gnashing of teeth from the optical designers and lots of ISP tweaking — which if you’re like my ex girlfriend you’re going to add back in with Instagram or Twitter filters to look “vintage,” you hipster. Test charts will tell us a lot, and there are many good choices, but good test scenes sometimes tell a lot more.
I hate pictures of keyboards in reviews since they’re the laziest subject of all to photograph when doing a review of a smartphone, but here’s one I couldn’t resist. The image is so noisy I can’t read the keys, and the totally homogenous desk looks awash with luminance noise. There isn’t much chroma (color) noise.
Here’s one I complain about a lot, huge halos around contrasty regions thanks to the sharpening kernel or unsharp mask applied to the image. This is an attempt by the OEM to add back in spatial resolution or contrast after killing it all with noise reduction, and after you see halos you won’t un-see them. We can also see some serious moire in the bottom left, partly why I love that scene.
This is a photo from a recently released device which clearly has some strong field curvature. Again the center of the image is easy to get nice and sharp, but if you look at the edges, it gets dramatically blurry. The center is easy, the edge of the field is hard.
There was a very popular phone which was criticized for having some purple color stray light visible in the image when a light source was just out of the field of view. It turns out stray light is a big issue for everyone, since obviously nobody wants a huge lens hood sticking out of their phone, or at least industrial designers don’t. Well, again, this isn’t an isolated problem for just one vendor, it’s something everyone has. I believe the purple color gets picked up from a magnesium fluoride antireflection coating or some other AR coating.
The image on the left is from a very popular device, and the image on the right is of the next generation of this popular device. The left image has a very pronounced green spot in the center, and then a definite red ring around the outside. After you see this pattern, it’s unlikely you’ll be able to un-see it. I used to play a game on Reddit looking for the green circle in people’s images, then going and checking EXIF, and about 90 percent of the time I could nail what smartphone this was coming from, just from the green spot. This is classic failure to correct for lens color shading, either their ISP couldn’t do it or they didn’t characterize it well enough, but it was fixed in the next generation. These lens shading errors are incredibly annoying when taking a photo of a subject with a flat monochromatic field, like a book, whiteboard, or so forth.
There are other things that I look for as well, aggressive noise reduction, again moire, bad auto white balance are pretty easy to spot. Another annoyance are cameras which completely miss focus, even on very contrasty scenes which should be easy to focus on with contrast based auto focus.
Facebook
Twitter
RSS
60 Comments
View All Comments
ltcommanderdata - Saturday, February 23, 2013 - link
http://www.sony.net/SonyInfo/News/Press/201208/12-...Sony's new Exmor RS sensors use a stacked structure which places the circuit section underneath the pixel section instead of beside it which should free up more room for more pixels or larger pixels within a given sensor module area or enable smaller modules. They also add a dedicated white channel for a RGBW coding which they claim improves low light performance. Any comment on the efficacy of these techniques?
Apple has been using Sony image sensors for both the iPhone 4S and iPhone 5 so that new Sony IMX135 with 13.13MP, 1/3.06", 1.12 μm pixel sensor looks like a prime candidate for the iPhone 5S.
slatanek - Saturday, February 23, 2013 - link
Well done, Brian. At last someone trying to explain and stress how much more is there to tiny smartphone cameras than just the megapixels. A few years ago I felt like we got over it and the race was over, but now few years later the race just continues but in the smartphone realm (previously compact cameras had the same issues where ultimate picture quality was compromised just to get to a higher number of megapixels on the box). Hopefully this will change in near future, but somehow I'm afraid that were stuck with it just as with the horrible quality displays in notebooks. And it's not that people don't want good quality, it's just that the behemot companys are not willing to take the risk.Anyways, thanks for a good read.
Shftup - Saturday, February 23, 2013 - link
Brain - Well done!This is a fanatastic article. Its well written, relatively easy to understand for any layman, and most importantly makes the reader coming back for more.
jabber - Saturday, February 23, 2013 - link
....for a smartphone to give me as good a picture quality as I got from my 3.2MP Nikon from 2004.When I find one I'll be happy.
I think a lot of the phone companies need to start poaching the optics and software specialists from the camera companies. It's all very well looking at the spec lists for components but its another thing entirely to make them all work together to produce a decent picture.
In the past for me near decent cams have been ruined by over zealous compression settings (Palm Pre2) or what appears to be zero configuration of the imaging processors (Nexus4).
No one yet has tweaked every part of the chain to provide a truly viable alternative to taking a $250+ camera along instead.
Never had anything to do with megapixels IMO. It's other factors that let them down.
I reckon in another 2 years we'll have it pretty much there.
MrSpadge - Sunday, February 24, 2013 - link
Same here, my Sony DSC-W5 from 2005 or so is still way better than my smartphone. If I could get at least similar quality in a new phone that would easily make it worth 50 - 100€ more for me.Tarwin - Saturday, February 23, 2013 - link
I enjoyed the article, it cleared up some doubts I had and taught me a few new things.In light of this article I hope you go into more detail in the One review when you eventually do it. I assume you will give special attention to the c amera due to how it goes against the trend and HTC's focus on it but I also hope you mention how it fits into all this a bit for those of us who read both articles. Thanks again.
DominicG - Saturday, February 23, 2013 - link
Hi Brian,much enjoyed your intro to camera optics. However you state:
"If we look at the airy disk diameter formed from a perfect diffraction limited ... we get a spot size around 3.0 microns"
You mentioned this is a back illuminated chip so that the light is focussing in a medium of refractive index ~3.5. Therefore the wavelength of red light inside the medium is ~700/3.5 = 200nm - still much smaller than the pixel. It is a bit more complicated than this since the optical resolution is of course determined not only by the wavelength but also the numerical aperture (NA) - the light refracts at the air-silicon interface so that a beam in air at 37 deg (2omega =75deg - your example lens), becomes a beam in silicon of only 10 deg. 10 deg in silicon gives a theoretical resolution (Rayleigh condition) of 0.7microns. Anyway, did you take all these factors into account?
fokka - Sunday, February 24, 2013 - link
thanks for this insightful article, brian, that's the sort of read i i visit this site for regularly!i have to say that i expected a little more (side-) content, though, like a more thorough look at the htc one initially pictured, one of the two reasons i clicked on the article. also a comparison with nokias pureview-approach would've been nice, since it's on the complete other side of the spectrum.
but this is just to nitpick, as i enjoyed the read none the less.
StormyParis - Sunday, February 24, 2013 - link
Great articleStormyParis - Sunday, February 24, 2013 - link
Seeing the interest everyone has in good pictures, and the relatively small size and cost of the lens apparatus, wouldn't it make sense to have 2 distinct cameras on a smartphone, and coordinate them to get better image quality ?