Understanding Camera Optics & Smartphone Camera Trends, A Presentation by Brian Klug
by Brian Klug on February 22, 2013 5:04 PM EST- Posted in
- Smartphones
- camera
- Android
- Mobile
The Camera Module & CMOS Sensor Trends
So after we have the lenses, what does that go into? Turns out there is some standardization, and that standardization for packaging is called a module. The module consists of of course our lens system, an IR filter, voice coil motor for focusing, and finally the CMOS and fanout ribbon cable. Fancy systems with OIS will contain a more complicated VCM and also a MEMS gyro somewhere in the module.
Onto CMOS, which is of course the image sensor itself. Most smartphone CMOSes end up being between 1/4“ and 1/3” in optical format, which is pretty small. There are some outliers for sure, but at the high end this is by far the prevailing trend. Optical format is again something we need to go look at a table for or consult the manufacturer about. Front facing sensors are way smaller, unsurprisingly. The size of the CMOS in most smartphones has been relatively fixed because going to a larger sensor would necessitate a thicker optical system, thus the real trend to increase megapixels has been more of smaller pixels.
The trend in pixel size has been pretty easy to follow, with each generation going to a different size pixel to drive megapixel counts up. The current generation of modern pixels is around 1.1 microns square, basically any 13 MP smartphone is shipping 1.1 microns, like the Optimus G, and interestingly enough others are using 1.1 microns at 8 MP to drive thinner modules, like the thinner Optimus G option or Nexus 4. The previous generation of 8 MP sensors were using 1.4 micron pixels, and before that at 5 MP we were talking 1.65 or 1.75 micron pixels. Those are pretty tiny pixels, and if you stop and think about a wave of very red light at around 700nm, we’re talking about 1.5 waves with 1.1 micron pixels, around 2 waves at 1.4 microns, and so forth. There’s really not much smaller you can go, it doesn’t make sense to go smaller than one wave.
There was a lot of talk about the difference between backside (BSI) and front side illumination (FSI) for systems as well. BSI images directly through silicon into the active region of the pixel, whereas FSI images through metal layers which incur reflections and a smaller area and thus loss of light. BSI has been around for a while in the industrial and scientific field for applications wanting the highest quantum efficiency (conversion of photons to electrons), and while they were adopted in smartphone use to increase the sensitivity (quantum efficiency) of these pixels, there’s an even more important reason. With pixels this small in 2D profile (eg 1.4 x 1.4 microns) the actual geometry of a pixel began to look something like a long hallway, or very tall cylinder. The result would be quantum blur where a photon being imaged onto the surface of the pixel, converted to an electron, might not necessarily map to the appropriate active region underneath - it takes an almost random walk for some distance. In addition the numerical aperture of these pixels wouldn’t be nearly good enough for the systems they would be paired with.
Around the time I received the One X and One S last year, I finally became curious about whether we could ever see nice bokeh (blurry background) with an F/2.0 system and small pixels. While trapped on some flight somewhere, I finally got bored enough to go quantify what this would be, and a side effect of this was some question about whether an ideal, diffraction limited (no aberrations, ideal, if we had perfect optics) system could even resolve a spot the size of the pixels on these sensors.
It turns out that we can’t, really. If we look at the airy disk diameter formed from a perfect diffraction limited HTC One X or S camera system (the parameters I chose since at the time this was, and still is, the best system on paper), we get a spot size around 3.0 microns. There’s some fudge factor here since interpolation takes place thanks to there being a bayer grid atop the CMOS that then is demosaiced, more on that later, so we’re close to being at around the right size, but obviously 1.1 microns is just oversampling.
Oh, and also here are some hyperfocal distance plots as a function of pixel size and F/# for the same system. It turns out that everything is in focus pretty close to your average smartphone, so you have to be petty close to the subject to get a nice bokeh effect.
Facebook
Twitter
RSS
60 Comments
View All Comments
gadjade - Friday, February 22, 2013 - link
How can I start reading when there is no mention of any Nokia products? HTC One should not be even included in the article because it's not even considered a breakthrough in camera phones.Krysto - Saturday, February 23, 2013 - link
Next time try reading more than the first paragraph.Diagrafeas - Saturday, February 23, 2013 - link
How many MP or to be more accurate Million Sub Pixels are these Nokia sensors because these numbers come surely after interpolation or pixel shift or something...Also do we have information about HTC One sensor?
Is it one Bayer sensor or 2-3 stacked like Foveon ones?
Krysto - Saturday, February 23, 2013 - link
While HTC definitely deserves credit for discontinuing a trend of ever more MPs, I don't think they went far enough. There's still an unclenched status quo thinking in smartphones about sensor size. Virtually all of them have a 1/3.2" sensor.No one has even thought about making that one much larger? Even after Nokia showed Pureview 808 and got a lot of praise for it? Really? What is wrong with all these companies? Do they want us to spell it out for them?
I get that clean phone design is a big factor, but you can't just keep doing things the same way everyone has always done it. And then they wonder why they can't beat Samsung. Sure Samsung has a lot of marketing power, but they also play a little less safe than everyone else. They add the S-pen to devices, even though it adds quite a bit of cost and then they need that device to compete with others, so it's a risk for them that the consumers might not want to pay extra for it - but they still do it. They also started the "phablet" trend all by themselves. And while these things don't necessarily have mass market appeal, they get a lot of publicity and quite a lot of passionate fans and customers for those devices.
So why aren't the other manufacturers experimenting in the same way not just with incrementally better cameras, but WAY better cameras, that they put in phones. I'm talking putting camera capabilities in a phone that could add $100 or even $200 to the retail price of the device. That's being BOLD in the market. That's being DIFFERENT.
So I want to see them come up with devices that have 1" large sensors of 5-8 MP, with some high quality lenses, and powerful ISP's and software behind them. Create that and you get at least 3 different types of consumers coming to buy that phone (the type of consumers that yell :SHUT UP AND TAKE MY MONEY!): professional photographers (they buy thousands of dollars worth of equipment for fun, and they need phones, too), amateur photographers (people who love taking great photos with their phones), and you'd also pretty much convert the whole (in time of course) point and shoot market to your device.
So the potential for sales is right there for reaping. And these people wouldn't get about a slight bump on the back, if they can get a phone that is 5x better than anything else on the market at the time of shipping. I want to see that kind of LEAP in smartphone cameras, not just these regular "2013 camera is slightly better than the 2012 camera", and so on.
slatanek - Saturday, February 23, 2013 - link
I think you're generally right, but these days from manufacturer's point of view it's very easy to sink the ship by being bold and not being understood well by the market/consumers. It is then very easy for their opponents to do nasty counter marketing to make things worse even further.As a photographer myself I really would like to see a no compromise smartphone camera (give glass!!! give me bigger sensor with less megapixels!!!), but I guess we're not at that point just yet. Only recently manufacturers of compact cameras started to show some interest for making cameras geared toward serious photography, so I guess it's still a few years wait to see that approach in the smartphone world.
slatanek - Saturday, February 23, 2013 - link
Besides I guess 1" sensor in a smartphone will never happen due to physical constrains - you need a lot of light gathering power to lit such a huge sensor not to mention the focal distance needs to be considerably bigger for the field of view not to be ultra wideKrysto - Saturday, February 23, 2013 - link
Pureview 808 had a 1/1.2" which is pretty close, and didn't look too bad:http://images.fonearena.com/blog/wp-content/upload...
Tarwin - Saturday, February 23, 2013 - link
That's kinda the point. It didn't look TOO BAD, which is a far cry from looking great. I also remember reading some reviews which complained about the ergonomics. Also the pure view was more of a niche experimental product.It's kind of like some article that I read years agonwhere itnsaidnthat according to research (or a survey or something) consumers, when buying a tv), were most interested in size THEN image quality and then all the other stuff. Most phone buyers are similar, they want a better camera but are not willing to give up styling for it, at least not without some re-education.
As for HTC not going far enough. I think it has more to do with the fact that they're not doing too well as is and most likely feel like they're taking enough risks at the moment.
Personally I would be willing to give up some slimness (but hopefully not styling) for a decidedly improved camera. I hope HTC does well with the One and that they are willing to experiment a bit more in future generations. But then again I am not sure yet how I feel about the loss in detail. Another site got their hands on a One and have some comparison shots between the On eand the iPhone 5
Tarwin - Saturday, February 23, 2013 - link
Oops, accidentally pressed post.Well, in the comparison shots I prefernthe images ofnthe One, but I occasionally neednthe detail procided with 8MP but then again only sometimes. When 4K TVs vecome mainstream and I also have one then I'll kost likely think it's the minimum pics should be taken at.
Krysto - Monday, February 25, 2013 - link
They can make it more stylish than that. I wouldn't mind if my phone looked a bit like a point and shoot, and didn't have a perfectly flat back surface.Again, I'm not saying these phones are for everyone. Note phones are not for everyone either. In fact I don't think I can ever see myself owning a Note phone. But yet millions of people have them, and those who do love it, and wouldn't imagine going back to a smaller phone.
That's the type of market I think such a phone can target. A niche market indeed, but a big niche nonetheless. And I would be part of that niche.