Earlier this week, Aptina invited me out to their San Jose office to take a look at their new Clarity+ CFA (Color Filter Array) technology and 13 MP AR1331CP CMOS image sensor which includes it. Clarity+ is Aptina's new 2x2 RC,CB CFA which includes two clear pixels where the traditional Bayer array places green pixels. Others have tried (and failed) with clear pixel filter arrays in the past, Aptina believes it has the clear pixels in the right place with their new filter and the right opportunity with industry state and availability of powerful ISPs. Aptina claims it can realize a 3-4 dB improvement in SNR with Clarity+ over Bayer, while keeping the same sharpness, avoiding color artifacts that plagued other clear pixel filters, and taking only a 0.2 dB hit in the 38 or so dB of dynamic range for a smartphone sensor. 

If you've been reading along with our smartphone coverage for some period of time, you'll know one of the big challenges facing mobile device and image sensor makers has to do with size – pixel size to be exact. The prevailing industry trend has been a steady march towards smaller pixels for a number of related reasons – with smaller pitch you can include more pixels in the same area, or keep the same number of pixels in a smaller area, thus allowing the handset maker to reduce the z-height of their image module thanks to a smaller area it has to form an image onto. The tradeoff of course is less area over which each pixel can integrate light during an exposure, something which up until this last 1.1µm step has largely been able to be mitigated by pixel design. The step from 1.4µm to 1.1µm however does come with a tradeoff in sensitivity, and as a result you see industry moves to mitigate this with OIS, multiple exposures, or avoiding it altogether.

Pixels in a digital camera only count photons, not the wavelength (color) of those incident photons. To resolve more than an image of luminance, you need a color filter array, and interpolate (demosaic) the color of each pixel based on its neighbors. The dominant color filter array for the last twenty or so years of digital imaging has been Bayer, a 2x2 pixel array consisting of RGB pixels, arranged RG,GB, with twice the number of green pixels to mimic human eye response which peaks in sensitivity in the green. This has been dominant in part thanks to how simple it is and how well it works – for most practical imaging applications, Bayer is great. Of course, adding a color filter means rejecting some light, in this case only photons inside the spectral response of the filter get passed, and in the smartphone space where pixels are getting smaller and smaller, every photon matters. For that reason, there's been much attention on using clear pixels somewhere in the array to increase sensitivity. There have been a number of different arrays tried, RG,CB, and 4x4 RG,BC arrays, all with some tradeoff. 

An immediate question is how Aptina avoids losing dynamic range in a CMOS sensor with clear pixels (the claim is 0.2 dB loss from the ~38 dB of dynamic range). The answer is that the clear pixel structure has been accordingly modified so it doesn't saturate before the blue and red pixels.

 Aptina's Clarity+ system is a subtractive system with clear pixels in the place of Bayer's green, blue and red in the same place, and still 2x2. The result is equivalent sharpness and lack of color artifacts, assuming good recovery of the green channel from clear without introducing more noise. Apina does this by using the luminance image from the clear pixels early on in the imaging pipeline to suppress any chroma noise, and resolves chroma from the RB and recovered G channels. 

The next question is then – how does Aptina's image processing pipeline fit into a world built around Bayer? The near term solution is for handset makers to use Aptina's own ISP which includes Clarity+ onboard, something it tells us to expect the first round of devices to ship with. Another alternative is an Aptina stage in the image processing pipeline that converts the RC,CB image data into Bayer-domain RG,GB data for use in current image processing pipelines. Long term, however, Aptina has already ported various stages of its image pipeline to current SoCs with good results, and is talking with the big SoC vendors about inclusion in the next round of tapeouts. The nice thing is that RC,CB still looks a lot like Bayer after the green channel is recovered, from an image processing perspective, as opposed to some of the other alternative filters. Aptina will continue to ship its CMOS sensors in Bayer as well as Clarity+ variants, with the CP suffix obviously connoting inclusion of the new color filter array.


(left) AR1331CP with Clarity+, (right) AR1230 with Bayer

How does this all work in practice? Aptina showed us a demonstration of its new AR1331CP 13 MP 1.1µm image sensor versus its AR1230 12MP 1.1µm sensor of its image sensor. At higher lux levels, there wasn't any visible reduction in image sharpness, and at low lux (10-8) the gain in SNR with the same integration time is dramatic. 

Moving beyond Bayer is something the smartphone industry has tried and failed to do before, with Clarity+ it looks like Aptina might have a much better chance at doing so without any of the loss of resolution, introduction of color artifacts, or other tradeoffs that other proposed clear pixel filters have introduced. There's still the issue of ISP and getting third party silicon vendors to adopt the platform, but handset vendors are enthusiastic enough about improving sensitivity in low light scenarios that there's sufficient pressure to do so. 

Source: Aptina (whitepaper - PDF)

POST A COMMENT

14 Comments

View All Comments

  • voodoobunny - Wednesday, July 17, 2013 - link

    Would this work with the cameras in the HTC One and Nokia 1020? If so, Aptima could be sitting on an IP goldmine - build their own 13MP sensors, and still license the technology to HTC and Nokia for lots o' money... Reply
  • Krysto - Friday, July 19, 2013 - link

    Thos companies are kind of pursuing their own ways for camera quality. You will probably see something like this in the Motorola X, though. Reply
  • blanarahul - Wednesday, July 17, 2013 - link

    Cool! But why green? Why not make the blue pixels clear pixels? Reply
  • A5 - Wednesday, July 17, 2013 - link

    Human eye's sensitivity peaks in the green area of the spectrum, but I don't know enough about the tech to say how they take that knowledge and turn it into a useful image. Reply
  • makerofthegames - Wednesday, July 17, 2013 - link

    Green sits in the middle, frequency-wise. Red is lower-frequency, blue is higher. I imagine that makes the math easier, having it in the middle rather than at one of the ends. Although I would have to wonder about out-of-range colors - does the clear also pick up infrared or ultraviolet? Reply
  • DanNeely - Wednesday, July 17, 2013 - link

    P&S/DSLR cameras use separate IR/UV filters. Given the thinness requirements in mobile; if I had to guess I'd say the coatings that absorb them are applied on top of an existing element. Probably part of the lens assembly since it's made of the same materials as a discrete filter.

    The filter itself is needed; otherwise you get screwed up color balance. ex plant leaves tend to be bright in IR.
    Reply
  • DanNeely - Wednesday, July 17, 2013 - link

    Because removing the green filters gets 50% of the light into the luminance channel instead of only 25% Reply
  • patrickjchase - Wednesday, July 17, 2013 - link

    They replaced green instead of red or blue because G is the largest contributor to L* to begin with. Intuitively (i.e. without going into trade secret knowledge) G is therefore the channel that they can "recover" from L* while causing the least noise.

    L* is simply perceptually linearized Y (as in CIEXYZ), and Y is about 60% green depending on the source RGB chromaticities. Blue would actually be the *worse* choice as Y is <10% B.
    Reply
  • thesavvymage - Wednesday, July 17, 2013 - link

    woosh Reply
  • Mayuyu - Wednesday, July 17, 2013 - link

    Less luminance noise in exchange for chroma noise probably. Reply

Log in

Don't have an account? Sign up now