Optimized Image Processing for Automotive Image Sensors with Novel Color Filter Arrays

Traditionally, image sensors have been optimized to produce images that look natural to humans. For images consumed by algorithms, what matters is capturing the most information. We can achieve this via higher resolution, but higher resolution means lower sensitivity. To increase resolution and maintain high sensitivity, color information can be sacrificed—but in automotive applications, color is critical. In response, suppliers offer image sensors that capture color information using novel color filter arrays (CFAs). Instead of the traditional RGGB array, these sensors use patterns like red-clear-clear-green (RCCG). These approaches yield good results for perception algorithms, but what about cases where images are used by both algorithms and humans? Can we reconstruct a natural-looking image from an image sensor using a non-standard CFA? In this talk, we explore novel CFAs and introduce Nextchip’s vision processor, which supports reconstruction of natural-looking images from image sensors with novel CFAs, including RGB-IR sensors.