Camera Image Quality Tradeoff Processing of Image Sensor Re-mosaic using Deep Neural Network

Abstract

Recently, with the release of 108 mega pixel resolution image sensor, the photo quality of smartphone camera, including detail, and texture, is getting much higher. This became possible only because by utilizing the remosaic technology which re-organize color filter arrays into the Bayer patterns compatible to existing Image Signal Processor (ISP) of commodity AP. However, the optimized parameter configurations of the remosaic block require lots of efforts and long tuning period in order to secure the desired image quality level and sensor characteristics. This paper proposes a deep neural network based camera auto-tuning system for the remosaic ISP block. Firstly, considering the learning phase, big image quality database is created in the random way using reference image and tuning register. Second, the virtual ISP model has been trained in order that predicts image quality by changing sensor tuning registers. Finally, the optimization layer generates the sensor remosaic parameters in order to achieve the user’s target image quality expectation. By experiment, the proposed system has been verified to secure the image quality at the level of professionally hand-tuned photography. Especially, the remosaic artifact of false color, color desaturation and line broken artifacts are improved significantly by more than 23%, 4%, and 12%, respectively.