Pseudo-ISP: Learning pseudo in-camera signal processing pipeline from a color image denoiser

Abstract

Existing deep denoisers usually ignore the problem of noise discrepancy between training and test images caused by the different noise modeling of sensor and in-camera signal processing (ISP) pipeline, which inevitably degrades the denoising performance. In this paper, we present an unpaired learning scheme to adapt the color image denoisers for handling test images with noise discrepancy. To this end, we consider a practical training setting equipped with a pre-trained color denoiser, a set of test noisy sRGB images, and an unpaired set of clean sRGB images. Then, we propose to alternate between two modules, i.e., Pseudo-ISP training for learning noise modeling of sensor and in-camera signal processing (ISP) pipeline and denoiser adaption. Instead of modeling the complex noise in sRGB images, we assume the signal-dependent and spatially independent noise in rawRGB space, and specifically design the Pseudo-ISP to learn the pseudo ISP pipeline and pseudo rawRGB noise model jointly. Applying the learned Pseudo-ISP to the unpaired set of clean sRGB images, the corresponding realistic noisy sRGB observations can be easily obtained. Conversely, using the generated paired data for model fine-tuning, the color denoiser can be well adapted for test images. Through iterating between Pseudo-ISP training and denoiser adaption, denoising performance of deep denoisers can be further improved. Experiments show that our Pseudo-ISP not only can boost simple Gaussian blurring-based denoiser to achieve competitive performance against CBDNet, but also is effective in improving state-of-the-art deep denoisers, e.g., CBDNet and RIDNet.