CSRDNN: An Integrated Scheme for Single Satellite Image Colorization and Super-Resolution Using Deep Neural Networks

Abstract

Deep convolutional neural networks have respectively achieved significant success in image super-resolution and colorization. The DNN has a strong capability to generate high quality images. Both colorization and super-resolution (SR) can be regarded as an independent pixel mapping problem, and this work combines these two visual problems into an integrated task. In this work, we propose an end-to-end model for accomplishing single satellite image colorization and SR simultaneously. Our model comprises two phases: features extraction network and recovery network. First, the residual receptive field block structure is introduced in features extraction network to learn better feature representations for image colorization and SR. Residual Receptive Field Block(RRFB) is improved by expanding the receptive field and enhancing the context connection from inception model. Second, the extracted features are transformed to a color high-resolution image by a recovery architecture. In this work, U-net is employed as the key structure of the recovery architecture. Besides, the squeeze-and-excitation blocks and complex residual blocks are incorporated into the proposed model to increase the reconstruction performance. To verify the performance, our method is compared with the state-of-the-art methods of SR and colorization. The experiments show that proposed method can get competitive in visual effect and evaluation index compared with the existing methods. In the end, the panchromatic dataset is also used to validate our model, and a good color high-resolution image can be obtained by giving a gray and low-resolution panchromatic image.