A deep-learning reconstruction method for remote sensing images with large thick cloud cover

Abstract

Thick cloud and cloud shadow often exist in optical remote sensing (RS) images, which limit the application range of RS data. To solve this challenging problem, we propose a multitemporal-based method of cloud removal to reconstruct the large cloud-cover region in RS images. The method can combine the advantages of high accuracy of deep-learning method and low computation cost of traditional method, and reduce the difficulty of establishing multi-temporal dataset to a certain extent. In the proposed method, the complex problem of thick cloud removal is decomposed into two simple processes: coarse stage and refined stage. In the stage of coarse cloud removal, it is found that the cloud-cover images are distributed more regularly in frequency domain than in the space domain. Therefore, the frequency domain residual module is introduced, and the frequency domain attention module is designed to achieve more focused fusion of three-temporal features. In the stage of refined cloud removal, the refined network is designed to retain the effective features of the coarse stage and further refine the reconstruction result based on the image self-correlation. We carried out sufficient experiments to demonstrate the superiority of the proposed method compared with four classical methods, which include WLR, STSCNN, RFRnet and PSTCR. In qualitative experiments, the proposed method has obvious advantages in information accuracy, visual naturalness and semantic rationality. In quantitative experiments, compared with the best of four comparison methods, average values of Peak Signal to Noise Ratio (PSNR) on three-temporal images are improved by 5.1934 dB, 2.7729 dB and 7.2364 dB respectively, and average values of Correlation Coefficient (CC) on three-temporal images are increased by 0.1048, 0.0110 and 0.0966 respectively. Moreover, the proposed method has an obvious advantage in computation cost because it can reconstruct multiple cloud-cover images synchronously.