Thin cloud removal in optical remote sensing images based on generative adversarial networks and physical model of cloud distortion

Abstract

Cloud contamination is an inevitable problem in optical remote sensing images. Unlike thick clouds, thin clouds do not completely block out background which makes it possible to restore background information. In this paper, we propose a semi-supervised method based on generative adversarial networks (GANs) and a physical model of cloud distortion (CR-GAN-PM) for thin cloud removal with unpaired images from different regions. A physical model of cloud distortion which takes the absorption of cloud into consideration was also defined in this paper. It is worth noting that many state-of-the-art methods based on deep learning require paired cloud and cloud-free images from the same region, which is often unavailable or time-consuming to collect. CR-GAN-PM has two main steps: first, the cloud-free background and cloud distortion layers were decomposed from an input cloudy image based on GANs and the principles of image decomposition; then, the input cloudy image was reconstructed by putting those layers into the redefined physical model of cloud distortion. The decomposition process ensured that the decomposed background layer was cloud-free and the reconstruction process ensured that generated background layer was correlated with the input cloudy image. Experiments were conducted on Sentinel-2A imagery to validate the proposed CR-GAN-PM. Averaged over all testing images, the SSIMs values (structural similarity index measurement) of CR-GAN-PM were 0.72, 0.77, 0.81 and 0.83 for visible and NIR bands respectively. Those results were similar to the end-to-end deep learning-based methods and better than traditional methods. The number of input bands and values of hyper-parameters affected little on the performance of CR-GAN-PM. Experimental results show that CR-GAN-PM is effective and robust for thin cloud removal in different bands.