Blind single-image-based thin cloud removal using a cloud perception integrated fast Fourier convolutional network

Abstract

Remote sensing images are frequently contaminated by clouds that often degrade the performance of subsequent applications. Cloud removal, therefore, is a standard step in remote sensing image preprocessing, and single-image-based thin cloud removal is a well-established area of research. Existing single-image-based thin cloud removal methods however, lack the capacity for simultaneous executions of efficient long-range modeling and physical attribute consideration. To extend this work and fill this gap, a novel blind single-image-based thin cloud removal method, called cloud perception integrated fast Fourier convolutional network (CP-FFCN), was designed and implemented. The CP-FFCN consists of two modules: the cloud perception module (CPM) and a fast Fourier convolution (FFC)-conducted reconstruction module (FFCN). The CPM uses a frequency spatial attention mechanism to realize long-range modeling of clouds, globally detect them in the cloudy image. It helps the CP-FFCN remove the clouds without external prior knowledge of the cloud distribution. The reconstruction module was designed with an FFC-conducted U-Net architecture to recover the clean images from cloudy scenarios, guided by the locations of clouds as detected by the CPM. In addition, the FFC blocks deployed in the encoder and decoder components in the U-Net architecture selectively learn the attributes of clouds and fogs from the frequency spectrograms to remove the clouds and reconstruct the underlying ground objects. The CP-FFCN selectively learns the frequency features for adequate cloud separation and at the same time efficiently models the long-range information for comprehensive scenario reconstruction with the help of these two modules. We adopted the Google Earth data and Landsat-8 imagery to train the CP-FFCN model and evaluate it on simulated and naturally occurring cloudy scenarios. The visual outcomes illustrate that the proposed CP-FFCN successfully removes thin and small-scale thick clouds with complex ground object scenarios, without external cloud masks and additional reference data. The quantitative analyses further demonstrate the higher effectiveness of the CP-FFCN when compared with several other state-of-the-art thin cloud removal methods, yielding a PSNR value over 39.24 and a SSIM value over 0.98 on the Landsat 8 images.