Thin Cloud Removal Research Articles

Thin Cloud Removal Generative Adversarial Network Based on Sparse Transformer in Remote Sensing Images

Thin clouds in Remote Sensing (RS) imagery can negatively impact subsequent applications. Current Deep Learning (DL) approaches often prioritize information recovery in cloud-covered areas but may not adequately preserve information in cloud-free regions, leading to color distortion, detail loss, and visual artifacts. This study proposes a Sparse Transformer-based Generative Adversarial Network (SpT-GAN) to solve these problems. First, a global enhancement feature extraction module is added to the generator’s top layer to enhance the model’s ability to preserve ground feature information in cloud-free areas. Then, the processed feature map is reconstructed using the sparse transformer-based encoder and decoder with an adaptive threshold filtering mechanism to ensure sparsity. This mechanism enables that the model preserves robust long-range modeling capabilities while disregarding irrelevant details. In addition, inverted residual Fourier transformation blocks are added at each level of the structure to filter redundant information and enhance the quality of the generated cloud-free images. Finally, a composite loss function is created to minimize error in the generated images, resulting in improved resolution and color fidelity. SpT-GAN achieves outstanding results in removing clouds both quantitatively and visually, with Structural Similarity Index (SSIM) values of 98.06% and 92.19% and Peak Signal-to-Noise Ratio (PSNR) values of 36.19 dB and 30.53 dB on the RICE1 and T-Cloud datasets, respectively. On the T-Cloud dataset, especially with more complex cloud components, the superior ability of SpT-GAN to restore ground details is more evident.

Variational mode decomposition-enhanced thin cloud removal using UNet vision transformer cycle-consistent generative adversarial network

Variational mode decomposition-enhanced thin cloud removal using UNet vision transformer cycle-consistent generative adversarial network

MCDNet: Multilevel cloud detection network for remote sensing images based on dual-perspective change-guided and multi-scale feature fusion

Cloud detection plays a crucial role in the preprocessing of optical remote sensing images. While extensive deep learning-based methods have shown strong performance in detecting thick clouds, their ability to identify thin and broken clouds is often inadequate due to their sparse distribution, semi-transparency, and similarity to background regions. To address this limitation, we introduce a multilevel cloud detection network (MCDNet) capable of simultaneously detecting thick and thin clouds. This network effectively enhances the accuracy of identifying thin and broken clouds by integrating a dual-perspective change-guided mechanism (DPCG) and a multi-scale feature fusion module (MSFF). The DPCG creates a dual-input stream by combining the original image with the thin cloud removal image, and then utilizes a dual-perspective feature fusion module (DPFF) to perform feature fusion and extract change features, thereby improving the model's ability to perceive thin cloud regions and mitigate inter-class similarity in multilevel cloud detection. The MSFF enhances the model's sensitivity to broken clouds by utilizing multiple non-adjacent low-level features to remedy the missing spatial information in the high-level features during multiple downsampling. Experimental results on the L8-Biome and WHUS2-CD datasets demonstrate that MCDNet significantly enhances the detection performance of both thin and broken clouds, and outperforms state-of-the-art methods in accuracy and efficiency. The code of MCDNet is available in https://github.com/djw-easy/MCDNet.

Unsupervised remote sensing image thin cloud removal method based on contrastive learning

AbstractCloud removal algorithm is a crucial step of remote sensing image preprocessing. The current mainstream remote sensing image cloud removal algorithms are implemented based on deep learning, and most of them are supervised. A large number of data pairs are required for training to achieve cloud removal. However, real with/without cloud image pairs datasets are difficult to obtain in the real world, and the models obtained by training on synthetic datasets often need to generalize better to natural scenes. And the existing unsupervised thin cloud removal methods based on Cycle‐GAN framework with considerable model complexity and unstable training are not an excellent solution to the problem of lack of paired datasets. Based on this, in this paper, the authors propose an unsupervised remote sensing image thin cloud removal method based on contrastive learning—GAN‐UD. It is a network consisting of a frequency‐spatial attention generator and a discriminator. In addition, the authors introduce local contrastive loss and global content loss to constrain the content of the generated images to ensure that the generated cloud‐free images are consistent with the input cloud images in terms of image content. Experimental results show that the proposed method in this paper can still effectively remove thin clouds from remote sensing images without paired training datasets, outperforms current unsupervised cloud removal methods, and achieves comparable performance to supervised methods.

Double Rank-One Prior: Thin Cloud Removal by Visible Bands

Double Rank-One Prior: Thin Cloud Removal by Visible Bands

HF-T2CR:High-Fidelity Thin and Thick Cloud Removal in Optical Satellite Images through SAR Fusion

HF-T2CR:High-Fidelity Thin and Thick Cloud Removal in Optical Satellite Images through SAR Fusion

TCME: Thin Cloud Removal Network for Optical Remote Sensing Images Based on Multidimensional Features Enhancement

TCME: Thin Cloud Removal Network for Optical Remote Sensing Images Based on Multidimensional Features Enhancement

PM-LSMN: A Physical-Model-Based Lightweight Self-Attention Multiscale Net for Thin Cloud Removal

PM-LSMN: A Physical-Model-Based Lightweight Self-Attention Multiscale Net for Thin Cloud Removal

DecloudNet: Cross-Patch Consistency is a Nontrivial Problem for Thin Cloud Removal From Wide-Swath Multispectral Images

DecloudNet: Cross-Patch Consistency is a Nontrivial Problem for Thin Cloud Removal From Wide-Swath Multispectral Images

Blind single-image-based thin cloud removal using a cloud perception integrated fast Fourier convolutional network

Remote sensing images are frequently contaminated by clouds that often degrade the performance of subsequent applications. Cloud removal, therefore, is a standard step in remote sensing image preprocessing, and single-image-based thin cloud removal is a well-established area of research. Existing single-image-based thin cloud removal methods however, lack the capacity for simultaneous executions of efficient long-range modeling and physical attribute consideration. To extend this work and fill this gap, a novel blind single-image-based thin cloud removal method, called cloud perception integrated fast Fourier convolutional network (CP-FFCN), was designed and implemented. The CP-FFCN consists of two modules: the cloud perception module (CPM) and a fast Fourier convolution (FFC)-conducted reconstruction module (FFCN). The CPM uses a frequency spatial attention mechanism to realize long-range modeling of clouds, globally detect them in the cloudy image. It helps the CP-FFCN remove the clouds without external prior knowledge of the cloud distribution. The reconstruction module was designed with an FFC-conducted U-Net architecture to recover the clean images from cloudy scenarios, guided by the locations of clouds as detected by the CPM. In addition, the FFC blocks deployed in the encoder and decoder components in the U-Net architecture selectively learn the attributes of clouds and fogs from the frequency spectrograms to remove the clouds and reconstruct the underlying ground objects. The CP-FFCN selectively learns the frequency features for adequate cloud separation and at the same time efficiently models the long-range information for comprehensive scenario reconstruction with the help of these two modules. We adopted the Google Earth data and Landsat-8 imagery to train the CP-FFCN model and evaluate it on simulated and naturally occurring cloudy scenarios. The visual outcomes illustrate that the proposed CP-FFCN successfully removes thin and small-scale thick clouds with complex ground object scenarios, without external cloud masks and additional reference data. The quantitative analyses further demonstrate the higher effectiveness of the CP-FFCN when compared with several other state-of-the-art thin cloud removal methods, yielding a PSNR value over 39.24 and a SSIM value over 0.98 on the Landsat 8 images.

An Innovative Approach for Effective Removal of Thin Clouds in Optical Images Using Convolutional Matting Model

Clouds are the major source of clutter in optical remote sensing (RS) images. Approximately 60% of the Earth’s surface is covered by clouds, with the equatorial and Tibetan Plateau regions being the most affected. Although the implementation of techniques for cloud removal can significantly improve the efficiency of remote sensing imagery, its use is severely restricted due to the poor timeliness of time-series cloud removal techniques and the distortion-prone nature of single-frame cloud removal techniques. To thoroughly remove thin clouds from remote sensing imagery, we propose the Saliency Cloud Matting Convolutional Neural Network (SCM-CNN) from an image fusion perspective. This network can automatically balance multiple loss functions, extract the cloud opacity and cloud top reflectance intensity from cloudy remote sensing images, and recover ground surface information under thin cloud cover through inverse operations. The SCM-CNN was trained on simulated samples and validated on both simulated samples and Sentinel-2 images, achieving average peak signal-to-noise ratios (PSNRs) of 30.04 and 25.32, respectively. Comparative studies demonstrate that the SCM-CNN model is more effective in performing cloud removal on individual remote sensing images, is robust, and can recover ground surface information under thin cloud cover without compromising the original image. The method proposed in this article can be widely promoted in regions with year-round cloud cover, providing data support for geological hazard, vegetation, and frozen area studies, among others.

Thin cloud correction method for visible remote sensing images using a spectral transformation scheme

ABSTRACT Thin cloud contamination adversely influences the interpretation of ground surface information in optical remote sensing images, especially for visible spectral bands with short wavelengths. To eliminate the thin cloud effect in visible remote sensing images, this study proposes a thin cloud correction method based on a spectral transformation scheme. Given the strong linear correlation among the three visible bands, a spectral transformation scheme is presented to derive intermediate images and further estimate spectrally varied transmission maps for the three visible bands. An improved strategy that locally estimates atmospheric light was also developed to produce spatially varied atmospheric light maps. Spectrally varied transmission maps and spatially varied atmospheric light maps contribute to the complete removal of thin clouds. Several remotely sensed images featuring various land covers were collected from Landsat 8 Operational Land Imager, Landsat 7 Enhanced Thematic Mapper Plus, and GaoFen-2 to perform simulated and real tests and verify the effectiveness and universality of the presented approach. Four existing thin cloud correction algorithms were used as baseline for comprehensive evaluation, including three single-image-based approaches and a deep-learning-based approach. Results demonstrate that the proposed method outperformed the other four baseline methods, yielding more natural and clearer cloud-free images. The ground surface information in the cloud-contaminated images can be appropriately recovered by the proposed method, with high determination coefficients (>0.8906) and low root-mean-square errors (<2.4711) and spectral angle maps (<0.8870). In summary, the proposed method can completely remove thin clouds, faithfully recover ground surface information, and effectively work for visible remote sensing images from diverse sensors. Furthermore, the factors potentially affecting correction performance and the applicability to open-source datasets were investigated.

Wavelet Integrated Convolutional Neural Network for Thin Cloud Removal in Remote Sensing Images

Cloud occlusion phenomena are widespread in optical remote sensing (RS) images, leading to information loss and image degradation and causing difficulties in subsequent applications such as land surface classification, object detection, and land change monitoring. Therefore, thin cloud removal is a key preprocessing procedure for optical RS images, and has great practical value. Recent deep learning-based thin cloud removal methods have achieved excellent results. However, these methods have a common problem in that they cannot obtain large receptive fields while preserving image detail. In this paper, we propose a novel wavelet-integrated convolutional neural network for thin cloud removal (WaveCNN-CR) in RS images that can obtain larger receptive fields without any information loss. WaveCNN-CR generates cloud-free images in an end-to-end manner based on an encoder–decoder-like architecture. In the encoding stage, WaveCNN-CR first extracts multi-scale and multi-frequency components via wavelet transform, then further performs feature extraction for each high-frequency component at different scales by multiple enhanced feature extraction modules (EFEM) separately. In the decoding stage, WaveCNN-CR recursively concatenates the processed low-frequency and high-frequency components at each scale, feeds them into EFEMs for feature extraction, then reconstructs the high-resolution low-frequency component by inverse wavelet transform. In addition, the designed EFEM consisting of an attentive residual block (ARB) and gated residual block (GRB) is used to emphasize the more informative features. ARB and GRB enhance features from the perspective of global and local context, respectively. Extensive experiments on the T-CLOUD, RICE1, and WHUS2-CR datasets demonstrate that our WaveCNN-CR significantly outperforms existing state-of-the-art methods.

Feedback Network for Compact Thin Cloud Removal

The thin cloud removal (CR) technique has great practical value for the application of remote sensing images. Existing deep-learning-based methods have attained remarkable achievements. However, most of them neglect the inherent feature correlations in deeper layers due to learning in a successive manner. In this letter, we propose a compact thin cloud removal network based on the feedback (FB) mechanism, called CRFB-Net, which leverages the high-level features as feedback information to modulate shallow representations. CRFB-Net employs the recurrent architecture to achieve such a feedback scheme. Specifically, the restoration process does not terminate after obtaining an output. In this case, the output of intermediate iterations will flow into the next iteration as feedback. For better utilization of feedback, a multiscale feature fusion block (MFFB) is designed to refine the low-level representations from three scales. Furthermore, we introduce a curriculum learning strategy to train the CRFB-Net by gradually increasing the complexity of restoration, through which a sharper result is produced step by step. Extensive experiments demonstrate the superiority of our CRFB-Net, outperforming state-of-the-art.

Semi-supervised thin cloud removal with mutually beneficial guides

We present a semi-supervised thin cloud removal framework referred to as mutually beneficial guides. It is featured by two fundamental operations, i.e., (a) beneficial model guide and (b) beneficial objective guide. They are operated in terms of the iterative training of two benefactor networks, i.e., (a) a model benefactor network and (b) an objective benefactor network. The beneficial objective guide benefits the model benefactor network by guiding its training with the cloud removal results of cloudy images given by the objective benefactor network at each iteration. This operation explores the cloudy images whose cloud-free images are unavailable for training, and thus augments the normally insufficient training data for cloud removal in a semi-supervised fashion. The beneficial model guide benefits the objective benefactor network by transferring a historically optimal model benefactor network to it at each iteration. This operation enables the objective benefactor network to provide reliable cloud removal results for training the model benefactor network. Therefore, the two benefactor networks mutually benefit each other from different aspects during the iterative training such that their capabilities of removing clouds are gradually enhanced. Specifically, we construct the benefactor networks in terms of a cross-scale content fusion structure where multi-scale features are fused to the pixel-to-pixel residual processing blocks. The structure is in favor of both coherent and high-resolution cloud removal results. Experiments on both Landsat-8 and Sentinel-2A images validate that the framework of mutually beneficial guides outperforms other methods in thin cloud removal. The implementation code of the framework of mutually beneficial guides is available at: https://gitee.com/xuzunxiao/mutually-beneficial-guides.

A spatial-spectral adaptive thin-cloud removal method based on slow feature analysis

ABSTRACT Clouds cover corrupts the spatial and spectral information of optical remote sensing (RS) images, which seriously affects the use of RS data. To solve the problem of missing information, a spatial-spectral adaptive method based on slow feature analysis (SFA) is proposed to restore cloudy scenes in this letter. SFA converts the sequence signal into slowly varying signal signatures and the clouds will be located in the first component. We propose spatial and spectral adaptive correction methods to reduce interference from highlighted pixels and jointly constrain the cloud coefficients in each band according to reflectance and gradient. The effectiveness of our method is verified on Landsat-8 OLI simulated and real cloudy datas, the restored results are visually rich in texture detail and moderately corrected. The average PSNR of the four real scenes is 42.9738 dB and coefficient of determination (R 2) is 0.8203, and many indicators have proved that our method is better than the existing methods.

Cloudy Image Arithmetic: A Cloudy Scene Synthesis Paradigm With an Application to Deep-Learning-Based Thin Cloud Removal

We present a cloudy scene synthesis paradigm that produces cloudy images for arbitrary optical cloud-free observations. The synthesis paradigm consists of two fundamental operations, i.e., 1) cloud self-subtraction and 2) cloud addition-to-scene. Cloud self-subtraction extracts cloud ingredient images from cloudy images of weak texture regions (typically sea areas). The cloud ingredient images exhibit clouds in more realistic forms than simulated clouds. Cloud addition-to-scene incorporates the cloud ingredient images into arbitrary cloud-free land images, synthesizing cloudy scenes. It provides a means of constructing data pairs of cloud-free scene images and cloudy scene images, which are highly needed but considerably insufficient in the remote sensing literature. We refer to the overall paradigm consisting of the two fundamental operations as cloudy image arithmetic. We explore the use of the cloudy image arithmetic for the purpose of thin cloud removal. To this end, we develop a multi scale generative adversarial net (MSGAN) that removes thin clouds from cloudy scenes. We use the cloudy image arithmetic to construct a comprehensive training dataset for the MSGAN. Experimental evaluations validate that the cloudy image arithmetic synthesizes good cloudy scenes and the MSGAN with aid of the cloudy image arithmetic gives effective results in thin cloud removal.

An Effective Network Integrating Residual Learning and Channel Attention Mechanism for Thin Cloud Removal

Cloud-contaminated images seriously disturb the effective information in ground observations and reduce the availability of remote sensing images. This letter presents a thin cloud removal method for cloud-contaminated images based on a residual channel attention network. Compared with the present thin cloud removal methods not showing different attention to cloud components and ground details, the proposed method introduces channel attention mechanism into the residual learning (RL) path, which achieves suppressing thin clouds and enhancing the details of ground scenes simultaneously. Based on this, both residual channel attention module (RCAM) and residual group (RG) are constructed, playing the role of basic modules to stack an encoder–decoder-based residual channel attention network, which effectively avoids the loss of ground information in the deep layers. Compared with recent state-of-the-art (SOTA) methods, experimental results on both actual and synthesized cloudy images show the proposed method’s superiority in reconstructing rich details of ground scenes.

Thin Cloud Removal for Remote Sensing Images Using a Physical-Model-Based CycleGAN With Unpaired Data

Thin cloud removal from remote sensing (RS) images is challenging. Recently, deep-learning-based methods have achieved excellent results using supervised training on paired image data. However, in practice, real paired image data are unavailable. Therefore, in this letter, we propose a novel thin cloud removal method, a physical-model-based CycleGAN (PM-CycleGAN), which can be trained using only unpaired data. The PM-CycleGAN training process comprises forward and backward loops. The forward loop first decomposes a cloudy image into a cloud-free image, thin cloud thickness map, and thickness coefficient using three generators. Then, it combines these three components using a physical model to reconstruct the original cloudy image to obtain the cycle consistency constraint. The backward loop first uses the physical model to synthesize a cloud-free image, thin cloud thickness map, and thickness coefficient into a cloudy image, which are then decomposed into the original three components using the three generators. Visual and quantitative comparisons against several state-of-the-art (SOTA) methods on a cloudy image dataset demonstrated the superiority of PM-CycleGAN.

Multimodel Fusion Method for Cloud Detection in Satellite Laser Footprint Images

Laser footprint images (LFIs) are auxiliary sensors of satellite laser altimeters, which are mainly used to determine whether the laser pulse is obscured by clouds, and establish a link between the laser and the high-resolution image, which requires an algorithm for achieving high recognition accuracy while preserving the edge contour features of clouds. This study uses the AdaBoost algorithm to achieve the fusion of multiple semantic segmentation models and combines the dark channel priori model to enhance the overall cloud detection effect, realize the removal of thin clouds, and improve the matching accuracy of footprint and altimeter images. The experimental results show that the model outperforms the basic cloud detection model by approximately 14.89% and that the contours of the extracted cloud-covered area are more consistent with subjective human perceptions. Furthermore, the matching accuracy of the optimized footprint and high-resolution altimeter images improved by approximately 17.67%.