Cloud Detection Method Research Articles

Cloud and snow detection of remote sensing images based on improved Unet3+

Cloud detection is an important step in remote sensing image processing and a prerequisite for subsequent analysis and interpretation of remote sensing images. Traditional cloud detection methods are difficult to accurately detect clouds and snow with very similar features such as color and texture. In this paper, the features of cloud and snow in remote sensing images are deeply extracted, and an accurate cloud and snow detection method is proposed based on the advantages of Unet3+ network in feature fusion. Firstly, color space conversion is performed on remote sensing images, RGB images and HIS images are used as input of Unet3+ network. Resnet 50 is used to replace the Unet3+ feature extraction network to extract remote sensing image features at a deeper level, and add the Convolutional Block Attention Module in Resnet50 to improve the network’s attention to cloud and snow. Finally, the weighted cross entropy loss is constructed to solve the problem of unbalanced sample number caused by high proportion of background area in the image. The results show that the proposed method has strong adaptability and moderate computation. The mPA value, mIoU value and mPrecision value can reach 92.76%, 81.74% and 86.49%, respectively. Compared with other algorithms, the proposed method can better eliminate all kinds of interference information in remote sensing images of different landforms and accurately detect cloud and snow in images.

A hybrid generative adversarial network for weakly-supervised cloud detection in multispectral images

Cloud detection is a crucial step in the optical satellite image processing pipeline for Earth observation. Clouds in optical remote sensing images seriously affect the visibility of the background and greatly reduce the usability of images for land applications. Traditional methods based on thresholding, multi-temporal or multi-spectral information are often specific to a particular satellite sensor. Convolutional Neural Networks for cloud detection often require labeled cloud masks for training that are very time-consuming and expensive to obtain. To overcome these challenges, this paper presents a hybrid cloud detection method based on the synergistic combination of generative adversarial networks (GAN) and a physics-based cloud distortion model (CDM). The proposed weakly-supervised GAN-CDM method (available online https://github.com/Neooolee/GANCDM) only requires patch-level labels for training, and can produce cloud masks at pixel-level in both training and testing stages. GAN-CDM is trained on a new globally distributed Landsat 8 dataset (WHUL8-CDb, available online doi:https://doi.org/10.5281/zenodo.6420027) including image blocks and corresponding block-level labels. Experimental results show that the proposed GAN-CDM method trained on Landsat 8 image blocks achieves much higher cloud detection accuracy than baseline deep learning-based methods, not only in Landsat 8 images (L8 Biome dataset, 90.20% versus 72.09%) but also in Sentinel-2 images (“S2 Cloud Mask Catalogue” dataset, 92.54% versus 77.00%). This suggests that the proposed method provides accurate cloud detection in Landsat images, has good transferability to Sentinel-2 images, and can quickly be adapted for different optical satellite sensors.

Cloud Detection Method Based on All-Sky Polarization Imaging.

Sky cloud detection has a significant application value in the meteorological field. The existing cloud detection methods mainly rely on the color difference between the sky background and the cloud layer in the sky image and are not reliable due to the variable and irregular characteristics of the cloud layer and different weather conditions. This paper proposes a cloud detection method based on all-sky polarization imaging. The core of the algorithm is the “normalized polarization degree difference index” (NPDDI). Instead of relying on the color difference information, this index identifies the difference between degree of polarization (DoPs) of the cloud sky and the clear sky radiation to achieve cloud recognition. The method is not only fast and straightforward in the algorithm, but also can detect the optical thickness of the cloud layer in a qualitative sense. The experimental results show a good cloud detection performance.

MFCD-Net: Cross Attention Based Multimodal Fusion Network for DPC Imagery Cloud Detection

As one kind of remote sensing image (RSI), Directional Polarimetric Camera (DPC) data are of great significance in atmospheric radiation transfer and climate feedback. The availability of DPC images is often hindered by clouds, and effective cloud detection is the premise of many applications. Conventional threshold-based cloud detection methods are limited in performance and generalization capability. In this paper, we propose an effective learning-based 3D multimodal fusion cloud detection network (MFCD-Net) model. The network is a three-input stream architecture with a 3D-Unet-like encoder-decoder structure to fuse the multiple modalities of reflectance image, polarization image Q, and polarization image U in DPC imagery, with consideration of the angle and spectral information. Furthermore, cross attention is utilized in fusing the polarization features into the spatial-angle-spectral features in the reflectance image to enhance the expression of the fused features. The dataset used in this paper is obtained from the DPC cloud product and the cloud mask product. The proposed MFCD-Net achieved excellent cloud detection performance, with a recognition accuracy of 95.74%, according to the results of the experiments.

CD_HIEFNet: Cloud Detection Network Using Haze Optimized Transformation Index and Edge Feature for Optical Remote Sensing Imagery

Clouds in optical remote sensing images are an unavoidable existence that greatly affect the utilization of these images. Therefore, accurate and effective cloud detection is an indispensable step in image preprocessing. To date, most researchers have tried to use deep-learning methods for cloud detection. However, these studies generally use computer vision technology to improve the performances of the models, without considering the unique spectral feature information in remote sensing images. Moreover, due to the complex and changeable shapes of clouds, accurate cloud-edge detection is also a difficult problem. In order to solve these problems, we propose a deep-learning cloud detection network that uses the haze-optimized transformation (HOT) index and the edge feature extraction module for optical remote sensing images (CD_HIEFNet). In our model, the HOT index feature image is used to add the unique spectral feature information from clouds into the network for accurate detection, and the edge feature extraction (EFE) module is employed to refine cloud edges. In addition, we use ConvNeXt as the backbone network, and we improved the decoder to enhance the details of the detection results. We validated CD_HIEFNet using the Landsat-8 (L8) Biome dataset and compared it with the Fmask, FCN8s, U-Net, SegNet, DeepLabv3+ and CloudNet methods. The experimental results showed that our model has excellent performance, even in complex cloud scenarios. Moreover, according to the extended experimental results for the other L8 dataset and the Gaofen-1 data, CD_HIEFNet has strong performance in terms of robustness and generalization, thus helping to provide new ideas for cloud detection-related work.

Cloudformer V2: Set Prior Prediction and Binary Mask Weighted Network for Cloud Detection

Cloud detection is an essential step in optical remote sensing data processing. With the development of deep learning technology, cloud detection methods have made remarkable progress. Among them, researchers have started to try to introduce Transformer into cloud detection tasks due to its excellent performance in image semantic segmentation tasks. However, the current Transformer-based methods suffer from training difficulty and low detection accuracy of small clouds. To solve these problems, this paper proposes Cloudformer V2 based on the previously proposed Cloudformer. For the training difficulty, Cloudformer V2 uses Set Attention Block to extract intermediate features as Set Prior Prediction to participate in supervision, which enables the model to converge faster. For the detection of small clouds, Cloudformer V2 decodes the features by a multi-scale Transformer decoder, which uses multi-resolution features to improve the modeling accuracy. In addition, a binary mask weighted loss function (BW Loss) is designed to construct weights by counting pixels classified as clouds; thus, guiding the network to focus on features of small clouds and improving the overall detection accuracy. Cloudformer V2 is experimented on the dataset from GF-1 satellite and has excellent performance.

DETECTION OF CLOUDS IN MEDIUM-RESOLUTION SATELLITE IMAGERY USING DEEP CONVOLUTIONAL NEURAL NETS

Abstract. Cloud detection is an inextricable pre-processing step in remote sensing image analysis workflows. Most of the traditional rule-based and machine-learning-based algorithms utilize low-level features of the clouds and classify individual cloud pixels based on their spectral signatures. Cloud detection using such approaches can be challenging due to a multitude of factors including harsh lighting conditions, the presence of thin clouds, the context of surrounding pixels, and complex spatial patterns. In recent studies, deep convolutional neural networks (CNNs) have shown outstanding results in the computer vision domain. These methods are practiced for better capturing the texture, shape as well as context of images. In this study, we propose a deep learning CNN approach to detect cloud pixels from medium-resolution satellite imagery. The proposed CNN accounts for both the low-level features, such as color and texture information as well as high-level features extracted from successive convolutions of the input image. We prepared a cloud-pixel dataset of approximately 7273 randomly sampled 320 by 320 pixels image patches taken from a total of 121 Landsat-8 (30m) and Sentinel-2 (20m) image scenes. These satellite images come with cloud masks. From the available data channels, only blue, green, red, and NIR bands are fed into the model. The CNN model was trained on 5300 image patches and validated on 1973 independent image patches. As the final output from our model, we extract a binary mask of cloud pixels and non-cloud pixels. The results are benchmarked against established cloud detection methods using standard accuracy metrics.

Fusion Information Multi-View Classification Method for Remote Sensing Cloud Detection

In recent years, many studies have been carried out to detect clouds on remote sensing images. Due to the complex terrain, the variety of clouds, the density, and content of clouds are various, and the current model has difficulty accurately detecting the cloud in the image. In our strategy, a multi-view data training set based on super pixel is constructed. View A uses multi-level network to extract the boundary, texture, and deep abstract feature of super pixels. View B is the statistical feature of the three channels of the image. Privilege information View P contains the cloud content of super pixels and the tag status of adjacent super pixels. Finally, we propose a cloud detection method for remote sensing image classification based on multi-view support vector machine (SVM). The proposed method is tested on images of different terrain and cloud distribution in GF-1_WHU and Cloud-38 remote sensing datasets. Visual performance and quantitative analysis show that the method has excellent cloud detection performance.

Multi-feature embedded learning SVM for cloud detection in remote sensing images

To improve remote sensing image transmission efficiency, we propose a cloud detection method using a multi-feature embedded learning support vector machine (SVM) to address cloud coverage occupying channel transmission bandwidth. Specifically, we first consider the imaging and physical properties of the clouds to construct a multi-feature space of cloud and non-cloud samples, which mainly includes five valuable features of grayscale, geometry, contrast, correlation, and angular second moment. Subsequently, we regard cloud detection (CDRSI) of remote sensing images as a binary classification problem, and construct a classifier by using multi-feature embedded learning SVM. Finally, the CDRSI is implemented by image block operations. Additionally, we build a large-scale real-world Remote Sensing Image Cloud Detection Benchmark (RSICDB) including 1520 images, where 790 non-cloud images and 430 cloud images are used as training datasets, 150 of which as test samples with the corresponding 150 mask results. Experimental results demonstrate that the proposed method can detect clouds with higher accuracy and robustness than compared methods.

Hybrid Cloud Detection Algorithm Based on Intelligent Scene Recognition

Abstract All-sky images derived from ground-based imaging equipment have become an important means of recognizing and quantifying cloud information. Accurate cloud detection is a prerequisite for obtaining important cloud information from an all-sky image. Existing cloud segmentation algorithms can achieve high accuracy. However, for different scenes, such as completely cloudy with obscured sun and partly cloudy with unobscured sun, the use of specific algorithms can further improve segmentation. In this study, a hybrid cloud detection algorithm based on intelligent scene recognition (HCD-ISR) is proposed. It uses suitable cloud segmentation algorithms for images in different scenes recognized by ISR, so as to utilize the various algorithms to their full potential. First, we developed an ISR method to automatically classify the all-sky images into three scenes. In scene A, the sky is completely clear; in scene B, the sky is partly cloudy with unobscured sun; and in scene C, the sun is completely obscured by clouds. The experimental results show that the ISR method can correctly identify 93% of the images. The most suitable cloud detection algorithm was selected for each scene based on the relevant features of the images in that scene. A fixed thresholding (FT) method was used for the images in scene C. For the most complicated scene, that is, scene B, the clear-sky background difference (CSBD) method was used to identify cloud pixels based on a clear-sky library (CSL). The images in the CSL were automatically filtered by ISR. Compared to FT, adaptive thresholding (AT), and CSBD methods, the proposed HCD-ISR method has the highest accuracy (95.62%). The quantitative evaluation and visualization results show that the proposed HCD-ISR algorithm makes full use of the advantages of different cloud detection methods, and is more flexible and robust.

Cloud detection for satellite cloud images based on fused FCN features

ABSTRACT Cloud detection for satellite cloud images is a challenging image processing task owing to the blurring of cloud boundaries, multiplicity, and complexity of cloud types. Currently, the commonly used cloud detection methods include original full convolutional neural network (original FCN), FCN with an 8-pixel stride (FCN- 8s), FCN with a 2-pixel stride (FCN-2s), and so on. However, the aforementioned methods exclusively rely on a single network layer the final layer feature map; thus, shallow cloud image information, such as cloud profile information may not be captured. In this letter, a cloud detection method for satellite cloud images based on fused FCN features is proposed. The proposed method effectively fuses spatial and high-level semantic information, and a voting ensemble strategy is used to improve the accuracy and robustness of cloud detection. Finally, the experimental results demonstrate that the average overall accuracy (OA), average producer’ accuracy (PA), and average user’ accuracy (UA) of the proposed method for multiple training sample sizes and image sizes of the collected Fengyun satellite (FY-2 G) cloud image database increased by 7.15%, 9.04%, and 8.46%, respectively, relative to the average accuracies of the original FCN, FCN-8s, FCN-2s, SegNet, and DeepLabV3 methods.

ECDNet: A bilateral lightweight cloud detection network for remote sensing images

Cloud detection is one of the critical tasks in remote sensing image pre-processing and it has attracted extensive research interest. In recent years, deep neural networks based cloud detection methods have surpassed the traditional methods (threshold-based methods and conventional machine learning-based methods). However, current approaches mainly focus on improving detection accuracy. The computation complexity and large model size are ignored. To tackle this problem, we propose a lightweight deep learning cloud detection model: Efficient Cloud Detection Network (ECDNet). This model is based on the encoder-decoder structure. In the encoder, a two-path architecture is proposed to extract the spatial and semantic information concurrently. One pathway is the detail branch. It is designed to capture low-level detail spatial features with only a few parameters. The other pathway is the semantic branch, which is mainly for capturing context features. In the semantic branch, a proposed dense pyramid module (DPM) is designed for multi-scale contextual information extraction. The number of parameters and calculations in DPM is greatly reduced by features reusing. Besides, a FusionBlock is developed to merge these two kinds of information. Then the extreme lightweight decoder recovers the cloud mask to the same scale as the input image step by step. To improve performance, boost loss is introduced without inference cost increment. We evaluate the proposed method on two public datasets: LandSat8 and MODIS. Extensive experiments demonstrate that the proposed ECDNet achieves comparable accuracy as the state-of-art cloud detection methods, and meantime has a much smaller model size and less computation burden.

Ten-minute prediction of solar irradiance based on cloud detection and a long short-term memory (LSTM) model

With the inherent volatility of solar radiation, accurate short-term prediction of solar irradiance is essential to cost-efficiently stabilise and operate power grids. Cloud is the major influencing factor for variations in solar irradiance at a minute scale. This study proposes a short-term solar irradiance prediction model based on a deep learning network. Firstly, a new hybrid cloud detection method is proposed to calculate cloud coverage under different sky condition. The relationship between cloud coverage at the current moment and the global horizontal irradiance (GHI) 10 min later is directly analysed to avoid the error of assuming cloud motion as rigid motion. Meteorological parameters with strong correlation with the GHI in ten minutes are admitted into the model input, mainly including relative humidity and Aerosol Optical Depth (AOD) of 500 nm in this study. The GHI of two-time intervals ahead are also selected after autocorrelation analysis to represent other comprehensive impacts. Together with cloud, the meteoritical parameters and the historical GHI are taken as the input of a deep learning network simultaneously, which is based on long short-term memory network (LSTM) and optimised by Bayesian Optimisation (BO). The solar irradiance after ten minutes is the output of the model. Experiments were conducted to evaluate the performance of the proposed forecast model comparing with some benchmark models. Results showed that the proposed model outperformed other models with a normalised root mean square error (nRMSE) of 15.25% under all sky conditions. An improvement of 8.23% was reached compared with persistent model.

A Maritime Cloud-Detection Method Using Visible and Near-Infrared Bands over the Yellow Sea and Bohai Sea

Accurate cloud-masking procedures to distinguish cloud-free pixels from cloudy pixels are essential for optical satellite remote sensing. Many studies on satellite-based cloud-detection have been performed using the spectral characteristics of clouds in terms of reflectance and temperature. This study proposes a cloud-detection method using reflectance in four bands: 0.56 μm, 0.86 μm, 1.38 μm, and 1.61 μm. Methodologically, we present a conversion relationship between the normalized difference water index (NDWI) and the green band in the visible spectrum for thick cloud detection using moderate-resolution imaging spectroradiometer (MODIS) observations. NDWI consists of reflectance at the 0.56 and 0.86 μm bands. For thin cloud detection, the 1.38 and 1.61 μm bands were applied with empirically determined threshold values. Case study analyses for the four seasons from 2000 to 2019 were performed for the sea surface area of the Yellow Sea and Bohai Sea. In the case studies, the comparison of the proposed cloud-detection method with the MODIS cloud mask (CM) and Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation data indicated a probability of detection of 0.933, a false-alarm ratio of 0.086, and a Heidke Skill Score of 0.753. Our method demonstrated an additional important benefit in distinguishing clouds from sea ice or yellow dust, compared to the MODIS CM products, which usually misidentify the latter as clouds. Consequently, our cloud-detection method could be applied to a variety of low-orbit and geostationary satellites with 0.56, 0.86, 1.38, and 1.61 μm bands.

Comparative Analysis of Several Typical Landsat 8 OLI Cloud Detection Methods

The Landsat 8 Operational Land Imager (OLI) is a high-resolution satellite sensor that is carried on the next-generation Landsat which was launched by the National Aeronautics and Space Administration (NASA) in 2013. Compared to the previous land observation satellites in this series, Landsat 8 OLI has been further optimized in regard to its band setting and data acquisition frequency. The application scope of the data and the ability of information extraction have been further expanded and enhanced. However, the existence of clouds reduces the efficiency and the quality of satellite data. Therefore, in order to identify cloud pixels with a high level of accuracy and efficiency, a variety of cloud detection methods have been developed for Landsat 8 OLI sensors. Distinct treatment methods have been adopted for identifying different surface backgrounds and various types of cloud characteristics in order to improve the accuracy of cloud detection. The present study compares and analyzes the detection results from five typical cloud detection algorithms for different cloud types over different ground types. In addition, it discusses the various cloud detection algorithms used for different types of adaptive backgrounds and clouds in order to shed light upon the comprehensive application of different methods. Subsequently, these findings may provide a reference for the development of an algorithm with a higher precision of cloud detection.

Cloud Detection in ZY-3 Multi-Angle Remote Sensing Images

Cloud pollution on remote sensing images seriously affects the actual use rate of remote sensing images. Therefore, cloud detection of remote sensing images is an indispensable part of image preprocessing and image availability screening. Aiming at the lack of short wave infrared and thermal infrared bands in ZY-3 high-resolution satellite images resulting in the poor detection effect, considering the obvious difference in geographic height between cloud and ground surface objects, this paper proposes a thick and thin cloud detection method combining spectral information and digital height model (DHM) based on multi-scale features-convolutional neural network (MF-CNN) model. To verify the importance of DHM height information in cloud detection of ZY-3 multi-angle remote sensing images, this paper implements cloud detection comparison of the data set with and without DHM height information based on the MF-CNN model. The experimental results show that the ZY-3 multi-angle image with DHM height information can effectively improve the confusion between highlighted surface and thin cloud, which also means the assistance of DHM height information can make up for the disadvantage of high-resolution image lacking short wave infrared and thermal infrared bands.

Optical remote sensing cloud detection based on random forest only using the visible light and near-infrared image bands

ABSTRACT Cloud detection is of great significance for optical remote sensing images. Most existing cloud detection approaches often rely on many thresholds among multiple bands with a wide spectrum range, and normally just can be applied to specific satellite data. Cloud detection is difficult in some sensor data with limited number of available spectral bands. To tackle this challenge, we propose a cloud detection method based on random forest (RFCD) only with the most common RGB and NIR bands. The RFCD normalizes different sensor data by calculating the top-of-atmosphere (TOA) reflectance and combines stable spectral and texture features extracted from the most common bands, which improves accuracy, reduces reliance on multi-band information, and improves the extendibility of the method. Moreover, the RFCD effectively avoids the complicated threshold setting and reduces subjective factors. Few parameters and strong generalization ability of RF further improve the extendibility of RFCD and provide the application possibility in a variety of data. Experimental results show the total validity obtained by 107 Landsat-8 images is 93.46%. The accuracy of the RFCD is higher than the Function of mask method and similar to the SegNet deep learning method, while the RFCD need fewer training samples and hyperparameters which makes it easier to be used. The extended experimental results show RFCD also gets good detection results in Sentinel-2 and GaoFen-1 images. The new RFCD is an accurate cloud detection method with strong extendibility and stability, also avoiding the complicated threshold setting.

LCDNet: Light-Weighted Cloud Detection Network for High-Resolution Remote Sensing Images

Cloud detection is one of the important links in high-resolution remote sensing image processing. Cloud detection methods can be mainly divided into three categories: threshold methods, clustering methods based on machine learning, and deep learning methods. The traditional threshold method needs cumbersome manual calibration, which has high cost and poor universality. The generalization of clustering based method is very poor. In addition, existing deep learning methods tend to have many model parameters and high training costs. To solve the above problems, a light-weighted cloud detection network (LCDNet) based on deep learning method is proposed. The network can complete the task of high-precision segmentation with less parameters and computation. Its light-weighted bottleneck (LB) layer can quickly capture the multi-scale feature information in the image and segment the cloud with high efficiency. The gated channel excitation (GCE) module can effectively reduce the feature redundancy in the network, so as to highlight the details of the cloud layer and improve the detection accuracy. The function of light-weighted self attention (LSA) module is to quickly establish the spatial location information in the feature map, so as to locate the unpredictable targets and reduce the false detection rate. The experimental results show that the model shows excellent performance on cloud and cloud shadow datasets with 353.76 k parameters and 456.28Mmac computation. In addition, the training results on GF1_WHU and L8 Biome datasets further show that our model has excellent generalization performance, which is of great significance for the efficient implementation of cloud detection.

Robust Thick Cloud Removal for Multitemporal Remote Sensing Images Using Coupled Tensor Factorization

The existing nonblind cloud and cloud shadow (cloud/shadow) removal methods for remote sensing (RS) images are based on the assumption that cloud/shadow masks are accurately given. Since the masks are usually manually labeled or detected by cloud detection methods, whose accuracy cannot be well guaranteed, the cloud/shadow removal effect may be affected. In this article, we suggest a robust thick cloud/shadow removal (RTCR) method that meets the problem with an inaccurate mask. To faithfully reconstruct the multitemporal information, a coupled tensor factorization is used to explore the relationship between the abundances of the multitemporal images in the same scene. Moreover, an efficient algorithm is developed to solve the proposed model based on the augmented Lagrange multiplier method. The experimental results under accurate masks and inaccurate masks demonstrate its robustness and superiority for thick cloud/shadow removal.

A Lightweight Deep Learning-Based Cloud Detection Method for Sentinel-2A Imagery Fusing Multiscale Spectral and Spatial Features

Clouds are a very important factor in the availability of optical remote sensing images. Recently, deep learning-based cloud detection methods have surpassed classical methods based on rules and physical models of clouds. However, most of these deep models are very large which limits their applicability and explainability, while other models do not make use of the full spectral information in multi-spectral images such as Sentinel-2. In this paper, we propose a lightweight network for cloud detection, fusing multi-scale spectral and spatial features (CDFM3SF) and tailored for processing all spectral bands in Sentinel- 2A images. The proposed method consists of an encoder and a decoder. In the encoder, three input branches are designed to handle spectral bands at their native resolution and extract multiscale spectral features. Three novel components are designed: a mixed depth-wise separable convolution (MDSC) and a shared and dilated residual block (SDRB) to extract multi-scale spatial features, and a concatenation and sum (CS) operation to fuse multi-scale spectral and spatial features with little calculation and no additional parameters. The decoder of CD-FM3SF outputs three cloud masks at the same resolution as input bands to enhance the supervision information of small, middle and large clouds. To validate the performance of the proposed method, we manually labeled 36 Sentinel-2A scenes evenly distributed over mainland China. The experiment results demonstrate that CD-FM3SF outperforms traditional cloud detection methods and state-of-theart deep learning-based methods in both accuracy and speed.