Cloud Detection Method Research Articles

Benchmarking Deep Learning Models for Cloud Detection in Landsat-8 and Sentinel-2 Images

The systematic monitoring of the Earth using optical satellites is limited by the presence of clouds. Accurately detecting these clouds is necessary to exploit satellite image archives in remote sensing applications. Despite many developments, cloud detection remains an unsolved problem with room for improvement, especially over bright surfaces and thin clouds. Recently, advances in cloud masking using deep learning have shown significant boosts in cloud detection accuracy. However, these works are validated in heterogeneous manners, and the comparison with operational threshold-based schemes is not consistent among many of them. In this work, we systematically compare deep learning models trained on Landsat-8 images on different Landsat-8 and Sentinel-2 publicly available datasets. Overall, we show that deep learning models exhibit a high detection accuracy when trained and tested on independent images from the same Landsat-8 dataset (intra-dataset validation), outperforming operational algorithms. However, the performance of deep learning models is similar to operational threshold-based ones when they are tested on different datasets of Landsat-8 images (inter-dataset validation) or datasets from a different sensor with similar radiometric characteristics such as Sentinel-2 (cross-sensor validation). The results suggest that (i) the development of cloud detection methods for new satellites can be based on deep learning models trained on data from similar sensors and (ii) there is a strong dependence of deep learning models on the dataset used for training and testing, which highlights the necessity of standardized datasets and procedures for benchmarking cloud detection models in the future.

A geographic information-driven method and a new large scale dataset for remote sensing cloud/snow detection

Geographic information such as the altitude, latitude, and longitude are common but fundamental meta-records in remote sensing image products. In this paper, it is shown that such a group of records provides important priors for cloud and snow detection in remote sensing imagery. The intuition comes from some common geographical knowledge, where many of them are important but are often overlooked. For example, it is generally known that snow is less likely to exist in low-latitude or low-altitude areas, and clouds in different geographic may have various visual appearances. Previous cloud and snow detection methods simply ignore the use of such information, and perform detection solely based on the image data (band reflectance). Due to the neglect of such priors, most of these methods are difficult to obtain satisfactory performance in complex scenarios (e.g., cloud-snow coexistence). In this paper, a novel neural network called “Geographic Information-driven Network (GeoInfoNet)” is proposed for cloud and snow detection. In addition to the use of the image data, the model integrates the geographic information at both training and detection phases. A “geographic information encoder” is specially designed, which encodes the altitude, latitude, and longitude of imagery to a set of auxiliary maps and then feeds them to the detection network. The proposed network can be trained in an end-to-end fashion with dense robust features extracted and fused. A new dataset called “Levir_CS” for cloud and snow detection is built, which contains 4,168 Gaofen-1 satellite images and corresponding geographical records, and is over 20× larger than other datasets in this field. On “Levir_CS”, experiments show that the method achieves 90.74% intersection over union of cloud and 78.26% intersection over union of snow. It outperforms other state of the art cloud and snow detection methods with a large margin. Feature visualizations also show that the method learns some important priors which is close to the common sense. The proposed dataset and the code of GeoInfoNet are available in https://github.com/permanentCH5/GeoInfoNet.

MCNet: Multi-Scale Feature Extraction and Content-Aware Reassembly Cloud Detection Model for Remote Sensing Images

Cloud detection plays a vital role in remote sensing data preprocessing. Traditional cloud detection algorithms have difficulties in feature extraction and thus produce a poor detection result when processing remote sensing images with uneven cloud distribution and complex surface background. To achieve better detection results, a cloud detection method with multi-scale feature extraction and content-aware reassembly network (MCNet) is proposed. Using pyramid convolution and channel attention mechanisms to enhance the model’s feature extraction capability, MCNet can fully extract the spatial information and channel information of clouds in an image. The content-aware reassembly is used to ensure that sampling on the network can recover enough in-depth semantic information and improve the model cloud detection effect. The experimental results show that the proposed MCNet model has achieved good detection results in cloud detection tasks.

Impact of the One-Stream Cloud Detection Method on the Assimilation of AMSU-A Data in GRAPES

Clouds affect the assimilation of microwave data from satellites and therefore the detection of clouds is important under both clear sky and cloudy conditions. We introduce a new cloud detection method based on the assimilation of data from the advanced microwave sounder unit A (AMSU-A) and the microwave humidity sounder (MHS) into the global and regional assimilation and prediction system (GRAPES) and use forecast experiments to evaluate its impact. The new cloud detection method can retain more observational data than the current method in GRAPES, thereby improving the assimilation of AMSU-A data. Verification of the method showed that, by improving the forecast of the lower-level air temperature and geopotential height, the new cloud detection method improved the forecast of the track of two typhoons. The forecast of a large-scale weather system in GRAPES was also improved by the new method in the later period of the forecast.

Cloud detection in Landsat-8 imagery in Google Earth Engine based on a deep convolutional neural network

ABSTRACT Google Earth Engine (GEE) provides a convenient platform for applications based on optical satellite imagery of large areas. With such data sets, the detection of cloud is often a necessary prerequisite step. Recently, deep learning-based cloud detection methods have shown their potential for cloud detection but they can only be applied locally, leading to inefficient data downloading time and storage problems. This letter proposes a method to directly perform cloud detection in Landsat-8 imagery in GEE based on deep learning (DeepGEE-CD). A deep convolutional neural network (DCNN) was first trained locally, and then the trained DCNN was deployed in the JavaScript client of GEE. An experiment was undertaken to validate the proposed method with a set of Landsat-8 images and the results show that DeepGEE-CD outperformed the widely used function of mask (Fmask) algorithm. The proposed DeepGEE-CD approach can accurately detect cloud in Landsat-8 imagery without downloading it, making it a promising method for routine cloud detection of Landsat-8 imagery in GEE.

A Cloud Detection Method Using Convolutional Neural Network Based on Gabor Transform and Attention Mechanism with Dark Channel Subnet for Remote Sensing Image

Cloud detection, as a crucial step, has always been a hot topic in the field of optical remote sensing image processing. In this paper, we propose a deep learning cloud detection Network that is based on the Gabor transform and Attention modules with Dark channel subnet (NGAD). This network is based on the encoder-decoder framework. The information on texture is an important feature that is often used in traditional cloud detection methods. The NGAD enhances the attention of the network towards important texture features in the remote sensing images through the proposed Gabor feature extraction module. The channel attention module that is based on the larger scale features and spatial attention module that is based on the dark channel subnet have been introduced in NGAD. The channel attention module highlights the important information in a feature map from the channel dimensions, weakens the useless information, and helps the network to filter this information. A dark channel subnet with spatial attention module has been designed in order to further reduce the influence of the redundant information in the extracted features. By introducing a “dark channel”, the information in the feature map is reconstructed from the spatial dimension. The NGAD is validated while using the Gaofen-1 WFV imagery in four spectral bands. The experimental results show that the overall accuracy of NGAD reaches 97.42% and the false alarm rate reaches 2.22%. The efficiency of cloud detection using NGAD exceeds the state-of-art image segmentation network model and remote sensing image cloud detection model.

CloudX-net: A robust encoder-decoder architecture for cloud detection from satellite remote sensing images

Cloud Detection is an important pre-processing step for any application involving remote sensing data. This paper presents a deep learning based CloudX-Net architecture, that can detect cloud cover with improved accuracy in comparison to the benchmark from satellite remote sensing images. The proposed CloudX-Net model reduces the number of parameters needed for accurate predictions and thus make deep learning based cloud detection method very efficient. Atrous Spatial Pyramid Pooling (ASPP) and Separable convolution are used to optimize the network. For experimentation, we have used Landsat 8 images and 38-Cloud dataset and trained the architectures using Soft Jaccard loss function. Comparing several quantifying metrics result from various recent deep learning architectures proves the efficiency and effectiveness of the proposed CloudX-Net model for cloud detection from satellite images. The source code and data are available athttps://github.com/shyamfec/CloudXNet.

Cloud Detection of SuperView-1 Remote Sensing Images Based on Genetic Reinforcement Learning

Cloud pixels have massively reduced the utilization of optical remote sensing images, highlighting the importance of cloud detection. According to the current remote sensing literature, methods such as the threshold method, statistical method and deep learning (DL) have been applied in cloud detection tasks. As some cloud areas are translucent, areas blurred by these clouds still retain some ground feature information, which blurs the spectral or spatial characteristics of these areas, leading to difficulty in accurate detection of cloud areas by existing methods. To solve the problem, this study presents a cloud detection method based on genetic reinforcement learning. Firstly, the factors that directly affect the classification of pixels in remote sensing images are analyzed, and the concept of pixel environmental state (PES) is proposed. Then, PES information and the algorithm’s marking action are integrated into the “PES-action” data set. Subsequently, the rule of “reward–penalty” is introduced and the “PES-action” strategy with the highest cumulative return is learned by a genetic algorithm (GA). Clouds can be detected accurately through the learned “PES-action” strategy. By virtue of the strong adaptability of reinforcement learning (RL) to the environment and the global optimization ability of the GA, cloud regions are detected accurately. In the experiment, multi-spectral remote sensing images of SuperView-1 were collected to build the data set, which was finally accurately detected. The overall accuracy (OA) of the proposed method on the test set reached 97.15%, and satisfactory cloud masks were obtained. Compared with the best DL method disclosed and the random forest (RF) method, the proposed method is superior in precision, recall, false positive rate (FPR) and OA for the detection of clouds. This study aims to improve the detection of cloud regions, providing a reference for researchers interested in cloud detection of remote sensing images.

Detection and characterization of cloud enhancement events for solar irradiance using a model-independent, statistically-driven approach

Cloud enhancement events are instances in which the cloud pattern increases global horizontal irradiance on the ground above levels higher than would otherwise be expected during clear sky with cloudless conditions, and in general, these events are characterized by high irradiances occurring during highly changing transient conditions. Even though it is a well-known although infrequent phenomenon, currently no consensus exists as to its formal definition. However, several studies point out to a need for a better understanding of this phenomenon due to their effects on mathematical models or photovoltaic systems behavior during these instances, as results show significant deviations with respect to the expected results in normal conditions. Due to these issues, the present work intends to explore cloud enhancement definitions and detection methods, for which a novel model-independent, statistically-driven approach is proposed and compared to three other models found in the literature. Assessments are made regarding the main aspects of the phenomenon: frequency of occurrence, clearness index, diffuse fraction and its relationship with solar resource variability. Data from eight measurements sites throughout Chile, with different climatic characteristics is used. To further support the analysis made, monthly maximum expected clear sky irradiance for each site was estimated through satellite-based data as well as with estimation from ground measurements. Finally, the highest irradiances are presented for each measurement site reporting, the record irradiances for one-minute data thermopile and photodiode pyranometers.

Cloud Detection From Paired CrIS Water Vapor and CO₂ Channels Using Machine Learning Techniques

Accurate cloud detection using infrared (IR) data is very challenging due to the limitations and uncertainties from many aspects in the satellite IR remote sensing. This article proposes an end-to-end cloud detection method for the Cross-track IR Sounder (CrIS) using machine learning (ML) techniques. The brightness temperatures from paired CrIS channels in the longwave and midwave water vapor bands and the longwave and shortwave CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> bands are used. After obtaining the linear regression coefficients for each of the selected channel pairs, a complete set of CrIS full spectral resolution (FSR) cloud detection index (FCDI) is derived from the temperature difference between the regression and observation for each channel pair. It is shown that FCDI captures cloud location and structure well by comparing with the cloud products (CPs) from the Visible IR Imaging Radiometer Suite (VIIRS). After collocating FCDI with VIIRS CP, ML techniques such as the extreme learning machine, support vector machine, and multilayer perceptron are used to train the collocated FCDIs for cloud detection. Simulation results show that the accuracy of FCDI cloud detection is slightly above 80%. Moreover, the results encourage the use of water vapor bands in FCDI, in addition to CO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> bands.

An Effective Method for Detecting Clouds in GaoFen-4 Images of Coastal Zones

Cloud-cover information is important for a wide range of scientific studies, such as the studies on water supply, climate change, earth energy budget, etc. In remote sensing, correct detection of clouds plays a crucial role in deriving the physical properties associated with clouds that exert a significant impact on the radiation budget of planet earth. Although the traditional cloud detection methods have generally performed well, these methods were usually developed specifically for particular sensors in a particular region with a particular underlying surface (e.g., land, water, vegetation, and man-made objects). Coastal regions are known to have a variety of underlying surfaces, which represent a major challenge in cloud detection. Therefore, there is an urgent requirement for developing a cloud detection method that could be applied to a variety of sensors, situations, and underlying surfaces. In the present study, a cloud detection method based on spatial and spectral uniformity of clouds was developed. In addition to having a spatially uniform texture, a spectrally approximate value was also present between the blue and green bands of the cloud region. The blue and green channel data appeared more uniform over the cloudy region, i.e., the entropy of the cloudy region was lower than that of the cloud-free region. On the basis of this difference in entropy, it would be possible to categorize the satellite images into cloud region images and cloud-free region images. Furthermore, the performance of the proposed method was validated by applying it to the data from various sensors across the coastal zone of the South China Sea. The experimental results demonstrated that compared to the existing operational algorithms, EN-clustering exhibited higher accuracy and scalability, and also performed robustly regardless of the spatial resolution of the different satellite images. It is concluded that the EN-clustering algorithm proposed in the present study is applicable to different sensors, different underlying surfaces, and different regions, with the support of NDSI and NDBI indices to remove the interference information from snow, ice, and man-made objects.

Lightweight U-Net for cloud detection of visible and thermal infrared remote sensing images

Accurate and rapid cloud detection is exceedingly significant for improving the downlink efficiency of on-orbit data, especially for the microsatellites with limited power and computational ability. However, the inference speed and large model limit the potential of on-orbit implementation of deep-learning-based cloud detection method. In view of the above problems, this paper proposes a lightweight network based on depthwise separable convolutions to reduce the size of model and computational cost of pixel-wise cloud detection methods. The network achieves lightweight end-to-end cloud detection through extracting feature maps from the images to generate the mask with the obtained maps. For the visible and thermal infrared bands of the Landsat 8 cloud cover assessment validation dataset, the experimental results show that the pixel accuracy of the proposed method for cloud detection is higher than 90%, the inference speed is about 5 times faster than that of U-Net, and the model parameters and floating-point operations are reduced to 12.4% and 12.8% of U-Net, respectively.

Accurate cloud detection in high-resolution remote sensing imagery by weakly supervised deep learning

Cloud cover is a common and inevitable phenomenon that often hinders the usability of optical remote sensing (RS) image data and further interferes with continuous cartography based on RS image interpretation. In the literature, the off-the-shelf cloud detection methods either require various hand-crafted features or utilize data-driven features using deep networks. Overall, deep networks achieve much better performance than traditional methods using hand-crafted features. However, the current deep networks used for cloud detection depend on massive pixel-level annotation labels, which require a great deal of manual annotation labor. To reduce the labor needed for annotating the pixel-level labels, this paper proposes a weakly supervised deep learning-based cloud detection (WDCD) method using block-level labels indicating only the presence or the absence of cloud in one RS image block. In the training phase, a new global convolutional pooling (GCP) operation is proposed to enhance the ability of the feature map to represent useful information (e.g., spatial variance). In the testing phase, the trained deep networks are modified to generate the cloud activation map (CAM) via the local pooling pruning (LPP) strategy, which prunes the local pooling layers of the deep networks that are trained in the training phase to improve the quality (e.g., spatial resolution) of CAM. One large RS image is cropped into multiple overlapping blocks by a sliding window, and then the CAM of each block is generated by the modified deep networks. Based on the correspondence between the image blocks and CAMs, multiple corresponding CAMs are collected to mosaic the CAM of the large image. By segmenting the CAM using a statistical threshold against a clear-sky surface, the pixel-level cloud mask of the testing image can be obtained. To verify the effectiveness of our proposed WDCD method, we collected a new global dataset, for which the training dataset contains over 200,000 RS image blocks with block-level labels from 622 large GaoFen-1 images from all over the world; the validation dataset contains 5 large GaoFen-1 images with pixel-level annotation labels, and the testing dataset contains 25 large GaoFen-1 and ZiYuan-3 images with pixel-level annotation labels. Even under the extremely weak supervision, our proposed WDCD method could achieve excellent cloud detection performance with an overall accuracy (OA) as high as 96.66%. Extensive experiments demonstrated that our proposed WDCD method obviously outperforms the state-of-the-art methods. The collected datasets have been made publicly available online (https://github.com/weichenrs/WDCD).

CLOUD CLASSIFICATION FOR GROUND-BASED SKY IMAGE USING RANDOM FOREST

Abstract. The use of solar power as a renewable energy has grown rapidly over the last few decades. However, the amount of solar radiation reaching the ground vary significantly in the short term. Clouds are the main factor. In this paper, a novel cloud detection method for ground-based sky images is proposed. First, the multiple features from the sky images, including spectral, texture and colour features are combined into a feature set. Then, Random Forest with this feature set is used to classify different types of cloud and clear sky. The experimental results show that cumulus and cirrus clouds can be identified from sky images. Combined with random forest, three types of features and various feature combinations are used for cloud classification, respectively. The classification accuracy with multiple features is higher than that of single-type features and dual-type features.

Using Minimum Component and CNN for Satellite Remote Sensing Image Cloud Detection

Cloud detection is an important part of remote sensing (RS) image preprocessing. For earth observation tasks, the reliability of RS images will be judged based on the presence of clouds. A large number of cloud detection methods have been developed. There are two difficulties for cloud detection. First, it is hard to detect thin clouds and ragged clouds. Second, clouds are hard to distinguish from photometrically similar regions, such as snow. The rise of deep learning has brought new methods to address the above problems. In this letter, we propose a novel end-to-end neural network that detects clouds without additional manual work. Furthermore, we develop an RGB minimum component transformation mechanism that is useful for discriminating clouds from snow. Moreover, we have made our data set public to help others for further research. Our method increases the precision of cloud detection to 93.73%.

Cloud/haze detection in airborne videos using a convolutional neural network

In airborne videos surveillance, moving object detection and target tracking are the key steps. However, under bad weather conditions, the presence of clouds and haze or even smoke coming from buildings can make the processing of these videos very challenging. Current cloud detection or classification methods only consider a single image. Moreover, the images they use are often captured by satellites or planes at high altitudes with very long ranges to clouds, which can help distinguish cloudy regions from non-cloudy ones. In this paper, a new approach for cloud and haze detection is proposed by exploiting both spatial and temporal information in airborne videos. In this method, several consecutive frames are divided into patches. Then, consecutive patches are collected as patch sets and fed into a deep convolutional neural network. The network is trained to learn the appearance of clouds as well as their motion characteristics. Therefore, instead of relying on single frame patches, the decision on a patch in the current frame is made based on patches from previous and subsequent consecutive frames. This approach, avoids discarding the temporal information about clouds in videos, which may contain important cues for discriminating between cloudy and non-cloudy regions. Experimental results show that using temporal information besides the spatial characteristics of haze and clouds can greatly increase detection accuracy.

AUTOMATIC CLOUD DETECTION METHOD BASED ON GENERATIVE ADVERSARIAL NETWORKS IN REMOTE SENSING IMAGES

Abstract. Clouds in optical remote sensing images seriously affect the visibility of background pixels and greatly reduce the availability of images. It is necessary to detect clouds before processing images. In this paper, a novel cloud detection method based on attentive generative adversarial network (Auto-GAN) is proposed for cloud detection. Our main idea is to inject visual attention into the domain transformation to detect clouds automatically. First, we use a discriminator (D) to distinguish between cloudy and cloud free images. Then, a segmentation network is used to detect the difference between cloudy and cloud-free images (i.e. clouds). Last, a generator (G) is used to fill in the different regions in cloud image in order to confuse the discriminator. Auto-GAN only requires images and their labels (1 for a cloud-free image, 0 for a cloudy image) in the training phase which is more time-saving to acquire than existing methods based on CNNs that require pixel-level labels. Auto-GAN is applied to cloud detection in Sentinel-2A Level 1C imagery. The results indicate that Auto-GAN method performs well in cloud detection over different land surfaces.

TEMPORAL REPETITION DETECTION FOR GROUND VISIBILITY ASSESSMENT

Abstract. Assessing ground visibility is a crucial step in automatic satellite image analysis. Nevertheless, several recent Earth observation satellite constellations lack specially designed spectral bands and use a frame camera, precluding spectrum-based and parallax-based cloud detection methods. An alternative approach is to detect the parts of each image where the ground is visible. This can be done by comparing locally pairs of registered images in a temporal series: matching regions are necessarily cloud free. Indeed, the ground has persistent patterns that can be observed repetitively in the time series while the appearance of clouds changes at each date. To detect reliably the “visible” ground, we propose here an a contrario local image matching method coupled with an efficient greedy algorithm.

Cloud Detection for Satellite Imagery Using Deep Learning

Cloud is the most uncertain factor in the climate system and has a huge impact on climate change. Therefore, the study of changes in cloudiness is of great importance for understanding climate and climate change. Cloud detection is also an important research area in satellite remote sensing image pre-processing. But cloud detection is a difficult task due to various noise disturbances in the remote sensing data itself, as well as factors such as ice and snow on the ground. With the rapid development of artificial intelligence technology, deep learning methods have achieved great success in methods such as image processing and classification. In this study, we use the modified U-Net architecture that introduces the attention mechanism for cloud detection. The experimental results show that the method proposed in this paper has a significant accuracy advantage over the traditional cloud detection method, especially in snowy areas and other areas covered by bright non-cloud objects. The effectiveness of this method makes it a great potential for other optical image processing as well.

A Novel Classification Extension-Based Cloud Detection Method for Medium-Resolution Optical Images

Accurate cloud detection using medium-resolution multispectral satellite imagery (such as Landsat and Sentinel data) is always difficult due to the complex land surfaces, diverse cloud types, and limited number of available spectral bands, especially in the case of images without thermal bands. In this paper, a novel classification extension-based cloud detection (CECD) method was proposed for masking clouds in the medium-resolution images. The new method does not rely on thermal bands and can be used for masking clouds in different types of medium-resolution satellite imagery. First, with the support of low-resolution satellite imagery with short revisit periods, cloud and non-cloud pixels were identified in the resampled low-resolution version of the medium-resolution cloudy image. Then, based on the identified cloud and non-cloud pixels and the resampled cloudy image, training samples were automatically collected to develop a random forest (RF) classifier. Finally, the developed RF classifier was extended to the corresponding medium-resolution cloudy image to generate an accurate cloud mask. The CECD method was applied to Landsat-8 and Sentinel-2 imagery to test the performance for different satellite images, and the well-known function of mask (FMASK) method was employed for comparison with our method. The results indicate that CECD is more accurate at detecting clouds in Landsat-8 and Sentinel-2 imagery, giving an average F-measure value of 97.65% and 97.11% for Landsat-8 and Sentinel-2 imagery, respectively, as against corresponding results of 90.80% and 88.47% for FMASK. It is concluded, therefore, that the proposed CECD algorithm is an effective cloud-classification algorithm that can be applied to the medium-resolution optical satellite imagery.