Cloud Detection Method Research Articles

A transformer-based cloud detection approach using Sentinel 2 imageries

ABSTRACT Presence of clouds blocks the view of Earth’s surface objects in optical imageries, thus compromising their application and usability. Identifying and removing the clouds become a crucial task during image preprocessing. Recently deep learning (DL)-based cloud detection methods have shown improved performance, but capturing global semantic features and long-range dependencies necessitates a careful selection of DL classifiers to further enhance their effectiveness. Keeping this in view, the present study proposes a novel spatial-spectral attention transformer for cloud detection (SSATR-CD) with a spatial-spectral attention module that generates an enhanced feature map to replace convolution by using the image patches directly. To implement the proposed approach, a new Sentinel-2 data set with various types of cloud covers over India (IndiaS2) was created and tested with the proposed method. Alongside this, an additional benchmarked data set (WHUS2-CD) was also considered to check the ability of the proposed model to different regions of the world by applying model-based transfer learning. The result highlights the effectiveness and efficiency of the SSATR-CD approach in both cases.

Cloud detection of multi-type satellite images based on spectral assimilation and deep learning

ABSTRACT With strong self-learning and data analysis capabilities, deep learning is essential in cloud detection. However, many high-quality samples are the key to deep learning cloud detection methods. Different satellite image cloud detection techniques need to select adequate representative and high-quality samples and make corresponding cloud masks, which not only requires professional knowledge but also consumes a lot of workforce and time. To improve the generalization ability of the deep learning cloud detection model and quickly apply it to the cloud detection of different satellite images, this paper proposes a deep learning cloud detection method based on spectral assimilation for multiple types of satellite images (SAUNetCD). Under the condition of using fewer deep learning data samples, the deep learning model is used to achieve automatic cloud detection of multiple satellite data. Taking Landsat 8 OLI, Landsat 9 OLI, GF-1 WFV, and Sentinel 2A as examples, this paper selects Landsat 8 OLI data as the source data of cloud detection. The experimental results show that the spectral assimilation method improves the generalization ability of the deep learning cloud detection model and improves the cloud detection accuracy by nearly 20%. It realizes the fast cloud detection application of different satellite images by deep learning. It provides an effective way for cloud detection of multiple types of satellite images.

Automatic cloud detection method for optical remote sensing images combining channel attention mechanism and multi-feature fusion

Cloud detection is a crucial step in remote sensing image pre-processing, but there are still issues such as cloud miss detection and false detection. Therefore, this work suggests a multi-feature fusion and channel attention mechanism combined approach for automatically recognizing clouds. Based on DeepLabV3+, to make the feature extraction faster and more accurate, a lightweight network structure MobileNetV2 is used as the backbone network. To effectively distinguish snow and clouds, channel attention is added to the low-level semantic information, which makes the network more able to focus on essential features. Meantime, mid-level semantic features are added to the decoder module to make use of more abundant features, thus reducing the false detection of mist. Experimental findings on the GF1 WHU dataset demonstrate that the method’s detection accuracy is improved by 2.1% when compared to DeepLabV3+, and it performs well in the cloud detection job of optical remote sensing images.

MTCSNet: Mean Teachers Cross-Supervision Network for Semi-Supervised Cloud Detection

Cloud detection methods based on deep learning depend on large and reliable training datasets to achieve high detection accuracy. There will be a significant impact on their performance, however when the training data are insufficient or when the label quality is low. Thus, to alleviate this problem, a semi-supervised cloud detection method, named the mean teacher cross-supervision cloud detection network (MTCSNet) is proposed. This method enforces both consistency and accuracy on two cloud detection student network branches, which are perturbed with different initializations, for the same input image. For each of the two student branches, the respective teacher branches, used to generate high-quality pseudo labels, are constructed using an exponential moving average method (EMA). A pseudo one-hot label, produced by one teacher network branch, supervises the other student network branch with the standard cross-entropy loss, and vice versa. To incorporate additional prior information into the model, the presented method uses near-infrared bands instead of red bands as model inputs and injects strong data augmentations on unlabeled images fed into the student model. This induces the model to learn richer representations and ensure consistency constraints on the predictions of the same unlabeled image across different batches. To attain a more refined equilibrium between the supervised and semi-supervised loss in the training process, the proposed cloud detection network learns the optimal weights based on homoscedastic uncertainty, thus effectively exploiting the advantages of semi-supervised tasks and elevating the overall performance. Experiments on the SPARCS and GF1-WHU public cloud detection datasets show that the proposed method outperforms several state-of-the-art semi-supervised algorithms when only a limited number of labeled samples are available.

Cloud detection method based on clear sky background under multiple weather conditions

This paper deals with an image processing methodology based on a clear sky library for cloud detection. It is a part of a project which aims at using the detection results of ground-based cloud images in combination with solar irradiation data to complete power forecasts for solar power plants at ultra-short-term horizons (15 min). This paper focuses on the implementation of the cloud detection process. To overcome the problem of detection failure caused by uneven background brightness distribution in the sky, detection work is achieved by building a library of clear sky background images for various weather conditions at different solar zenith angles and combining background elimination methods through image rotation and image matching algorithms. A hybrid algorithm is proposed for detection failure when the sun is blocked by the cloud to achieve secondary processing. The test results show that this method has good detection accuracy at the visual level compared to traditional algorithms, especially for situations with low cloud and sky discrepancies such as thin clouds and hazy weather.

A cloud detection neural network for above-aircraft clouds using airborne cameras

For aerosol, cloud, land, and ocean remote sensing, the development of accurate cloud detection methods, or cloud masks, is extremely important. For airborne passive remotesensing, it is also important to identify when clouds are above the aircraft since their presence contaminates the measurements of nadir-viewing passive sensors. We describe the development of a camera-based approach to detecting clouds above the aircraft via a convolutional neural network called the cloud detection neural network (CDNN). We quantify the performance of this CDNN using human-labeled validation data where we report 96% accuracy in detecting clouds in testing datasets for both zenith viewing and forward-viewing models. We present results from the CDNN based on airborne imagery from the NASA Aerosol Cloud meteorology Interactions oVer the western Atlantic Experiment (ACTIVATE) and the Clouds, Aerosol, and Monsoon Processes Philippines Experiment (CAMP2Ex). We quantify the ability of the CDNN to identify the presence of clouds above the aircraft using a forward-looking camera mounted inside the aircraft cockpit compared to the use of an all-sky upward-looking camera that is mounted outside the fuselage on top of the aircraft. We assess our performance by comparing the flight-averaged cloud fraction of zenith and forward CDNN retrievals with that of the prototype hyperspectral total-diffuse Sunshine Pyranometer (SPN-S) instrument’s cloud optical depth data. A comparison of the CDNN with the SPN-S on time-specific intervals resulted in 93% accuracy for the zenith viewing CDNN and 84% for the forward-viewing CDNN. The comparison of the CDNNs with the SPN-S on flight-averaged cloud fraction resulted in an agreement of .15 for the forward CDNN and .07 for the zenith CDNN. For CAMP2Ex, 53% of flight dates had above-aircraft cloud fraction above 50%, while for ACTIVATE, 52% and 54% of flight dates observed above-aircraft cloud fraction above 50% for 2020 and 2021, respectively. The CDNN enables cost-effective detection of clouds above the aircraft using an inexpensive camera installed in the cockpit for airborne science research flights where there are no dedicated upward-looking instruments for cloud detection, the installation of which requires time-consuming and expensive aircraft modifications, in addition to added mission cost and complexity of operating additional instruments.

Cloud Detection From High-Resolution Remote Sensing Images Based on Convolutional Neural Networks With Geographic Features and Contextual Information

The diversity and complexity of the subsurface, as well as the similarity of cloud features to those of highlighted surface objects (especially snowy areas), are the main reasons for the current limitations in cloud detection accuracy. This letter proposes a semantic segmentation network fusing geographic features with contextual information for cloud detection named GCI_CD. We design three modules: geographical-cloud information fusion module (to distinguish between clouds and snow), separable ResNet with CSAM encoder (to enhance channel and spatial association information extraction), contextual information integration module (to extract multi-scale features). To the best of our knowledge, this is the first time that geographic information has been fused in a cloud detection method. The validity of our proposed method is demonstrated using a high-resolution GF-1 dataset containing the near-infrared band and dividing the snow-covered and non-snow-covered areas. The experimental results show that GCI_CD has a better performance not only in snow-covered regions but also in the non-snow-covered regions. Especially for snow-covered images, the IOU of GCI_CD’s cloud detection results are improved by 5% on average compared to other cloud detection methods.

Cloud Detection in Optical Remote Sensing Images With Deep Semi-Supervised and Active Learning

Clouds hinder the surface observation by optical remote sensing sensors. It is of great significance to detect clouds and non-clouds in remote sensing images. Compared with traditional cloud detection methods, deep learning methods usually achieve promising detection results. Moreover, large-scale, high-quality labeled datasets can effectively improve the accuracy and generalization of deep learning models. However, this incurs a great deal of label effort and cost. In this paper, we proposes a cloud detection method based on deep semi-supervised learning(SSL) and active learning(AL) in optical remote sensing images named SSAL-CD. SSL part of SSAL-CD is implemented by the cross-validation between two-way neural networks, which can effectively train in the case of insufficient labeled data and reduce the cost of labeling. AL part proposes a suitable query strategy based on the uncertainty, degree of divergence and diversity information provided by the training model. The query strategy can screen out high-value samples for SSL, which further improves the detection effect. Repeat the process of SSL and AL until the accuracy meets requirements or the labeling budget is exhausted. Extensive experimental results in Landsat-8 Biome dataset demonstrate that SSAL-CD can achieve state-of-the-art segmentation performance with a small number of labels.

A Priori Land Surface Reflectance Synergized With Multiscale Features Convolution Neural Network for MODIS Imagery Cloud Detection

Moderate Resolution Imaging Spectrometer (MODIS) images are widely used in land, ocean, and atmospheric monitoring, due to their wide spectral coverage, high temporal resolution, and convenient data acquisition. Accurate cloud detection is critical to the fine processing and application of MODIS images. Owing to spatial resolution limitations and the influence of mixed pixels, most MODIS cloud detection algorithms struggle to effectively recognize of clouds and ground objects. Here, we propose a novel cloud detection method based on land surface reflectance (LSR) and a multiscale feature convolutional neural network to achieve high-precision cloud detection, particularly for thin clouds and clouds over bright surface. A monthly surface reflectance dataset was constructed by MODIS products (MOD09A1) and employed to provide background information for cloud detection. Difference-based samples were obtained using surface reflectance as well MODIS images of different phases based on difference operations. The multiscale feature network (MFCD-Net) using an atrous spatial pyramid pooling (ASPP) and a channel and spatial attention module (CSAM) integrated low-level spatial features and high-level semantic information to capture multiscale features and generate a high-precision cloud mask. For cloud detection experiments and quantitative analysis, 61 MODIS images acquired at different times on various underlying surface types were used. Cloud detection results were compared to those of UNet, Deeplabv3+, UNet++, PSPNet, and top of atmosphere-based (MFCD-TOA) methods. The proposed method performed well, with the highest overall accuracy (96.55%), precision (92.13%), and recall (88.90%). It improved cloud detection accuracy in various scenarios, reducing thin cloud omission and bright surface misidentification.

Cloud Detection Method Based on Spatial–Spectral Features and Encoder–Decoder Feature Fusion

Cloud Detection Method Based on Spatial–Spectral Features and Encoder–Decoder Feature Fusion

A Daytime Cloud Detection Method for Advanced Meteorological Imager Using Visible and Near-Infrared Bands

A Daytime Cloud Detection Method for Advanced Meteorological Imager Using Visible and Near-Infrared Bands

A 3-D Cloud Detection Method for FY-4A GIIRS and Its Application in Operational Numerical Weather Prediction System

A 3-D Cloud Detection Method for FY-4A GIIRS and Its Application in Operational Numerical Weather Prediction System

Boundary-Aware Bilateral Fusion Network for Cloud Detection

Cloud detection is one of the key technologies in the field of remote sensing. Although extensive deep learning-based cloud detection methods achieve good performance, their detection results in confusing areas such as cloud boundaries and thin clouds are often not satisfactory due to the potential inter-class similarity and intra-class inconsistency of objects. To this end, we propose a Boundary-Aware Bilateral Fusion network (BABFNet), which effectively enhances cloud detection in confusing areas by introducing a boundary prediction branch as an auxiliary. To avoid the loss of details, the boundary prediction branch is designed to run at full resolution with a shallow architecture, while some Semantic Enhancement Modules (SEMs) are used to supplement high-level semantic information by introducing multi-level encoder features of the cloud detection branch. This feature sharing in turn drives the cloud detection branch to focus more on cloud boundaries during training. At the end of the network, a Bilateral Fusion Module (BFM) is added for information complementarity between features from these two branches. The features from the cloud detection branch provide multi-scale features to the boundary prediction branch for more accurate boundary prediction, while the features from the boundary prediction branch further serve as prior knowledge to help the cloud detection branch aggregate contextual information. To verify the effectiveness of the proposed method, we select four different networks as cloud detection branches and conduct comparative experiments on two public datasets, GF-1 WFV and MODIS. The experimental results show that the proposed method significantly enhances cloud detection in confusing areas.

CloudSEN12, a global dataset for semantic understanding of cloud and cloud shadow in Sentinel-2

Accurately characterizing clouds and their shadows is a long-standing problem in the Earth Observation community. Recent works showcase the necessity to improve cloud detection methods for imagery acquired by the Sentinel-2 satellites. However, the lack of consensus and transparency in existing reference datasets hampers the benchmarking of current cloud detection methods. Exploiting the analysis-ready data offered by the Copernicus program, we created CloudSEN12, a new multi-temporal global dataset to foster research in cloud and cloud shadow detection. CloudSEN12 has 49,400 image patches, including (1) Sentinel-2 level-1C and level-2A multi-spectral data, (2) Sentinel-1 synthetic aperture radar data, (3) auxiliary remote sensing products, (4) different hand-crafted annotations to label the presence of thick and thin clouds and cloud shadows, and (5) the results from eight state-of-the-art cloud detection algorithms. At present, CloudSEN12 exceeds all previous efforts in terms of annotation richness, scene variability, geographic distribution, metadata complexity, quality control, and number of samples.

A Cloud Detection Method Based on Spectral and Gradient Features for SDGSAT-1 Multispectral Images

Due to the limited penetration of visible bands, optical remote sensing images are inevitably contaminated by clouds. Therefore, cloud detection or cloud mask products for optical image processing is a very important step. Compared with conventional optical remote sensing satellites (such as Landsat series and Sentinel-2), sustainable development science Satellite-1 (SDGSAT-1) multi-spectral imager (MII) lacks a short-wave infrared (SWIR) band that can be used to effectively distinguish cloud and snow. To solve the above problems, a cloud detection method based on spectral and gradient features (SGF) for SDGSAT-1 multispectral images is proposed in this paper. According to the differences in spectral features between cloud and other ground objects, the method combines four features, namely, brightness, normalized difference water index (NDWI), normalized difference vegetation index (NDVI), and haze-optimized transformation (HOT) to distinguish cloud and most ground objects. Meanwhile, in order to adapt to different environments, the dynamic threshold using Otsu’s method is adopted. In addition, it is worth mentioning that gradient features are used to distinguish cloud and snow in this paper. With the test of SDGSAT-1 multispectral images and comparison experiments, the results show that SGF has excellent performance. The overall accuracy of images with snow surface can reach 90.80%, and the overall accuracy of images with other surfaces is above 94%.

Transfer-learning-based cloud detection for Zhuhai-1 satellite hyperspectral imagery

The Zhuhai-1 hyperspectral satellite can simultaneously obtain spectral information in 32 spectral bands and effectively obtain accurate information on land features through integrated hyperspectral observations of the atmosphere and land, while the presence of clouds can contaminate remote sensing images. To improve the utilization rate of hyperspectral images, this study investigates the cloud detection method for hyperspectral satellite data based on the transfer learning technique, which can obtain a model with high generalization capability with a small training sample size. In this study, for the acquired Level-1B products, the top-of-atmosphere reflectance data of each band are obtained by using the calibration coefficients and spectral response functions of the product packages. Meanwhile, to eliminate the data redundancy between hyperspectral bands, the data are downscaled using the principal component transformation method, and the top three principal components are extracted as the sample input data for model training. Then, the pretrained VGG16 and ResNet50 weight files are used as the backbone network of the encoder, and the model is updated and trained again using Orbita hyperspectral satellite (OHS) sample data to fine-tune the feature extraction parameters. Finally, the cloud detection model is obtained. To verify the accuracy of the method, the multi-view OHS images are visually interpreted, and the cloud pixels are sketched out as the baseline data. The experimental results show that the overall accuracy of the cloud detection model based on the Resnet50 backbone network can reach 91%, which can accurately distinguish clouds from clear sky and achieve high-accuracy cloud detection in hyperspectral remote sensing images.

Impacts of FY-4A AGRI Radiance Data Assimilation on the Forecast of the Super Typhoon “In-Fa” (2021)

This study assessed the impact of assimilating the Fengyun-4A (FY-4A) Advanced Geosynchronous Radiation Imager (AGRI) observations on the Super Typhoon “In-Fa” event based on the Weather Research and Forecasting Data Assimilation (WRFDA) system of the three-dimensional variational data assimilation (3DVAR) method. It was found that the two water vapor channels 9–10 from the full-disk AGRI datasets yield relatively stable results in terms of the track forecast of In-Fa. A new cloud-detection method using a Particle Filter (PF) was firstly employed to remove the cloud-affected observations by identifying the channel’s weighting function. Compared to the other cloud-detection schemes based on the AGRI “Cloud_Binary_Mask” (CLM) products, the PF method is conducive to reducing the track error of typhoon prediction after improving the utilization of observations under clear-sky conditions. Furthermore, the proposed cycling assimilation scheme has a potential positive effect on the intensity forecast of In-Fa. It seems that assimilating the FY-4A AGRI radiance data improves the predictability of Typhoon In-Fa by adjusting the atmospheric environment.

Cloud Detection of Gaofen-2 Multi-Spectral Imagery Based on the Modified Radiation Transmittance Map

For optical remote sensing images with high spatial resolution and low spectral number, the complexity of ground objects poses great challenges to cloud detection algorithms, such as the differentiation of clouds from objects with similar features as clouds and the identification of thin clouds. In this paper, a novel cloud detection method is proposed for Gaofen-2 remote sensing imagery. The radiation transmittance is estimated based on the dark channel prior, and the overestimated radiation transmittance is corrected using spectral features. A three-step post-processing strategy is adopted to eliminate misidentification introduced by the highlighted surfaces based on object geometric, textural, and boundary features. In the experiments, Gaofen-2 multispectral images with different cloud categories and cloud thicknesses are involved to evaluate the performance of the proposed method. The results show that the proposed method can obtain an average cloud detection accuracy of 0.9573 on six different clouds. The proposed algorithm can also effectively detect both thick and thin clouds with an average accuracy of more than 0.9517. The advantages of the method for thin cloud detection are further demonstrated by comparison with existing algorithms.

Optical Remote Sensing Image Cloud Detection with Self-Attention and Spatial Pyramid Pooling Fusion

Cloud detection is a key step in optical remote sensing image processing, and the cloud-free image is of great significance for land use classification, change detection, and long time-series landcover monitoring. Traditional cloud detection methods based on spectral and texture features have acquired certain effects in complex scenarios, such as cloud–snow mixing, but there is still a large room for improvement in terms of generation ability. In recent years, cloud detection with deep-learning methods has significantly improved the accuracy in complex regions such as high-brightness feature mixing areas. However, the existing deep learning-based cloud detection methods still have certain limitations. For instance, a few omission alarms and commission alarms still exist in cloud edge regions. At present, the cloud detection methods based on deep learning are gradually converted from a pure convolutional structure to a global feature extraction perspective, such as attention modules, but the computational burden is also increased, which is difficult to meet for the rapidly developing time-sensitive tasks, such as onboard real-time cloud detection in optical remote sensing imagery. To address the above problems, this manuscript proposes a high-precision cloud detection network fusing a self-attention module and spatial pyramidal pooling. Firstly, we use the DenseNet network as the backbone, then the deep semantic features are extracted by combining a global self-attention module and spatial pyramid pooling module. Secondly, to solve the problem of unbalanced training samples, we design a weighted cross-entropy loss function to optimize it. Finally, cloud detection accuracy is assessed. With the quantitative comparison experiments on different images, such as Landsat8, Landsat9, GF-2, and Beijing-2, the results indicate that, compared with the feature-based methods, the deep learning network can effectively distinguish in the cloud–snow confusion-prone region using only visible three-channel images, which significantly reduces the number of required image bands. Compared with other deep learning methods, the accuracy at the edge of the cloud region is higher and the overall computational efficiency is relatively optimal.

Cloud and snow detection of remote sensing images based on improved Unet3+

Cloud detection is an important step in remote sensing image processing and a prerequisite for subsequent analysis and interpretation of remote sensing images. Traditional cloud detection methods are difficult to accurately detect clouds and snow with very similar features such as color and texture. In this paper, the features of cloud and snow in remote sensing images are deeply extracted, and an accurate cloud and snow detection method is proposed based on the advantages of Unet3+ network in feature fusion. Firstly, color space conversion is performed on remote sensing images, RGB images and HIS images are used as input of Unet3+ network. Resnet 50 is used to replace the Unet3+ feature extraction network to extract remote sensing image features at a deeper level, and add the Convolutional Block Attention Module in Resnet50 to improve the network’s attention to cloud and snow. Finally, the weighted cross entropy loss is constructed to solve the problem of unbalanced sample number caused by high proportion of background area in the image. The results show that the proposed method has strong adaptability and moderate computation. The mPA value, mIoU value and mPrecision value can reach 92.76%, 81.74% and 86.49%, respectively. Compared with other algorithms, the proposed method can better eliminate all kinds of interference information in remote sensing images of different landforms and accurately detect cloud and snow in images.