Cloud Shadows Research Articles

Mobile sensor for clouds shadow detection and direct normal irradiance estimation

This paper presents a mobile sensor system to detect and estimate low direct normal irradiance (DNI) areas caused by clouds shadows. This work proposes using a team of unmanned aerial vehicles (UAVs) to localise and characterise the shadow of mobile clouds. This information can be used by the plant control system to minimise its effects over a solar plant.Simulations to test and discuss the algorithm are presented. The work presented here obtains a similar degree of precision as far as the estimation of the shape of the cloud shadow is concerned but with a much faster computational time than other algorithms described in literature.

Multi-scale strip pooling feature aggregation network for cloud and cloud shadow segmentation

Multi-scale strip pooling feature aggregation network for cloud and cloud shadow segmentation

Cloud/shadow segmentation based on multi-level feature enhanced network for remote sensing imagery

ABSTRACT In the application of remote sensing, cloud blocking brings trouble to the analysis of surface parameters and atmospheric parameters. Due to the complexity of the background, the influence of some cloud-like interferences (such as ice, snow, buildings, etc.) and the complexity of the cloud shape, the traditional deep learning method is difficult to segment the edge information of cloud and cloud shadow accurately, resulting in misjudgement at the edge. In order to solve these problems, a multilevel feature enhanced network is proposed for cloud/shadow segmentation. In this work, ResNet-18 is used as the backbone to extract all levels of semantic information, and Feature Enhancement Module is proposed to strengthen the feature information to obtain more effective feature information. Multiscale Fusion module is constructed to fuses multiscale features of deep information to obtain global feature information while considering local feature information. Finally, through the Feature Guidance module, low-level features are used to guide the high-level features to guide the recovery of spatial information and improve the efficiency of upsampling. On the data collected by Landsat-8, Sentinel-2, and HRC_WHU data set, the experimental results show that this method is superior to the existing methods.

Pragmatic Augmentation Algorithms for Deep Learning-Based Cloud and Cloud Shadow Detection in Remote Sensing Imagery

Identification of clouds and their shadows are two major preprocessing steps for providing an effective interpretation of remotely sensed optical images. Although deep learning-based methods have proved to deliver competent performance for detecting clouds and cloud shadows, improving their generalization ability for reliable results requires a large number of training images and accurate ground truths. As creating ground truth of cloud and cloud shadow is expensive, a practical way to generate more images and their ground truths is to use data augmentation methods. We propose two new data augmentation approaches (one for generating synthetic clouds in scenes and the other one for creating cloud shadows with different levels of shade) so as to achieve natural-looking images with little or no effort required for creating their ground truths. Our experiments show that while each of these approaches is capable of boosting cloud and cloud shadow segmentation individually, the fusion of them improves detecting cloud and shadow in a simultaneous learning pipeline, leading to outperforming the state-of-the-art results.

Robust Thick Cloud Removal for Multitemporal Remote Sensing Images Using Coupled Tensor Factorization

The existing nonblind cloud and cloud shadow (cloud/shadow) removal methods for remote sensing (RS) images are based on the assumption that cloud/shadow masks are accurately given. Since the masks are usually manually labeled or detected by cloud detection methods, whose accuracy cannot be well guaranteed, the cloud/shadow removal effect may be affected. In this article, we suggest a robust thick cloud/shadow removal (RTCR) method that meets the problem with an inaccurate mask. To faithfully reconstruct the multitemporal information, a coupled tensor factorization is used to explore the relationship between the abundances of the multitemporal images in the same scene. Moreover, an efficient algorithm is developed to solve the proposed model based on the augmented Lagrange multiplier method. The experimental results under accurate masks and inaccurate masks demonstrate its robustness and superiority for thick cloud/shadow removal.

Reconstructing Missing Information of Remote Sensing Data Contaminated by Large and Thick Clouds Based on an Improved Multitemporal Dictionary Learning Method

The presence of clouds and cloud shadows has limited the applications of optical remote sensing data. Currently, most cloud removal methods are focused on reconstructing remote sensing data contaminated by small or thin clouds. This study proposes an improved method based on multitemporal dictionary learning to reconstruct missing information of remote sensing data contaminated by large and thick clouds. First, the contaminated target image is initialized using all available adjacent cloud-free reference images. Second, reconstructed images from each of the reference images are produced using dictionary learning and sparse representation methods. Then, weights are determined for the abovementioned reconstructed images based on their reconstruction errors over uncontaminated regions and are used to generate the preliminary reconstruction result. Finally, an error correction step for the contaminated regions is applied to the preliminary result, which is then combined with the original uncontaminated pixels to produce the final reconstruction result. The proposed method was evaluated on simulated clouds/cloud shadows based on remote sensing data with various sizes and land cover types. Visual and quantitative analyses of the reconstruction results show that the proposed method outperformed the generally used geostatistical neighborhood similar pixel interpolator (GNSPI) and nonnegative matrix factorization and error correction (S-NMF-EC) methods. Therefore, the results indicated that the proposed method was capable of accurately and effectively reconstructing data contaminated by large and thick clouds.

LCDNet: Light-Weighted Cloud Detection Network for High-Resolution Remote Sensing Images

Cloud detection is one of the important links in high-resolution remote sensing image processing. Cloud detection methods can be mainly divided into three categories: threshold methods, clustering methods based on machine learning, and deep learning methods. The traditional threshold method needs cumbersome manual calibration, which has high cost and poor universality. The generalization of clustering based method is very poor. In addition, existing deep learning methods tend to have many model parameters and high training costs. To solve the above problems, a light-weighted cloud detection network (LCDNet) based on deep learning method is proposed. The network can complete the task of high-precision segmentation with less parameters and computation. Its light-weighted bottleneck (LB) layer can quickly capture the multi-scale feature information in the image and segment the cloud with high efficiency. The gated channel excitation (GCE) module can effectively reduce the feature redundancy in the network, so as to highlight the details of the cloud layer and improve the detection accuracy. The function of light-weighted self attention (LSA) module is to quickly establish the spatial location information in the feature map, so as to locate the unpredictable targets and reduce the false detection rate. The experimental results show that the model shows excellent performance on cloud and cloud shadow datasets with 353.76 k parameters and 456.28Mmac computation. In addition, the training results on GF1_WHU and L8 Biome datasets further show that our model has excellent generalization performance, which is of great significance for the efficient implementation of cloud detection.

Solar Image Cloud Removal based on Improved Pix2Pix Network

In ground-based observations of the Sun, solar images are often affected by appearance of thin clouds, which contaminate the images and affect the scientific results from data analysis. In this paper, the improved Pixel to Pixel Network (Pix2Pix) network is used to convert polluted images to clear images to remove the cloud shadow in the solar images. By adding attention module to the model, the hidden layer of Pix2Pix model can infer the attention map of the input feature vector according to the input feature vector. And then, the attention map is multiplied by the input feature map to give different weights to the hidden features in the feature map, adaptively refine the input feature map to make the model pay attention to important feature information and achieve better recovery effect. In order to further enhance the model’s ability to recover detailed features, perceptual loss is added to the loss function. The model was tested on the full disk H-alpha images datasets provided by Huairou Solar Observing Station, National Astronomical Observatories. The experimental results show that the model can effectively remove the influence of thin clouds on the picture and restore the details of solar activity. The peak signal-to-noise ratio (PSNR) reaches 27.3012 and the learned perceptual image patch similarity (LPIPS) reaches 0.330, which is superior to the existed dehaze algorithms.

Dual-Branch Network for Cloud and Cloud Shadow Segmentation

Cloud and cloud shadow segmentation is one of the most important issues in remote sensing image processing. Most of the remote sensing images are very complicated. In this work, a dual-branch model composed of Transformer and convolution network is proposed to extract semantic and spatial detail information of the image respectively to solve the problems of false detection and missed detection. To improve the model’s feature extraction, a Mutual Guidance Module is introduced so that the Transformer Branch and the Convolution Branch can guide each other for feature mining. Finally, in view of the problem of rough segmentation boundary, this work uses different features extracted by the Transformer Branch and the Convolution Branch for decoding, and repairs the rough segmentation boundary in the decoding part to make the segmentation boundary clearer. Experimental results on the Landsat-8, Sentinel-2 data, the public dataset HRC_WHU of Wuhan University and the public dataset SPARCS demonstrate the effectiveness of our method and its superiority to existing state-of-the-art Cloud and Cloud Shadow Segmentation approaches.

Spectral Temporal Information for Missing Data Reconstruction (STIMDR) of Landsat Reflectance Time Series

The number of Landsat time-series applications has grown substantially because of its approximately 50-year history and relatively high spatial resolution for observing long term changes in the Earth’s surface. However, missing observations (i.e., gaps) caused by clouds and cloud shadows, orbit and sensing geometry, and sensor issues have broadly limited the development of Landsat time-series applications. Due to the large area and temporal and spatial irregularity of time-series gaps, it is difficult to find an efficient and highly precise method to fill them. The Missing Observation Prediction based on Spectral-Temporal Metrics (MOPSTM) method has been proposed and delivered good performance in filling large-area gaps of single-date Landsat images. However, it can be less practical for a time series longer than one year due to the lack of mechanics that exclude dissimilar data in time series (e.g., different phenology or changes in land cover). To solve this problem, this study proposes a new gap-filling method, Spectral Temporal Information for Missing Data Reconstruction (STIMDR), and examines its performance in Landsat reflectance time series. Two groups of experiments, including 2000 × 2000 pixel Landsat single-date images and Landsat time series acquired from four sites (Kenya, Finland, Germany, and China), were performed to test the new method. We simulated artificial gaps to evaluate predicted pixel values with real observations. Quantitative and qualitative evaluations of gap-filled images through comparisons with other state-of-the-art methods confirmed the more robust and accurate performance of the proposed method. In addition, the proposed method was also able to fill gaps contaminated by extreme cloud cover for a period (e.g., winter in high-latitude areas). A down-stream task of random forest supervised classification through both gap-filled simulated datasets and the original valid datasets verified that STIMDR-generated products are relevant to the user community for land cover applications.

UATNet: U-Shape Attention-Based Transformer Net for Meteorological Satellite Cloud Recognition

Cloud recognition is a basic task in ground meteorological observation. It is of great significance to accurately identify cloud types from long-time-series satellite cloud images for improving the reliability and accuracy of weather forecasting. However, different from ground-based cloud images with a small observation range and easy operation, satellite cloud images have a wider cloud coverage area and contain more surface features. Hence, it is difficult to effectively extract the structural shape, area size, contour shape, hue, shadow and texture of clouds through traditional deep learning methods. In order to analyze the regional cloud type characteristics effectively, we construct a China region meteorological satellite cloud image dataset named CRMSCD, which consists of nine cloud types and the clear sky (cloudless). In this paper, we propose a novel neural network model, UATNet, which can realize the pixel-level classification of meteorological satellite cloud images. Our model efficiently integrates the spatial and multi-channel information of clouds. Specifically, several transformer blocks with modified self-attention computation (swin transformer blocks) and patch merging operations are used to build a hierarchical transformer, and spatial displacement is introduced to construct long-distance cross-window connections. In addition, we introduce a Channel Cross fusion with Transformer (CCT) to guide the multi-scale channel fusion, and design an Attention-based Squeeze and Excitation (ASE) to effectively connect the fused multi-scale channel information to the decoder features. The experimental results demonstrate that the proposed model achieved 82.33% PA, 67.79% MPA, 54.51% MIoU and 70.96% FWIoU on CRMSCD. Compared with the existing models, our method produces more precise segmentation performance, which demonstrates its superiority on meteorological satellite cloud recognition tasks.

Monitoring Small Water Bodies Using High Spatial and Temporal Resolution Analysis Ready Datasets

Basemap and Planet Fusion—derived from PlanetScope imagery—represent the next generation of analysis ready datasets that minimize the effects of the presence of clouds. These datasets have high spatial (3 m) and temporal (daily) resolution, which provides an unprecedented opportunity to improve the monitoring of on-farm reservoirs (OFRs)—small water bodies that store freshwater and play important role in surface hydrology and global irrigation activities. In this study, we assessed the usefulness of both datasets to monitor sub-weekly surface area changes of 340 OFRs in eastern Arkansas, USA, and we evaluated the datasets main differences when used to monitor OFRs. When comparing the OFRs surface area derived from Basemap and Planet Fusion to an independent validation dataset, both datasets had high agreement (r2 ≥ 0.87), and small uncertainties, with a mean absolute percent error (MAPE) between 7.05% and 10.08%. Pairwise surface area comparisons between the two datasets and the PlanetScope imagery showed that 61% of the OFRs had r2 ≥ 0.55, and 70% of the OFRs had MAPE <5%. In general, both datasets can be employed to monitor OFRs sub-weekly surface area changes, and Basemap had higher surface area variability and was more susceptible to the presence of cloud shadows and haze when compared to Planet Fusion, which had a smoother time series with less variability and fewer abrupt changes throughout the year. The uncertainties in surface area classification decreased as the OFRs increased in size. In addition, the surface area time series can have high variability, depending on the OFR environmental conditions (e.g., presence of vegetation inside the OFR). Our findings suggest that both datasets can be used to monitor OFRs sub-weekly, seasonal, and inter-annual surface area changes; therefore, these datasets can help improve freshwater management by allowing better assessment and management of the OFRs.

Object-Based on Land Cover Classification on LAPAN-A3 Satellite Imagery Using Tree Algorithm (Case Study: Rote Island)

LAPAN became serious about making a remote sensing satellite on its third-generation satellite. Launched a year after LAPAN-A2, the third-generation satellite, LAPAN-A3, brought LISAT as the main payload. LISAT is a multispectral camera with 4 bands (Red, Green, Blue, NIR) that can be used for land classification, agriculture monitoring, drought monitoring, and land use changing<em>. LAPAN-A3 is the third generation of micro-satellite developed by Satellite Technology Center – LAPAN. This satellite carries a multispectral push-broom sensor that can record the earth's surface at the visible and near-infrared spectrum. </em>This paper aims to determine object-based land cover classification in Rote Island using the LAPAN-A3 satellite image using the tree method algorithm. This classification technique is expected to increase the accuracy of land cover classification. This classification used the LAPAN-A3 satellite imagery of Rote Island. The first process was determined the segmentation with scale parameter 60, shape 0.5, and compactness 0.5. The result shows that OBIA classification on Rote Island, the area of the open land class is 233.67 km², the area of the settlement is 11.57 km², the body of water is 2006.21 km2, the area of low vegetation is 525.93 km², the area of high vegetation is 437.5 km², and there is no data (cloud and cloud shadows) on the LAPAN-A3 image of 45.78 km². The accuracy values obtained were producer 86.67%, KIA 83.02%, Helden 92.86%, Short 86.7%, KIA per class 82.72%, and 85.96%. This object-based classification can meet international and national land cover classification standards, namely at 80%.

A Supplementary Module to Improve Accuracy of the Quality Assessment Band in Landsat Cloud Images

Cloud contamination is a serious obstacle for the application of Landsat data. To popularize the applications of Landsat data, each Landsat image includes the corresponding Quality Assessment (QA) band, in which cloud and cloud shadow pixels have been flagged. However, previous studies suggested that Landsat QA band still needs to be modified to fulfill the requirement of Landsat data applications. In this study, we developed a Supplementary Module to improve the original QA band (called QA_SM). On one hand, QA_SM extracts spectral and geometrical features in the target Landsat cloud image from the original QA band. On the other, QA_SM incorporates the temporal change characteristics of clouds and cloud shadows between the target and reference images. We tested the new method at four local sites with different land covers and the Landsat-8 cloud cover validation dataset (“L8_Biome”). The experimental results show that QA_SM performs better than the original QA band and the multi-temporal method ATSA (Automatic Time-Series Analyses). QA_SM decreases omission errors of clouds and shadows in the original QA band effectively but meanwhile does not increase commission errors. Besides, the better performance of QA_SM is less affected by the selections of reference images because QA_SM considers the temporal change of land surface reflectance that is not caused by cloud contamination. By further designing a quantitative assessment experiment, we found that the QA band generated by QA_SM improves cloud-removal performance on Landsat cloud images, suggesting the benefits of the new method to advance the applications of Landsat data.

Schopenhauer in Dante’s Garden of Eden: Melancholy and Rubén Darío’s “Autumnal”

Schopenhauer in Dante’s Garden of Eden: Melancholy and Rubén Darío’s “Autumnal”

A method to extract cyanobacteria blooms from satellite imagery with no requirements for any prior atmospheric correction or cloud-masking

Lake Chaohu has been suffering from harmful cyanobacteria blooms, while the clouds pixels in satellite images are usually mistaken as cyanobacteria blooms by some traditional indicators, leading to the need for cloud masking in advance. In addition, atmospheric correction is another challenge due to lack of a general atmospheric correction method and the difficulties in evaluating its accuracy without in situ investigations. Fortunately, tasseled cap transformation (TCT) allows to extract vegetation properties directly from satellite imagery digital numbers (DN), which provides a perspective for extracting cyanobacteria blooms independent from atmospheric correction. This study focuses on how to use TCT to establish an indicator, which allows to extract cyanobacteria blooms directly from image DN values without conducting any atmospheric correction or cloud-masking. Training and test sets containing over 200,000 pixels are constructed from 18 Sentinel-2A/B MSI images acquired in different seasons in recent three years. Four components are derived from TCT and they could form up to 81 linear combinations. Experimental results performed on the training set show that the candidate, which combines the last three components with the coefficients of 1,-1 and 0, assigns cyanobacteria blooms pixels in a completely separated value range from water, cloud, cloud shadow and cloud edge pixels. The candidate is defined as ICW3C index. Its threshold value range of (175 330) is given and the pixels with ICW3C values greater than its threshold could be classified as cyanobacteria blooms. Comparisons between ICW3C and the floating algae index (FAI) on the test set show that ICW3C misclassifies 0.02% of cloud pixels and 1.55% of yellow cloud edge pixels as cyanobacteria blooms, however, 19.18% clouds, 13.74% yellow cloud edges and 19.34% blue-green cloud edges are incorrectly identified as cyanobacteria blooms by FAI. Comparisons between ICW3C and FAI performed on image regions over time show that, in clear-sky regions with cyanobacteria blooms, FAI extracts 5.81% more pixels, which mainly lay in the edge of cyanobacteria blooms. In cloud-covered image regions without cyanobacteria blooms, FAI misclassifies over 608 times as many cloud and cloud edge pixels as ICW3C. Sensitivity test results suggest that the change of ICW3C threshold within its value range (175 330) will not lead to serious increase in misclassification, and ICW3C performs stable to variations of viewing geometry. Extension tests indicate that ICW3C is applicable for several other sensors. Further researches are still needed to test whether ICW3C is suitable for other inland lakes or seas.

Characterization of Dry-Season Phenology in Tropical Forests by Reconstructing Cloud-Free Landsat Time Series

Fine-resolution satellite imagery is needed for characterizing dry-season phenology in tropical forests since many tropical forests are very spatially heterogeneous due to their diverse species and environmental background. However, fine-resolution satellite imagery, such as Landsat, has a 16-day revisit cycle that makes it hard to obtain a high-quality vegetation index time series due to persistent clouds in tropical regions. To solve this challenge, this study explored the feasibility of employing a series of advanced technologies for reconstructing a high-quality Landsat time series from 2005 to 2009 for detecting dry-season phenology in tropical forests; Puerto Rico was selected as a testbed. We combined bidirectional reflectance distribution function (BRDF) correction, cloud and shadow screening, and contaminated pixel interpolation to process the raw Landsat time series and developed a thresholding method to extract 15 phenology metrics. The cloud-masked and gap-filled reconstructed images were tested with simulated clouds. In addition, the derived phenology metrics for grassland and forest in the tropical dry forest zone of Puerto Rico were evaluated with ground observations from PhenoCam data and field plots. Results show that clouds and cloud shadows are more accurately detected than the Landsat cloud quality assessment (QA) band, and that data gaps resulting from those clouds and shadows can be accurately reconstructed (R2 = 0.89). In the tropical dry forest zone, the detected phenology dates (such as greenup, browndown, and dry-season length) generally agree with the PhenoCam observations (R2 = 0.69), and Landsat-based phenology is better than MODIS-based phenology for modeling aboveground biomass and leaf area index collected in field plots (plot size is roughly equivalent to a 3 × 3 Landsat pixels). This study suggests that the Landsat time series can be used to characterize the dry-season phenology of tropical forests after careful processing, which will help to improve our understanding of vegetation–climate interactions at fine scales in tropical forests.

Integration of Sentinel 1 and Sentinel 2 Satellite Images for Crop Mapping

Crop identification is key to global food security. Due to the large scale of crop estimation, the science of remote sensing was able to do well in this field. The purpose of this study is to study the shortcomings and strengths of combined radar data and optical images to identify the type of crops in Tarom region (Iran). For this purpose, Sentinel 1 and Sentinel 2 images were used to create a map in the study area. The Sentinel 1 data came from Google Earth Engine’s (GEE) Level-1 Ground Range Detected (GRD) Interferometric Wide Swath (IW) product. Sentinel 1 radar observations were projected onto a standard 10-m grid in GRD output. The Sen2Cor method was used to mask for clouds and cloud shadows, and the Sentinel 2 Level-1C data was sourced from the Copernicus Open Access Hub. To estimate the purpose of classification, stochastic forest classification method was used to predict classification accuracy. Using seven types of crops, the classification map of the 2020 growth season in Tarom was prepared using 10-day Sentinel 2 smooth mosaic NDVI and 12-day Sentinel 1 back mosaic. Kappa coefficient of 0.75 and a maximum accuracy of 85% were reported in this study. To achieve maximum classification accuracy, it is recommended to use a combination of radar and optical data, as this combination increases the chances of examining the details compared to the single-sensor classification method and achieves more reliable information.

KappaMask: AI-Based Cloudmask Processor for Sentinel-2

The Copernicus Sentinel-2 mission operated by the European Space Agency (ESA) provides comprehensive and continuous multi-spectral observations of all the Earth’s land surface since mid-2015. Clouds and cloud shadows significantly decrease the usability of optical satellite data, especially in agricultural applications; therefore, an accurate and reliable cloud mask is mandatory for effective EO optical data exploitation. During the last few years, image segmentation techniques have developed rapidly with the exploitation of neural network capabilities. With this perspective, the KappaMask processor using U-Net architecture was developed with the ability to generate a classification mask over northern latitudes into the following classes: clear, cloud shadow, semi-transparent cloud (thin clouds), cloud and invalid. For training, a Sentinel-2 dataset covering the Northern European terrestrial area was labelled. KappaMask provides a 10 m classification mask for Sentinel-2 Level-2A (L2A) and Level-1C (L1C) products. The total dice coefficient on the test dataset, which was not seen by the model at any stage, was 80% for KappaMask L2A and 76% for KappaMask L1C for clear, cloud shadow, semi-transparent and cloud classes. A comparison with rule-based cloud mask methods was then performed on the same test dataset, where Sen2Cor reached 59% dice coefficient for clear, cloud shadow, semi-transparent and cloud classes, Fmask reached 61% for clear, cloud shadow and cloud classes and Maja reached 51% for clear and cloud classes. The closest machine learning open-source cloud classification mask, S2cloudless, had a 63% dice coefficient providing only cloud and clear classes, while KappaMask L2A, with a more complex classification schema, outperformed S2cloudless by 17%.

Real-Time Automated Classification of Sky Conditions Using Deep Learning and Edge Computing

The radiometric quality of remotely sensed imagery is crucial for precision agriculture applications because estimations of plant health rely on the underlying quality. Sky conditions, and specifically shadowing from clouds, are critical determinants in the quality of images that can be obtained from low-altitude sensing platforms. In this work, we first compare common deep learning approaches to classify sky conditions with regard to cloud shadows in agricultural fields using a visible spectrum camera. We then develop an artificial-intelligence-based edge computing system to fully automate the classification process. Training data consisting of 100 oblique angle images of the sky were provided to a convolutional neural network and two deep residual neural networks (ResNet18 and ResNet34) to facilitate learning two classes, namely (1) good image quality expected, and (2) degraded image quality expected. The expectation of quality stemmed from the sky condition (i.e., density, coverage, and thickness of clouds) present at the time of the image capture. These networks were tested using a set of 13,000 images. Our results demonstrated that ResNet18 and ResNet34 classifiers produced better classification accuracy when compared to a convolutional neural network classifier. The best overall accuracy was obtained by ResNet34, which was 92% accurate, with a Kappa statistic of 0.77. These results demonstrate a low-cost solution to quality control for future autonomous farming systems that will operate without human intervention and supervision.