Cloud Detection Research Articles

Cloud detection of high-resolution remote sensing image based on improved U-Net

Cloud detection of high-resolution remote sensing image based on improved U-Net

A cloud detection neural network for above-aircraft clouds using airborne cameras

For aerosol, cloud, land, and ocean remote sensing, the development of accurate cloud detection methods, or cloud masks, is extremely important. For airborne passive remotesensing, it is also important to identify when clouds are above the aircraft since their presence contaminates the measurements of nadir-viewing passive sensors. We describe the development of a camera-based approach to detecting clouds above the aircraft via a convolutional neural network called the cloud detection neural network (CDNN). We quantify the performance of this CDNN using human-labeled validation data where we report 96% accuracy in detecting clouds in testing datasets for both zenith viewing and forward-viewing models. We present results from the CDNN based on airborne imagery from the NASA Aerosol Cloud meteorology Interactions oVer the western Atlantic Experiment (ACTIVATE) and the Clouds, Aerosol, and Monsoon Processes Philippines Experiment (CAMP2Ex). We quantify the ability of the CDNN to identify the presence of clouds above the aircraft using a forward-looking camera mounted inside the aircraft cockpit compared to the use of an all-sky upward-looking camera that is mounted outside the fuselage on top of the aircraft. We assess our performance by comparing the flight-averaged cloud fraction of zenith and forward CDNN retrievals with that of the prototype hyperspectral total-diffuse Sunshine Pyranometer (SPN-S) instrument’s cloud optical depth data. A comparison of the CDNN with the SPN-S on time-specific intervals resulted in 93% accuracy for the zenith viewing CDNN and 84% for the forward-viewing CDNN. The comparison of the CDNNs with the SPN-S on flight-averaged cloud fraction resulted in an agreement of .15 for the forward CDNN and .07 for the zenith CDNN. For CAMP2Ex, 53% of flight dates had above-aircraft cloud fraction above 50%, while for ACTIVATE, 52% and 54% of flight dates observed above-aircraft cloud fraction above 50% for 2020 and 2021, respectively. The CDNN enables cost-effective detection of clouds above the aircraft using an inexpensive camera installed in the cockpit for airborne science research flights where there are no dedicated upward-looking instruments for cloud detection, the installation of which requires time-consuming and expensive aircraft modifications, in addition to added mission cost and complexity of operating additional instruments.

Siamese KPConv: 3D multiple change detection from raw point clouds using deep learning

This study is concerned with urban change detection and categorization in point clouds. In such situations, objects are mainly characterized by their vertical axis, and the use of native 3D data such as 3D Point Clouds (PCs) is, in general, preferred to rasterized versions because of significant loss of information implied by any rasterization process. Yet, for obvious practical reasons, most existing studies only focus on 2D images for change detection purpose. In this paper, we propose a method capable of performing change detection directly within 3D data. Despite recent deep learning developments in remote sensing, to the best of our knowledge there is no such method to tackle multi-class change segmentation that directly processes raw 3D PCs. Thereby, based on advances in deep learning for change detection in 2D images and for analysis of 3D point clouds, we propose a deep Siamese KPConv network that deals with raw 3D PCs to perform change detection and categorization in a single step. Experimental results are conducted on synthetic and real data of various kinds (LiDAR, multi-sensors). Tests performed on simulated low density LiDAR and multi-sensor datasets show that our proposed method can obtain up to 80% of mean of IoU over classes of changes, leading to an improvement ranging from 10% to 30% over the state-of-the-art. A similar range of improvements is attainable on real data. Then, we show that pre-training Siamese KPConv on simulated PCs allows us to greatly reduce (more than 3,000×) the annotations required on real data. This is a highly significant result to deal with practical scenarios. Finally, an adaptation of Siamese KPConv is realized to deal with change classification at PC scale. Our network overtakes the current state-of-the-art deep learning method by 23% and 15% of mean of IoU when assessed on synthetic and public Change3D datasets, respectively. The code is available at the following link: https://github.com/IdeGelis/torch-points3d-SiameseKPConv.

A Robust Sphere Detection in a Realsense Point Cloud by USING Z-Score and RANSAC

Three-dimensional vision cameras, such as RGB-D, use 3D point cloud to represent scenes. File formats as XYZ and PLY are commonly used to store 3D point information as raw data, this information does not contain further details, such as metadata or segmentation, for the different objects in the scene. Moreover, objects in the scene can be recognized in a posterior process and can be used for other purposes, such as camera calibration or scene segmentation. We are proposing a method to recognize a basketball in the scene using its known dimensions to fit a sphere formula. In the proposed cost function we search for three different points in the scene using RANSAC (Random Sample Consensus). Furthermore, taking into account the fixed basketball size, our method differentiates the sphere geometry from other objects in the scene, making our method robust in complex scenes. In a posterior step, the sphere center is fitted using z-score values eliminating outliers from the sphere. Results show our methodology converges in finding the basketball in the scene and the center precision improves using z-score, the proposed method obtains a significant improvement by reducing outliers in scenes with noise from 1.75 to 8.3 times when using RANSAC alone. Experiments show our method has advantages when comparing with novel deep learning method.

PATMOS-x Version 6.0: 40 Years of Merged AVHRR and HIRS Global Cloud Data

Abstract A new version of the PATMOS-x multidecadal cloud record, version 6.0, has been produced and is available from the NOAA National Centers for Environmental Information. A description of the processes and methods used for generating the dataset are presented, with a focus on the differences between version 6.0 and the previous version of PATMOS-x, version 5.3. The new version appears both to be more stable, with less intersatellite variability, and to have more consistent polar cloud detection, phase distribution, and cloud-top height distribution when compared against the MODIS EOS record. Improvements in consistency and performance are attributed to the addition of multidimensional variables for cloud detection, constraining cloud retrievals to radiometric bands available throughout the record, and the addition of data from the HIRS instrument. Significance Statement The PATMOS-x project produces multidecadal cloudiness records from polar-orbiting satellites. Version 6.0 combines imager and sounder data from 15 satellites and shows significant improvements in accuracy and stability.

An integrated approach to reconstructing snow cover under clouds and cloud shadows on Sentinel-2 Time-Series images in a mountainous area

Snow is an important type of surface cover and an important freshwater resource in the arid regions of northwest China, and optical satellites are an important means for monitoring variation in snow cover. However, the radiation values measured by satellite sensors are affected by cloud cover and cloud shadows; thus, the reconstruction of snow cover under clouds and cloud shadows becomes a bottleneck problem especially in mountains areas. Based on the digital elevation model (DEM), in this paper, an integrated method of snow cover reconstruction on Sentinel-2 images is created by combining the improved SNOWL (snow line) algorithm considering unstable snow cover areas, Fmask cloud detection algorithms, and the Sen2Cor clear sky snow cover detection tool with a geospatial processing platform, GEE (Google Earth Engine). Considering the Babao River Basin as the study area, 474 Sentinel-2 scenes (10 m) were selected to obtain the snow cover spatiotemporal variations of 158 days from November 2016 to March 2021. Compared with the high-resolution GF-2 satellite images, the integrated method can extract snow under clouds and cloud shadows well, and the improved SNOWL algorithm improves the overall accuracy from 66.05 % to 84.26 %. The experimental results show that approximately half of the Babao River Basin is unstable snow cover, and areas above 3977 m are covered with snow for at least half a year.

Pillar-Based 3D Object Detection from Point Cloud with Multiattention Mechanism

Object detection in point clouds is a critical component in most autonomous driving systems. In this paper, in order to improve the effectiveness of image feature extraction and the accuracy of detection of point clouds, a pillar-based 3D point cloud object detection algorithm with multiattention mechanism is proposed, which includes three attention mechanisms SOCA, SOPA, and SAPI. The results show that the recognition accuracy of the optimized algorithm for cars, pedestrians, and cyclists on KITTI dataset is significantly improved on the detection benchmarks of BEV and 3D. Despite using only LiDAR, our algorithm outperforms PointPillars, which is one of the state-of-the-art algorithms for 3D object detection, with respect to both 3D and BEV view KITTI benchmarks while maintaining a relatively competitive speed.

Assessment of cloudless-to-cloud transition zone from downwelling longwave irradiance measurements

The transition zone has been defined as corresponding to atmospheric conditions between cloud-free (albeit containing a suspension of dry particles, i.e. aerosol) and clouds (air containing a suspension of highly hydrated particles, i.e. droplets or ice crystals). Since clouds significantly affect the infrared radiation reaching the Earth's surface, broadband infrared measurements, which are usually performed with pyrgeometers, are implicated in some of the methods proposed for detecting clouds. The present work uses one of these methods to characterize the transition between cloud-free and cloudy conditions. Two years of downwelling longwave irradiance measured with a pyrgeometer located in Girona (in the northeast of the Iberian Peninsula) have been used to determine the values of a cloud detection index. The method must be previously tuned against a set of selected cloud-free cases to detect cloudiness to a certain confidence level. The effect of changing the detection confidence level was studied and used to derive the occurrence of transition conditions. The performance of the cloudiness detection method, with the support of observations by broadband shortwave instruments, ceilometer retrievals and images captured by hemispherical camera, was then analyzed for some short time periods. The duration of the transition periods is presented here, showing that 41% are very short (10 min, the temporal resolution of the detection method), that the median of the distribution is about 20 min, and that, although periods of more than one hour are observed, longer durations steadily decrease in frequency. It was found that 10–15% of the observations included may correspond to the transition zone, thus strengthening the estimations from other shortwave-based methods that are only suitable for daylight periods.

Cloud Removal from Satellite Images Using a Deep Learning Model with the Cloud-Matting Method

Clouds seriously limit the application of optical remote sensing images. In this paper, we remove clouds from satellite images using a novel method that considers ground surface reflections and cloud top reflections as a linear mixture of image elements from the perspective of image superposition. We use a two-step convolutional neural network to extract the transparency information of clouds and then recover the ground surface information of thin cloud regions. Given the poor balance of the generated samples, this paper also improves the binary Tversky loss function and applies it on multi-classification tasks. The model was validated on the simulated dataset and ALCD dataset, respectively. The results show that this model outperformed other control group experiments in cloud detection and removal. The model better locates the clouds in images with cloud matting, which is built based on cloud detection. In addition, the model successfully recovers the surface information of the thin cloud region when thick and thin clouds coexist, and it does not damage the original image’s information.

Towards Edge-Precise Cloud and Shadow Detection on the GaoFen-1 Dataset: A Visual, Comprehensive Investigation

Remote sensing images are usually contaminated by opaque cloud and shadow regions when acquired, and therefore cloud and shadow detection arises as one of the essential prerequisites for restoration and prediction of the objects of interest underneath, which are required by further processing and analysis. Cutting-edge, learning-based segmentation techniques, given a well-labeled, sufficient sample set, are significantly developed for such a detection issue and can already achieve region-accurate or even pixel-precise performance. However, it may possibly be problematic to attempt to apply the sophisticated segmentation techniques to label-free datasets in a straightforward way, more specifically, to the remote sensing data generated by the Chinese domestic satellite GaoFen-1. We wish to partially address such a segmentation problem from a practical perspective rather than in a conceptual way. This can be performed by considering a hypothesis that a segmentor, which is sufficiently trained on the well-labeled samples of common bands drawn from a source dataset, can be directly applicable to the custom, band-consistent test cases from a target set. Such a band-consistent hypothesis allows us to present a straightforward solution to the GaoFen-1 segmentation problem by treating the well-labeled Landsat 8 Operational Land Imager dataset as the source and by selecting the fourth, the third, and the second bands, also known as the false-color bands, to construct the band-consistent samples and cases. Furthermore, we attempt to achieve edge-refined segmentation performance on the GaoFen-1 dataset by adopting our prior Refined UNet and v4. We finally verify the effectiveness of the band-consistent hypothesis and the edge-refined approaches by performing a relatively comprehensive investigation, including visual comparisons, ablation experiments regarding bilateral manipulations, explorations of critical hyperparameters within our implementation of the conditional random field, and time consumption in practice. The experiments and corresponding results show that the hypothesis of selecting the false-color bands is effective for cloud and shadow segmentation on the GaoFen-1 data, and that edge-refined segmentation performance of our Refined UNet and v4 can be also achieved.

Volcanic Cloud Detection and Retrieval Using Satellite Multisensor Observations

Satellite microwave (MW) and millimetre-wave (MMW) passive sensors can be used to detect volcanic clouds because of their sensitivity to larger volcanic particles (i.e., size bigger than 20 µm). In this work, we combine the MW-MMW observations with thermal-infrared (TIR) radiometric data from the Low Earth Orbit (LEO) spectroradiometer to have a complete characterisation of volcanic plumes. We describe new physical-statistical methods, which combine machine learning techniques, aimed at detecting and retrieving volcanic clouds of two highly explosive eruptions: the 2014 Kelud and 2015 Calbuco test cases. For the detection procedure, we compare the well-known split-window methods with a machine learning algorithm named random forest (RF). Our work highlights how the machine learning method is suitable to detect volcanic clouds using different spectral signatures without fixing a threshold. Moreover, the RF model allows images to be automatically processed with promising results (90% of the area correctly identified). For the retrieval procedure of the mass of volcanic particles, we consider two methods, one based on the maximum likelihood estimation (MLE) and one using the neural network (NN) architecture. Results show a good comparison of the mass obtained using the MLE and NN methods for all the analysed bands. Summing the MW-MMW and TIR estimates, we obtain the following masses: 1.11 ± 0.40 × 1011 kg (MLE method) and 1.32 ± 0.47 × 1011 kg (NN method) for Kelud; 4.48 ± 1.61 × 1010 kg (MLE method) and 4.32 ± 1.56 × 1010 kg (NN method) for Calbuco. This work shows how machine learning techniques can be an effective tool for volcanic cloud detection and how the synergic use of the TIR and MW-MMW observations can give more accurate estimates of the near-source volcanic clouds.

CIFNet: context information fusion network for cloud and cloud shadow detection in optical remote sensing imagery

Accurately and automatically detecting cloud and cloud shadow is one of the key steps in the analysis of optical remote sensing imagery. Currently, most cloud and cloud shadow detection methods are prone to false detection, and some clouds and cloud shadows may be missing from the detection results. Due to the lack of contextual information extraction and fusion capabilities, the accuracy of these cloud detection algorithms cannot be guaranteed. This article proposes a deep learning-based convolutional neural network context information fusion network (named CIFNet) to solve the problem of cloud and cloud shadow detection on Gaofen-1 wide field of view optical remote sensing imageries. First, we preprocess the dataset and make the corresponding labels. Next, we design a Resblock-cloud to capture global and local features and prevent the network from degrading. Then, we utilize a global context fusion module to fuse different levels of global context information by dense skip-connections. Finally, a multiscale context fusion module is designed to extract multiscale contextual relations between cloud and cloud shadow. Experimental results show that the proposed CIFNet method can obtain better cloud and cloud shadow boundaries in complex situations and therefore outperforms the competing methods.

An efficient method using the fusion of deep convolutional neural network features for cloud detection using Landsat-8 OLI spectral bands

An efficient method using the fusion of deep convolutional neural network features for cloud detection using Landsat-8 OLI spectral bands

Radiance consistency and deviation characteristics in the regions of overlap between three new-generation GEO meteorological satellites

ABSTRACT The combination of multi-source Geostationary Earth Orbit (GEO) satellites can realize simultaneous global observation with high frequency, which is significant to fine weather forecast and global climate change research. Here the radiance consistency and deviation characterization in the overlapping regions are analysed for three new generation GEO satellites, including the Advanced Himawari Imager (AHI) onboard the Himawari-8 satellite, the Advanced Baseline Imager (ABI) onboard the Geostationary Operational Environmental Satellite-17 (GOES-17) and the Advanced Geostationary Radiation Imager (AGRI) onboard FengYun satellite-4A (FY-4A). The result shows that the radiance has higher consistency for visible and near-infrared (VNIR) channels with correlation coefficient r > 0.98 than infrared (IR) channels with r > 0.90 in the overlapping regions of different GEO satellites. For FY-4A & Himawari-8, the average percentage deviation (APD) of VNIR channels is 5.54%–9.03%. The brightness temperature (BT) bias of IR channels is 1.16 K-2.50 K. For GOES-17 & Himawari-8, the APD of VNIR channels is 4.48%–6.71%, and the BT bias of IR channels is 0.96 K −2.76 K. Solar-viewing geometries significantly influence the radiance deviation in the over regions. The future work is to design a radiance fusion method and achieve cloud detection in ample space with high frequency by combining multi-source GEO satellites.

Image Compression Network Structure Based on Multiscale Region of Interest Attention Network

In this study, we proposed a region of interest (ROI) compression algorithm under the deep learning self-encoder framework to improve the reconstruction performance of the image and reduce the distortion of the ROI. First, we adopted a remote sensing image cloud detection algorithm for detecting important targets in images, that is, separating the remote sensing background from important regions in remote sensing images and then determining the target regions because most traditional ROI-based image compression algorithms utilize the manual labeling of the ROI to achieve region separation in images. We designed a multiscale ROI self-coding network from coarse to fine with a hierarchical super priority layer to synthesize images to reduce the spatial redundancy more effectively, thus greatly improving the distortion rate performance of image compression. By using a spatial attention mechanism for the ROI in the image compression network, we achieved better compression performance.

The Graph Neural Network Detector Based on Neighbor Feature Alignment Mechanism in LIDAR Point Clouds

Three-dimensional (3D) object detection has a vital effect on the environmental awareness task of autonomous driving scenarios. At present, the accuracy of 3D object detection has significant improvement potential. In addition, a 3D point cloud is not uniformly distributed on a regular grid because of its disorder, dispersion, and sparseness. The strategy of the convolution neural networks (CNNs) for 3D point cloud feature extraction has the limitations of potential information loss and empty operation. Therefore, we propose a graph neural network (GNN) detector based on neighbor feature alignment mechanism for 3D object detection in LiDAR point clouds. This method exploits the structural information of graphs, and it aggregates the neighbor and edge features to update the state of vertices during the iteration process. This method enables the reduction of the offset error of the vertices, and ensures the invariance of the point cloud in the spatial domain. For experiments performed on the KITTI public benchmark, the results demonstrate that the proposed method achieves competitive experimental results.

A Novel Algorithm of Haze Identification Based on FY3D/MERSI-II Remote Sensing Data

Since 2013, frequent haze pollution events in China have been attracting public attention, generating a demand to identify the haze areas using satellite observations. Many studies of haze recognition algorithms are based on observations from space-borne imagers, such as the Moderate Resolution Imaging Spectroradiometer (MODIS), the Visible Infrared Imaging Radiometer Suite (VIIRS) and the Advanced Himawari Imager (AHI). Since the haze pixels are frequently misidentified as clouds in the official cloud detection products, these algorithms mainly focus on recovering them from clouds. There are just a few studies that provide a more precise distinction between haze and clear pixels. The Medium Resolution Imaging Spectrometer II (MERSI-II), the imager aboard the FY-3D satellite, has similar bands to those of MODIS, hence, it appears to have equivalent application potential. This study proposes a novel MERSI haze mask (MHAM) algorithm to directly categorize haze pixels in addition to cloudy and clear ones. This algorithm is based on the fact that cloudy and clear pixels exhibit opposing visible channel reflectance and infrared channel brightness temperature characteristics, and clear pixels are relative brighter, and as well as this, there is a positive difference between their apparent reflectance values, at 0.865 μm and 1.64 μm, respectively, over bright surfaces. Compared with the Aqua/MODIS and MERSI-II official cloud detection products, these two datasets treat the dense aerosol loadings as certain clouds, possible clouds and possible clear pixels, and they treat distinguished light or moderate haze as possible clouds, possible clear pixels and certainly clear pixels, while the novel algorithm is capable of demonstrating the haze region’s boundary in a manner that is more substantially consistent with the true color image. Using the PM2.5 (particle matter with a diameter that is less than 2.5 μm) data monitored by the national air quality monitoring stations as the test source, the results indicated that when the ground-based PM2.5 ≥ 35 μg/cm3 is considered to be haze days, the samples with the recognition rate that is higher than 85% accounted for 72.22% of the total samples. When PM2.5 ≥ 50 μg/cm3 is considered as haze days, 83.33% of the samples had an identification rate that was higher than 85%. A cross-comparison with similar research methods showed that the method proposed in this study had better sensitivity to bright surface clear and haze areas. This study will provide a haze mask for subsequent quantitative inversion of aerosol characteristics, and it will further exert the application benefits of MERSI-II instrument aboard on FY3D satellite.

A Neural Network-based Cloud Mask for PREFIRE and Evaluation with Simulated Observations

Abstract The Polar Radiant Energy in the Far InfraRed Experiment (PREFIRE) will fill a gap in our understanding of polar processes and the polar climate by offering widespread, spectrally-resolved measurements through the Far InfraRed (FIR) with two identical CubeSat spacecraft. While the polar regions are typically difficult for skillful cloud identification due to cold surface temperatures, the reflection by bright surfaces, and frequent temperature inversions, the inclusion of the FIR may offer increased spectral sensitivity, allowing for the detection of even thin ice clouds. This study assesses the potential skill, as well as limitations, of a neural network-based cloud mask using simulated spectra mimicking what the PREFIRE mission will capture. Analysis focuses on the polar regions. Clouds are found to be detected approximately 90% of time using the derived neural network. The NN’s assigned confidence for whether a scene is ‘clear’ or ‘cloudy’ proves to be a skillful way in which quality flags can be attached to predictions. Clouds with higher cloud top heights are typically more easily detected. Low-altitude clouds over polar surfaces, which are the most difficult for the NN to detect, are still detected over 80% of the time. The FIR portion of the spectrum is found to increase the detection of clear scenes and increase mid-to-high altitude cloud detection. Cloud detection skill improves through the use of the overlapping fields of view produced by the PREFIRE instrument’s sampling strategy. Overlapping fields of view increase accuracy relative to the baseline NN while simultaneously predicting on a sub-FOV scale.

AERF: Adaptive ensemble random fuzzy algorithm for anomaly detection in cloud computing

The abnormality of system behavior is inevitable in cloud computing because of its complexity and scale. How to perform anomaly detection on the system’s operating data to discover abnormal behavior has become a popular research field. However, anomaly detection is a challenging research problem because data in cloud computing scenarios is continuous, imbalanced, and cannot be acquired at once. In this work, an adaptive ensemble random fuzzy (AERF) algorithm is proposed for anomaly detection in cloud computing systems. The random fuzzy rule-based method in the AERF selects samples randomly to enhance the diversity of base classifiers for dealing with disturbances caused by abnormal sample distribution effectively. A dynamic weighting strategy is used for fuzzy classifier ensembles to improve processing efficiency and anomaly detection accuracy. Experimental results show that the AERF yields 10% higher AUC and G-mean than existing methods on SMD and EMOSCloud datasets and 20% higher AUC and G-mean on five benchmarking datasets. The AERF yields 20% higher F1 than existing methods on EMOSCloud dataset. For all datasets, the AERF yields 50% faster training time than other methods.

Satellite Image Compression Guided by Regions of Interest

Small satellites empower different applications for an affordable price. By dealing with a limited capacity for using instruments with high power consumption or high data-rate requirements, small satellite missions usually focus on specific monitoring and observation tasks. Considering that multispectral and hyperspectral sensors generate a significant amount of data subjected to communication channel impairments, bandwidth constraint is an important challenge in data transmission. That issue is addressed mainly by source and channel coding techniques aiming at an effective transmission. This paper targets a significant further bandwidth reduction by proposing an on-the-fly analysis on the satellite to decide which information is effectively useful before coding and transmitting. The images are tiled and classified using a set of detection algorithms after defining the least relevant content for general remote sensing applications. The methodology makes use of the red-band, green-band, blue-band, and near-infrared-band measurements to perform the classification of the content by managing a cloud detection algorithm, a change detection algorithm, and a vessel detection algorithm. Experiments for a set of typical scenarios of summer and winter days in Stockholm, Sweden, were conducted, and the results show that non-important content can be identified and discarded without compromising the predefined useful information for water and dry-land regions. For the evaluated images, only 22.3% of the information would need to be transmitted to the ground station to ensure the acquisition of all the important content, which illustrates the merits of the proposed method. Furthermore, the embedded platform's constraints regarding processing time were analyzed by running the detection algorithms on Unibap's iX10-100 space cloud platform.