Cloud Detection Research Articles

Clouds accreting from the IGM are not able to feed the star formation of low-redshift disc galaxies

ABSTRACT Galactic halos accrete material from the intergalactic medium (IGM) and part of this accretion is expected to be in the form of cool (T ∼ 104 K) gas. A signature of this process could reside in the detection of numerous clouds in the circumgalactic medium (CGM). However, whether this material is able to accrete onto the galaxies and feed their star formation or, instead, evaporates into the CGM hot phase (corona, T ∼ 106 K), is not yet understood. Here, we investigate the evolution of cool CGM clouds accreted from the IGM and falling through the hot corona of low-redshift disc galaxies, using 3D high-resolution hydrodynamical simulations. We include the effects of gravity due to the dark matter halo, isotropic thermal conduction, radiative cooling, and an ionizing UV background. We explored different values of parameters such as the halo mass, coronal mass, initial cloud velocity and strength of the thermal conduction. We find that the clouds lose the vast majority of their mass at distances larger than half of the galaxy virial radius and are completely dissolved in the corona before reaching the central galaxy. Resolving the Field length with at least 5–7 cells is crucial to correctly capture the evolution of the infalling cool gas. Our results indicate that cool IGM accretion can not feed star formation in z ∼ 0 star-forming galaxies in halos with masses of 1011.9 M⊙ or above. This suggests that present-day massive star-forming galaxies can sustain their star formation only via the spontaneous or induced cooling of their hot corona.

Development of a compact space-borne Lidar for atmospheric aerosol and cloud detection

Space-borne lidar is preferred for observing different characteristic aerosols that should be distributed vertically in every corner of the world. Therefore, a compact space-borne lidar (CSBL) with a high repetition rate laser and photon counting mode is developed to meet the detection requirements in this paper. The detection principle is described in detail, and the design parameters of the CSBL are simulated. The CSBL is assembled and calibrated in its entirety after the development of a long-life, high-beam quality laser and a highly sensitive, quantum-efficient single-photon counting core component. Moreover, after ground and flight tests, the CSBL completed the integrated inversion of atmospheric aerosols and clouds, and it can remotely detect global atmospheric aerosols and cloud spatial distribution structures.

"Detecting and Removing Clouds Affected Regions from Satellite Images Using Deep Learning"

Deep Learning is becoming a very popular tool for generating and reconstructing images. Research has shown that deep learning algorithms can perform cutting-edge restoration tasks for various types of images. The performance of these algorithms can be achieved by training Deep Convolutional Neural Networks (DCNNs) with data from a large sample size. The processing of high-resolution satellite imagery becomes difficult when there are only a few images in a dataset. An approach based on the intrinsic properties of Deep Convolutional Neural Networks (DCNNs) is presented in this paper for the detection and removal of clouds from remote sensing images without any prior training. Our results demonstrated that the algorithm we used performed well when compared to trained algorithms.

AI-powered fault detection and mitigation in cloud computing infrastructures

In the evolving landscape of cloud computing, ensuring the reliability and resilience of cloud infrastructures has become paramount. This study investigates the application of artificial intelligence (AI) for fault detection and mitigation in cloud computing environments. Traditional fault detection methods often struggle to cope with the dynamic and complex nature of modern cloud infrastructures, leading to suboptimal performance and increased downtime. Our research leverages advanced AI algorithms to identify and mitigate faults in real-time, thereby enhancing system reliability and performance. The primary objectives of this research are to develop and validate AI models that can accurately detect faults in cloud computing infrastructures, evaluate the effectiveness of these models in mitigating detected faults, and compare their performance against traditional fault detection and mitigation techniques. We aim to demonstrate that AI-powered solutions can significantly reduce the incidence and impact of faults in cloud systems. Key findings of the study reveal that AI-powered fault detection models exhibit superior accuracy and speed compared to conventional methods. The implementation of these models resulted in a marked improvement in fault mitigation, reducing system downtime and enhancing overall service quality. Additionally, the study identifies specific AI techniques, such as machine learning and deep learning, that are particularly effective in this context.

Research on infrared hyperspectral remote sensing cloud detection method based on deep learning

ABSTRACT Infrared hyperspectral is susceptible to clouds, and accurately identifying whether the hyperspectral infrared sounder data is polluted by clouds is of great significance for numerical weather prediction and atmospheric parameter inversion. Since the complex spectral characteristics of clouds, the existing spectral threshold methods and machine learning methods have the difficulties of undetermined threshold and clear field of view (FOV) missed and false detections. In order to improve the cloud recognition accuracy of infrared hyperspectral data, three end-to-end cloud detection models combining deep neural network (DNN) and convolutional neural network (CNN) and long short-term memory network (LSTM) are proposed. In this paper, taking the High Spectral Infrared Atmospheric Sounder (HIRAS) equipped with Fengyun-3D (FY-3D) satellite as the research object, based on the same platform Moderate Resolution Spectral Imager-II (MERSI-II) cloud mask (CLM) product, the HIRAS Cloud dataset is established, and the accuracy test and qualitative analysis are carried out by using the test datasets and Typhoon Siamba, July 3, 2022, as well as the earth observation scene under the conditions of ice and snow surface. The test datasets analysis results show that the cloud detection accuracy of CNN and CNN-LSTM model is stable at 0.96, and the false alarm rate of cloud is 0.035 and 0.036, respectively, and the detection ability of DNN model is slightly inferior to the former two in the same hidden layer, with an accuracyof 0.94. In further qualitative research, we found that the CNN-LSTM model has high accuracy and robustness in infrared hyperspectral cloud detection, and the detection results in a variety of surface scenarios are consistent with the actual situation of whether clouds occur in the FOV of the instrument. Compared with CLM products, it can better identify clear ocean scenes, and provide fast and efficient cloud detection reference for data assimilation systems.

Gated aggregation network for cloud detection in remote sensing image

Gated aggregation network for cloud detection in remote sensing image

Multi-Supervised Feature Fusion Attention Network for Clouds and Shadows Detection

Cloud and cloud shadow detection are essential in remote sensing imagery applications. Few semantic segmentation models were designed specifically for clouds and their shadows. Based on the visual and distribution characteristics of clouds and their shadows in remote sensing imagery, this paper provides a multi-supervised feature fusion attention network. We design a multi-scale feature fusion block (FFB) for the problems caused by the complex distribution and irregular boundaries of clouds and shadows. The block consists of a fusion convolution block (FCB), a channel attention block (CAB), and a spatial attention block (SPA). By multi-scale convolution, FCB reduces excessive semantic differences between shallow and deep feature maps. CAB focuses on global and local features through multi-scale channel attention. Meanwhile, it fuses deep and shallow feature maps with non-linear weighting to optimize fusion performance. SPA focuses on task-relevant areas through spatial attention. With the three blocks above, FCB alleviates the difficulties of fusing multi-scale features. Additionally, it makes the network resistant to background interference while optimizing boundary detection. Our proposed model designs a class feature attention block (CFAB) to increase the robustness of cloud detection. The network achieves good performance on the self-made cloud and shadow dataset. This dataset is taken from Google Earth and contains remote sensing imagery from several satellites. The proposed model achieved a mean intersection over union (MIoU) of 94.10% on our dataset, which is 0.44% higher than the other models. Moreover, it shows high generalization capability due to its superior prediction results on HRC_WHU and SPARCS datasets.

Updated observations of clouds by MODIS for global model assessment

Abstract. This paper describes a new global dataset of cloud properties observed by MODIS relying on the current (collection 6.1) processing of MODIS data and produced to facilitate comparison with results from the MODIS observational proxy used in climate models. The dataset merges observations from the two MODIS instruments into a single netCDF file. Statistics (mean, standard deviation, and number of observations) are accumulated over daily and monthly timescales on an equal-angle grid for viewing and illumination geometry, cloud detection, cloud-top pressure, and cloud properties (optical thickness, effective particle size, and water path) partitioned by thermodynamic phase and an assessment as to whether the underlying observations come from fully or partly cloudy pixels. Similarly partitioned joint histograms are available for (1) optical thickness and cloud-top pressure, (2) optical thickness and particle size, and (3) cloud water path and particle size. Differences with standard data products, caveats for data use, and guidelines for comparison to the MODIS simulator are described. Data are available on daily (https://doi.org/10.5067/MODIS/MCD06COSP_D3_MODIS.062; NASA, 2022b) and monthly (https://doi.org/10.5067/MODIS/MCD06COSP_M3_MODIS.062; NASA, 2022a) timescales.

NSKY-CD: A System for Cloud Detection Based on Night Sky Brightness and Sky Temperature

Cloud cover is important meteorological information that still requires expensive equipment to be monitored from the ground, especially at night. The use of artificial lights at night causes light pollution, and clouds amplify this by reflecting light downward. In addition, cloud thermal radiation emissions affect sky temperature. In this study, we describe a system (NSKY-CD) that can be used to detect clouds at night since it integrates a sky quality meter that measures night sky brightness (NSB) and an air temperature and an infrared temperature sensor that measure sky temperature. We defined a cloud detection method based on fixed threshold values determined with two different procedures that we called ‘optimal’ and ‘antimode’. We then quantitatively assessed the performance of these methods in detecting the presence or absence of clouds in the urban area of Florence during two full moon cycles. Accuracy for the ‘optimal’ method varied between 87% and 91%, while for the ‘antimode’ method, it varied between 86% and 89%. Our results suggest that the two parameters are complementary since NSB has a better performance on moonless nights, and the difference between air temperature and sky temperature has a better performance on moonlit nights. Our method can also be used to analyze historical series of NSB to estimate cloud presence backwards, thus providing important information for meteorological, environmental and astronomical studies.

Global Cloudiness and Cloud Top Information from AVHRR in the 42-Year CLARA-A3 Climate Data Record Covering the Period 1979–2020

This paper investigates the quality of global cloud fraction and cloud-top height products provided by the third edition of the CM SAF cLoud, Albedo and surface RAdiation dataset from the AVHRR data (CLARA-A3) climate data record (CDR) produced by the EUMETSAT Climate Monitoring Satellite Application Facility (CM SAF). Compared with with CALIPSO–CALIOP cloud lidar data and six other cloud CDRs, including the predecessor CLARA-A2, CLARA-A3 has improved cloud detection, especially over ocean surfaces, and improved geographical variation and cloud detection efficiency. In addition, CLARA-A3 exhibits remarkable improvements in the accuracy of its global cloud-top height measurements. For example, in tropical regions, previous underestimations for high-level clouds are reduced by more than 2 km. By taking advantage of more realistic descriptions of global cloudiness, this study attempted to estimate trends in the observable fraction of low-level clouds, acknowledging their importance in producing a net climate cooling effect. The results were generally inconclusive in the tropics, mainly due to the interference of El Nino modes during the period under study. However, the analysis found small negative trends over oceanic surfaces outside the core tropical region. Further studies are needed to verify the significance of these results.

Simulation of Compact Spaceborne Lidar with High-Repetition-Rate Laser for Cloud and Aerosol Detection under Different Atmospheric Conditions

To provide references for the design of the lab’s upcoming prototype of the compact spaceborne lidar with a high-repetition-rate laser (CSLHRL), in this paper, the detection signal of spaceborne lidar was simulated by the measured signal of ground-based lidar, and then, the detection capability of spaceborne lidar under different atmospheric conditions was evaluated by means of the signal-to-noise ratio (SNR), volume depolarization ratio (VDR) and attenuated color ratio (ACR). Firstly, the Fernald method was used to invert the optical parameters of cloud and aerosol with the measured signal of ground-based lidar. Secondly, the effective signal of the spaceborne lidar was simulated according to the known atmospheric optical parameters and the parameters of the spaceborne lidar system. Finally, by changing the cumulative laser pulse number and atmospheric conditions, a simulation was carried out to further evaluate the detection performance of the spaceborne lidar, and some suggestions for the development of the system are given. The experimental results showed that the cloud layer and aerosol layer with an extinction coefficient above 0.3 km−1 could be easily obtained when the laser cumulative pulse number was 1000 and the vertical resolution was 15 m at night; the identification of moderate pollution aerosols and thick clouds could be easily identified in the daytime when the laser cumulative pulse number was 10,000 and the vertical resolution was 120 m.

Cloud mask algorithm from the EarthCARE Multi-Spectral Imager: the M-CM products

Abstract. The EarthCARE (Earth Clouds, Aerosols and Radiation Explorer) satellite mission will provide new insights into aerosol–cloud–radiation interactions by means of synergistic observations of the Earth's atmosphere from a collection of active and passive remote sensing instruments, flying on a single satellite platform. The Multi-Spectral Imager (MSI) will provide visible and infrared images in the cross-track direction with a 150 km swath and a pixel sampling at 500 m. The suite of MSI cloud algorithms will deliver cloud macro- and microphysical properties complementary to the vertical profiles measured from the Atmospheric Lidar (ATLID) and the Cloud Profiling Radar (CPR) instruments. This paper provides an overview of the MSI cloud mask algorithm (M-CM) being developed to derive the cloud flag, cloud phase and cloud type products, which are essential inputs to downstream EarthCARE algorithms providing cloud optical and physical properties (M-COP) and aerosol optical properties (M-AOT). The MSI cloud mask algorithm has been applied to simulated test data from the EarthCARE end-to-end simulator and satellite data from the Moderate Resolution Imaging Spectroradiometer (MODIS) as well as from the Spinning Enhanced Visible InfraRed Imager (SEVIRI). Verification of the MSI cloud mask algorithm to the simulated test data and the official cloud products from SEVIRI and MODIS demonstrates a good performance of the algorithm. Some discrepancies are found, however, for the detection of thin cirrus clouds over bright surfaces like desert or snow. This will be improved by tuning of the thresholds once real observations are available.

3-D Object Detection for Multiframe 4-D Automotive Millimeter-Wave Radar Point Cloud

Object detection is a crucial task in autonomous driving. Currently, object-detection methods for autonomous driving systems are primarily based on information from cameras and lidar, which may experience interference from complex lighting or poor weather. At present, the four-dimensional (4D) (x, y, z, v) millimeter-wave radar can provide a denser point cloud to achieve 3D object-detection tasks that are difficult to complete with traditional millimeter-wave radar. Existing 3D object point-cloud detection algorithms are mostly based on 3D lidar, these methods are not necessarily applicable to millimeter-wave radars, which have sparser data and more noise and include velocity information. This study proposes a 3D object-detection framework based on a multi-frame 4D millimeter-wave radar point cloud. First, the ego vehicle velocity information is estimated by the millimeter-wave radar, and the relative velocity information of the millimeter-wave radar point cloud is compensated to the absolute velocity. Second, by matching between millimeter-wave radar frames, the multi-frame millimeter-wave radar point cloud is matched to the last frame. Finally, the object is detected by the proposed multi-frame millimeter-wave radar point cloud detection network. Experiments are performed using our newly recorded TJ4DRadSet dataset in a complex traffic environment. The results showed that the proposed object-detection framework outperformed the comparison methods based on the 3D mean average precision. The experimental results and methods can be used as the baseline for other multi-frame 4D millimeter-wave radar detection algorithms.

A global historical twice-daily (daytime and nighttime) land surface temperature dataset produced by Advanced Very High Resolution Radiometer observations from 1981 to 2021

Abstract. Land surface temperature (LST) is a key variable for monitoring and evaluating global long-term climate change. However, existing satellite-based twice-daily LST products only date back to 2000, which makes it difficult to obtain robust long-term temperature variations. In this study, we developed the first global historical twice-daily LST dataset (GT-LST), with a spatial resolution of 0.05∘, using Advanced Very High Resolution Radiometer (AVHRR) Level-1b Global Area Coverage (GAC) data from 1981 to 2021. The GT-LST product was generated using four main processes: (1) GAC data reading, calibration, and preprocessing using open-source Python libraries; (2) cloud detection using the AVHRR-Phase I algorithm; (3) land surface emissivity estimation using an improved method considering annual land cover changes; (4) LST retrieval based on a nonlinear generalized split-window algorithm. Validation with in situ measurements from Surface Radiation Budget (SURFRAD) sites and Baseline Surface Radiation Network sites showed that the overall root-mean-square errors (RMSEs) of GT-LST varied from 1.6 to 4.0 K, and nighttime LSTs were typically better than daytime LSTs. Intercomparison with the Moderate Resolution Imaging Spectroradiometer LST products (MYD11A1 and MYD21A1) revealed that the overall root-mean-square difference (RMSD) was approximately 3.0 K. Compared with MYD11A1 LST, GT-LST was overestimated, and relatively large RMSDs were obtained during the daytime, spring, and summer, whereas the significantly smaller positive bias was obtained between GT-LST and MYD21A1 LST. Furthermore, we compared our newly generated dataset with a global AVHRR daytime LST product at the selected measurements of SURFRAD sites (i.e., measurements of these two satellite datasets were valid), which revealed similar accuracies for the two datasets. However, GT-LST can additionally provide nighttime LST, which can be combined with daytime observations estimating relatively accurate monthly mean LST, with an RMSE of 2.7 K. Finally, we compared GT-LST with a regional twice-daily AVHRR LST product over continental Africa in different seasons, with RMSDs ranging from 2.1 to 4.3 K. Considering these advantages, the proposed dataset provides a better data source for a range of research applications. GT-LST is freely available at https://doi.org/10.5281/zenodo.7113080 (1981–2000) (Li et al., 2022a), https://doi.org/10.5281/zenodo.7134158 (2001–2005) (Li et al., 2022b), and https://doi.org/10.5281/zenodo.7813607 (2006–2021) (J. H. Li et al., 2023).

Proactive and dynamic load balancing model for workload spike detection in cloud

Dynamic Load Balancing (DLB) has been an ever-green research topic since the beginning of cloud computing. DLB model distributes the available cloud resources (i.e., CPU, RAM, Storage, BW, etc.) among the VMs using a systematic approach to ensure efficient resource utilization. Overloading, an abnormal condition, occurs when the host encounters a resource shortage due to the high workload of Federated Learning. VM migration and consolidation are reliable load-balancing techniques to fix dynamic overloading issues. As part of the migration, the oversubscribed host VMs are moved to the other hosts till the load is regulated. Besides the high workload, the sudden spike in workload also leads the hosts to instant overloading. The former DLB models are efficient in handling general overloading conditions. Still, they are now suffering from limitations in taking the spike overloading due to 1) a sudden spike in workloads, 2) instant overloading, 3) recurrent migrations, and 4) SLA violations. To address these limitations in handling spike overloading, in this paper, we proposed a proactive and dynamic load balancing model (DLBM), which is designed to handle spike overloading with optimal solutions. The offered dynamic load balancing model with a spike detection algorithm helps detect workload spikes in advance to prevent the hosts from overloading and SLA violations. Experimental conducted using the Cloudsim tool and the planet lab workload dataset prove that the proposed DLBM identified the spikes in workload and efficiently controlled the recurrent migrations.

Cloud detection of multi-type satellite images based on spectral assimilation and deep learning

ABSTRACT With strong self-learning and data analysis capabilities, deep learning is essential in cloud detection. However, many high-quality samples are the key to deep learning cloud detection methods. Different satellite image cloud detection techniques need to select adequate representative and high-quality samples and make corresponding cloud masks, which not only requires professional knowledge but also consumes a lot of workforce and time. To improve the generalization ability of the deep learning cloud detection model and quickly apply it to the cloud detection of different satellite images, this paper proposes a deep learning cloud detection method based on spectral assimilation for multiple types of satellite images (SAUNetCD). Under the condition of using fewer deep learning data samples, the deep learning model is used to achieve automatic cloud detection of multiple satellite data. Taking Landsat 8 OLI, Landsat 9 OLI, GF-1 WFV, and Sentinel 2A as examples, this paper selects Landsat 8 OLI data as the source data of cloud detection. The experimental results show that the spectral assimilation method improves the generalization ability of the deep learning cloud detection model and improves the cloud detection accuracy by nearly 20%. It realizes the fast cloud detection application of different satellite images by deep learning. It provides an effective way for cloud detection of multiple types of satellite images.

A transformer-based cloud detection approach using Sentinel 2 imageries

ABSTRACT Presence of clouds blocks the view of Earth’s surface objects in optical imageries, thus compromising their application and usability. Identifying and removing the clouds become a crucial task during image preprocessing. Recently deep learning (DL)-based cloud detection methods have shown improved performance, but capturing global semantic features and long-range dependencies necessitates a careful selection of DL classifiers to further enhance their effectiveness. Keeping this in view, the present study proposes a novel spatial-spectral attention transformer for cloud detection (SSATR-CD) with a spatial-spectral attention module that generates an enhanced feature map to replace convolution by using the image patches directly. To implement the proposed approach, a new Sentinel-2 data set with various types of cloud covers over India (IndiaS2) was created and tested with the proposed method. Alongside this, an additional benchmarked data set (WHUS2-CD) was also considered to check the ability of the proposed model to different regions of the world by applying model-based transfer learning. The result highlights the effectiveness and efficiency of the SSATR-CD approach in both cases.

Detection of clouds in multiple wind velocity fields using ground-based infrared sky images

To improve the performance of an intra-hour global solar irradiance forecasting algorithm, it is important to detect multiple layers of clouds. Horizontal atmospheric wind shear causes wind velocity fields to have different directions and speeds. In images of clouds acquired using ground-based sky imagers, clouds may be moving in different wind layers. The information provided by a solar forecasting algorithm is necessary to optimize and schedule the solar generation resources and storage devices in a smart grid. This investigation studies the performance of unsupervised learning techniques when detecting the number of cloud layers in infrared sky images. The images are acquired using an innovative infrared sky imager mounted on a solar tracker. Different mixture models are used to infer the distribution of the cloud features. Multiple Bayesian metrics and a sequential hidden Markov model are implemented to find the optimal number of clusters in the mixture models, and their performances are compared. The motion vectors are computed using a probabilistic implementation of the Lucas-Kanade algorithm. The correlations between the cloud motion vectors and temperatures are analyzed to discover the method that leads to the most accurate results. The findings point that a sequential hidden Markov model outperforms the detection accuracy of standard Bayesian model selection metrics.

Effect of Bit Depth on Cloud Segmentation of Remote-Sensing Images

Due to the cloud coverage of remote-sensing images, the ground object information will be attenuated or even lost, and the texture and spectral information of the image will be changed at the same time. Accurately detecting clouds from remote-sensing images is of great significance to the field of remote sensing. Cloud detection utilizes semantic segmentation to classify remote-sensing images at the pixel level. However, previous studies have focused on the improvement of algorithm performance, and little attention has been paid to the impact of bit depth of remote-sensing images on cloud detection. In this paper, the deep semantic segmentation algorithm UNet is taken as an example, and a set of widely used cloud labeling dataset “L8 Biome” is used as the verification data to explore the relationship between bit depth and segmentation accuracy on different surface landscapes when the algorithm is used for cloud detection. The research results show that when the image is normalized, the effect of cloud detection with a 16-bit remote-sensing image is slightly better than that of an 8-bit remote sensing image; when the image is not normalized, the gap will be widened. However, using 16-bit remote-sensing images for training will take longer. This means data selection and classification do not always need to follow the highest possible bit depth when doing cloud detection but should consider the balance of efficiency and accuracy.

Automatic cloud detection method for optical remote sensing images combining channel attention mechanism and multi-feature fusion

Cloud detection is a crucial step in remote sensing image pre-processing, but there are still issues such as cloud miss detection and false detection. Therefore, this work suggests a multi-feature fusion and channel attention mechanism combined approach for automatically recognizing clouds. Based on DeepLabV3+, to make the feature extraction faster and more accurate, a lightweight network structure MobileNetV2 is used as the backbone network. To effectively distinguish snow and clouds, channel attention is added to the low-level semantic information, which makes the network more able to focus on essential features. Meantime, mid-level semantic features are added to the decoder module to make use of more abundant features, thus reducing the false detection of mist. Experimental findings on the GF1 WHU dataset demonstrate that the method’s detection accuracy is improved by 2.1% when compared to DeepLabV3+, and it performs well in the cloud detection job of optical remote sensing images.