Satellite Cloud Images Research Articles

Seeing Through Clouds in Satellite Images

This paper presents a neural network-based solution to recover pixels occluded by clouds in satellite images. We leverage radio frequency (RF) signals in the ultra/super-high frequency band that penetrate clouds to help reconstruct the occluded regions in multispectral images. We introduce the first multi-modal multi-temporal method for cloud removal. Our model uses publicly available satellite observations and produces daily cloud-free images. Experimental results show that our system outperforms several baselines on multiple metrics. We also demonstrate use cases of our system in digital agriculture, flood monitoring, and wildfire detection.

Faster and Lighter Meteorological Satellite Image Classification by a Lightweight Channel-Dilation-Concatenation Net

With the development of satellite photography, meteorologists are inclined to rely on methods for the automatic and efficient classification of weather images. However, many popular networks require numerous parameters and a lengthy inference time, making them unsuitable for real-time classification tasks. To solve these problems, a lightweight convolutional network termed the Channel-Dilation-Concatenation network (CDC-net) is constructed for meteorological satellite image classification. When extracting features, CDC-net utilizes depth-wise convolution rather than standard convolution. Additionally, a FeatureCopy operation was employed instead of a half-convolution operation. CDC-net extracts high-dimensional features and contains a local importance-based pooling layer, reducing the network's depth, the number of network parameters and inference time. Based on these techniques, the CDC-net achieves an accuracy of 93.56% on the Large-Scale Satellite Cloud Image Database for Meteorological Research, with a GPU inference time of 3.261 ms and 1.12 million parameters. Because many weather images reveal multiple weather patterns, multiple labels are necessary. Therefore, we propose a prediction method and conduct experiments on multi-label data. Experiments on single-label and multi-label meteorological satellite image datasets demonstrate the superiority of the CDC-net over other structures. Thus, the proposed CDC-net can provide a faster and lighter solution in meteorological satellite image classification.

Identification of tropical cyclone centre based on satellite images via deep learning techniques

AbstractA tropical cyclone (TC) is a highly destructive natural disaster. Accurate identification of key parameters of TCs is prerequisite for most TC‐related research and practices. The centre position is one of TC's basic parameters. However, comparison of TC best track data released by different meteorological institutes usually indicates a noticeable discrepancy for this parameter among varied data sources. In this study, efforts are made towards identifying the centre location of TCs via deep learning techniques, based on TC satellite cloud images (SCIs). Six deep learning models are analysed and compared. YOLOv4 model achieved a confidence of 99.84%, which is better than other models. In addition, we further explore the factors affecting the positioning accuracy of the YOLOv4 model and its application to the location identification of multiple TCs and the tracking of individual TCs. Results demonstrate that the YOLOv4 model has a probability exceeding 99% for identifying multiple TC locations and also performs well for single TC tracking.

Improvement and Application of a GAN Model for Time Series Image Prediction—A Case Study of Time Series Satellite Cloud Images

Predicting the shape evolution and movement of remote sensing satellite cloud images is a difficult task requiring the effective monitoring and rapid prediction of thunderstorms, gales, rainstorms, and other disastrous weather conditions. We proposed a generative adversarial network (GAN) model for time series satellite cloud image prediction in this research. Taking time series information as the constraint condition and abandoning the assumption of linear and stable changes in cloud clusters in traditional methods, the GAN model is used to automatically learn the data feature distribution of satellite cloud images and predict time series cloud images in the future. Through comparative experiments and analysis, the Mish activation function is selected for integration into the model. On this basis, three improvement measures are proposed: (1) The Wasserstein distance is used to ensure the normal update of the GAN model parameters; (2) establish a multiscale network structure to improve the long-term performance of model prediction; (3) combined image gradient difference loss (GDL) to improve the sharpness of prediction cloud images. The experimental results showed that for the prediction cloud images of the next four times, compared with the unimproved Mish-GAN model, the improved GDL-GAN model improves the PSNR and SSIM by 0.44 and 0.02 on average, and decreases the MAE and RMSE by 18.84% and 7.60% on average. It is proven that the improved GDL-GAN model can maintain good visualization effects while keeping the overall changes and movement trends of the prediction cloud images relatively accurate, which is helpful to achieve more accurate weather forecast. The cooperation ability of satellite cloud images in disastrous weather forecasting and early warning is enhanced.

LSCIDMR: Large-Scale Satellite Cloud Image Database for Meteorological Research.

People can infer the weather from clouds. Various weather phenomena are linked inextricably to clouds, which can be observed by meteorological satellites. Thus, cloud images obtained by meteorological satellites can be used to identify different weather phenomena to provide meteorological status and future projections. How to classify and recognize cloud images automatically, especially with deep learning, is an interesting topic. Generally speaking, large-scale training data are essential for deep learning. However, there is no such cloud images database to date. Thus, we propose a large-scale cloud image database for meteorological research (LSCIDMR). To the best of our knowledge, it is the first publicly available satellite cloud image benchmark database for meteorological research, in which weather systems are linked directly with the cloud images. LSCIDMR contains 104 390 high-resolution images, covering 11 classes with two different annotation methods: 1) single-label annotation and 2) multiple-label annotation, called LSCIDMR-S and LSCIDMR-M, respectively. The labels are annotated manually, and we obtain a total of 414 221 multiple labels and 40 625 single labels. Several representative deep learning methods are evaluated on the proposed LSCIDMR, and the results can serve as useful baselines for future research. Furthermore, experimental results demonstrate that it is possible to learn effective deep learning models from a sufficiently large image database for the cloud image classification.

Semantic segmentation of clouds in satellite images based on U-Net++ architecture and attention mechanism

The presence of clouds in satellite imagery may pose hindrances to the accurate and reliable analysis of the objects present on the land. Therefore, automatic cloud detection is a vital pre-processing step before lending the satellite images to any further analysis. This is a challenging task due to the varying thickness and densities of clouds. It is also very difficult to distinguish clouds from certain terrains such as snow and white sandy beaches. This paper proposes a deep learning based algorithm to solve the problem of cloud segmentation on the Landsat 8 multispectral dataset, 95-Cloud: SEUNet++. Specifically, the proposed model consists of a U-Net++ semantic segmentation model with a lightweight channel attention mechanism. We also experimented with using different encoder backbones in the U-Net++ encoder-decoder architecture such as ResNet (Residual Neural Networks) variants including ResNet-18, ResNet-34, ResNet-50, and ResNet-101, DenseNet-264, CSPNet (Cross Stage Partial Network), and EfficientNet-B8 and compared their performance. The experimental results show that the proposed architecture achieves an IoU (Intersection over Union) score of 91.8 %, improving the state-of-the-art on the task by 0.23 %. It also boosts the accuracy, precision, and recall values creating crisp cloud boundaries and detecting even thin layers of clouds. We also experimented using transfer learning and found that it has a positive impact on the cloud segmentation task. The proposed model also beats the original U-Net++ architecture in terms of various evaluation metrics such as the IoU score, accuracy, precision, and recall. The experimental results thereby demonstrate that our model is computationally efficient and achieves precise segmentation results.

Classification and Estimation of Typhoon Intensity from Geostationary Meteorological Satellite Images Based on Deep Learning

In this paper, a novel typhoon intensity classification and estimation network (TICAENet) is constructed to recognize typhoon intensity. The TICAENet model is based on the LeNet-5 model, which uses weight sharing to reduce the number of training parameters, and the VGG16 model, which replaces a large convolution kernel with multiple small kernels to improve feature extraction. Satellite cloud images of typhoons over the Northwest Pacific Ocean and the South China Sea from 1995–2020 are taken as samples. The results show that the classification accuracy of this model is 10.57% higher than that of the LeNet-5 model; the classification accuracy of the TICAENet model is 97.12%, with a classification precision of 97.00% for tropical storms, severe tropical storms and super typhoons. The mean absolute error (MAE) and root mean square error (RMSE) of the samples estimation in 2019 are 4.78 m/s and 6.11 m/s, and the estimation accuracy are 18.98% and 20.65% higher than that of the statistical method, respectively. Additionally, the model takes less memory and runs faster due to the weight sharing and multiple small kernels. The results show that the proposed model performs better than other methods. In general, the proposed model can be used to accurately classify typhoon intensity and estimate the maximum wind speed by extracting features from geostationary meteorological satellite images.

Cloud detection for satellite cloud images based on fused FCN features

ABSTRACT Cloud detection for satellite cloud images is a challenging image processing task owing to the blurring of cloud boundaries, multiplicity, and complexity of cloud types. Currently, the commonly used cloud detection methods include original full convolutional neural network (original FCN), FCN with an 8-pixel stride (FCN- 8s), FCN with a 2-pixel stride (FCN-2s), and so on. However, the aforementioned methods exclusively rely on a single network layer the final layer feature map; thus, shallow cloud image information, such as cloud profile information may not be captured. In this letter, a cloud detection method for satellite cloud images based on fused FCN features is proposed. The proposed method effectively fuses spatial and high-level semantic information, and a voting ensemble strategy is used to improve the accuracy and robustness of cloud detection. Finally, the experimental results demonstrate that the average overall accuracy (OA), average producer’ accuracy (PA), and average user’ accuracy (UA) of the proposed method for multiple training sample sizes and image sizes of the collected Fengyun satellite (FY-2 G) cloud image database increased by 7.15%, 9.04%, and 8.46%, respectively, relative to the average accuracies of the original FCN, FCN-8s, FCN-2s, SegNet, and DeepLabV3 methods.

Identification of tropical cyclones via deep convolutional neural network based on satellite cloud images

Abstract. Tropical cyclones (TCs) are one of the most destructive natural disasters. For the prevention and mitigation of TC-induced disasters, real-time monitoring and prediction of TCs is essential. At present, satellite cloud images (SCIs) are utilized widely as a basic data source for such studies. Although great achievements have been made in this field, there is a lack of concern about on the identification of TC fingerprints from SCIs, which is usually involved as a prerequisite step for follow-up analyses. This paper presents a methodology which identifies TC fingerprints via deep convolutional neural network (DCNN) techniques based on SCIs of more than 200 TCs over the northwestern Pacific basin. In total, two DCNN models have been proposed and validated, which are able to identify the TCs from not only single TC-featured SCIs but also multiple TC-featured SCIs. Results show that both models can reach 96 % of identification accuracy. As the TC intensity strengthens, the accuracy becomes better. To explore how these models work, heat maps are further extracted and analyzed. Results show that all the fingerprint features are focused on clouds during the testing process. For the majority of the TC images, the cloud features in TC's main parts, i.e., eye, eyewall, and primary rainbands, are most emphasized, reflecting a consistent pattern with the subjective method.

Discriminating Technique of Typhoon Rapid Intensification Trend Based on Artificial Intelligence

The trend detection of the sudden change of typhoon intensity has always been a difficult issue in typhoon forecast. Artificial intelligence (AI) can implicitly extract the complex features in the images through learning a large number of samples, and it has been widely applied in the meteorological field nowadays. In this study, based on the deep residual network (ResNet) model and the long short-term memory (LSTM) model, an automatic and objective method of identifying the trend of typhoon rapid intensification (RI) is presented through marking and learning the key information on the satellite images of the typhoons on the Northwest Pacific and South China Sea from 2005 to 2018. This method introduces the typhoon lifecycle indication and can effectively forecast and identify the trend of typhoon RI. By applying the detecting method in analyzing the operational typhoon satellite cloud images in 2019, we find that the method can well capture the sudden change tendency of typhoon intensity, and the threat score of independent sample estimation in 2019 reached 0.24. In addition, four typhoon cases with RI processes from 2019 to 2021 are tested, and the results show that the AI-based identification method of typhoon RI is superior to the traditional subjective intensity prediction method, and it has important application values.

Application of Shape Moments for Cloudiness Assessment in Marine Environmental Research

The search for clouds in satellite images is a challenging subject which still attracts a lot of attention due to the amount and quality of data, which is growing at a tremendous pace, the development of satellite techniques and methods, inexpensive equipment, and automation of satellite imaging processes. This paper presents a new approach to the assessment of cloudiness based on the use of the theory of moments with invariants. The values of moments with invariants, determined on the basis of the available cloudiness maps, create a new, valuable set of data, which are the geometrical parameters of the scene representing the cloud cover. In further research, the obtained data sets will be used in machine learning methods, deep machine learning methods, etc. The method is used for different conditions, including different angular positions of the Sun and time periods. The effectiveness of the method is checked on the basis of comparing the entropy results of the input maps after subtracting clouds masked by various methods. The obtained results additionally indicate the potential of the moments method as a support for the existing methods of estimating cloudiness over the sea surface.

Mission Replanning for Multiple Agile Earth Observation Satellites Based on Cloud Coverage Forecasting

Recent decades have witnessed a tremendous growth in the number of Earth observation satellites (EOSs), which presents a huge challenge for mission planning. For the EOSs with optical sensors particularly, the observation mission is significantly influenced by the uncertainty of cloud coverage, which has been identified as the most dominant factor for the invalidation of remote sensing images. To overcome this uncertainty, uncertainty programming methods, namely, chance constraint programming (CCP), stochastic expectation model, and robust optimization, are put forth. Despite their success, these approaches are limited in that they simplified the complex cloud coverage uncertainty, which may be different from the true cloud conditions, and they did not take the true cloud information into consideration. Motivated by these recent trends toward Big Data of satellite cloud images and machine learning for spatiotemporal prediction, this article explores a dynamic replanning scheme for multiple EOSs based on cloud forecasting. Specifically, we propose a new approach mainly in the following three steps: first, proactive scheduling based on a CCP is implemented and uploaded via ground control; second, cloud forecasting can be continuously conducted relying on the predictive recurrent neural network and the latest satellite cloud image; and third, mission replanning can be conducted according to the initial schedule and relatively accurate cloud information. Simulation results show that the cloud forecasting method is effective, and the replanning approach presents highly efficient and accurate scheduling results.

Deep Learning for Typhoon Intensity Classification Using Satellite Cloud Images

Abstract A tropical cyclone, also known as a typhoon, is one of the most destructive weather phenomena. Its intense cyclonic eddy circulations often cause serious damage to coastal areas. Accurate classification or prediction for typhoon intensity is crucial to disaster warning and mitigation management. But typhoon intensity-related feature extraction is a challenging task as it requires significant preprocessing and human intervention for analysis, and its recognition rate is poor due to various physical factors such as tropical disturbance. In this study, we built a Typhoon-CNNs framework, an automatic classifier for typhoon intensity based on a convolutional neural network (CNN). The Typhoon-CNNs framework utilized a cyclical convolution strategy supplemented with dropout zero-set, which extracted sensitive features of existing spiral cloud bands (SCBs) more effectively and reduces the overfitting phenomenon. To further optimize the performance of Typhoon-CNNs, we also proposed the improved activation function (T-ReLU) and the loss function (CE-FMCE). The improved Typhoon-CNNs was trained and validated using more than 10 000 multiple sensor satellite cloud images from the National Institute of Informatics. The classification accuracy reached to 88.74%. Compared with other deep learning methods, the accuracy of our improved Typhoon-CNNs was 7.43% higher than ResNet50, 10.27% higher than InceptionV3, and 14.71% higher than VGG16. Finally, by visualizing hierarchic feature maps derived from Typhoon-CNNs, we can easily identify the sensitive characteristics such as typhoon eyes, dense-shadowing cloud areas, and SCBs, which facilitate classifying and forecasting typhoon intensity.

UATNet: U-Shape Attention-Based Transformer Net for Meteorological Satellite Cloud Recognition

Cloud recognition is a basic task in ground meteorological observation. It is of great significance to accurately identify cloud types from long-time-series satellite cloud images for improving the reliability and accuracy of weather forecasting. However, different from ground-based cloud images with a small observation range and easy operation, satellite cloud images have a wider cloud coverage area and contain more surface features. Hence, it is difficult to effectively extract the structural shape, area size, contour shape, hue, shadow and texture of clouds through traditional deep learning methods. In order to analyze the regional cloud type characteristics effectively, we construct a China region meteorological satellite cloud image dataset named CRMSCD, which consists of nine cloud types and the clear sky (cloudless). In this paper, we propose a novel neural network model, UATNet, which can realize the pixel-level classification of meteorological satellite cloud images. Our model efficiently integrates the spatial and multi-channel information of clouds. Specifically, several transformer blocks with modified self-attention computation (swin transformer blocks) and patch merging operations are used to build a hierarchical transformer, and spatial displacement is introduced to construct long-distance cross-window connections. In addition, we introduce a Channel Cross fusion with Transformer (CCT) to guide the multi-scale channel fusion, and design an Attention-based Squeeze and Excitation (ASE) to effectively connect the fused multi-scale channel information to the decoder features. The experimental results demonstrate that the proposed model achieved 82.33% PA, 67.79% MPA, 54.51% MIoU and 70.96% FWIoU on CRMSCD. Compared with the existing models, our method produces more precise segmentation performance, which demonstrates its superiority on meteorological satellite cloud recognition tasks.

Discriminating the Multi-Frequency Microwave Brightness Temperature Anomalies Relating to 2017 Mw 7.3 Sarpol Zahab (Iran-Iraq Border) Earthquake

A Mw 7.3 earthquake occurred near the Iran-Iraq border on November 12, 2017, as the result of oblique-thrust squeezing of the Eurasian plate and the Arabian plate. By employing the spatio-temporally weighted two-step method (STW-TSM) and microwave brightness temperature (MBT) data from AMSR-2 instrument on board Aqua satellite, this paper investigates carefully the spatiotemporal features of multi-frequency MBT anomalies relating to the earthquake. Soil moisture (SM), satellite cloud image, regional geological map and surface landcover data are utilized to discriminate the potential MBT anomalies revealed from STW-TSM. The low-frequency MBT residual images shows that positive anomalies mainly occurred in the mountainous Urmia lake and the plain region, which were 300 km north and 200 km southwest about to the epicenter, respectively. The north MBT anomaly firstly appeared 51 days before the mainshock and its magnitude increased over time with a maximum of about +40K. Then the anomaly disappeared 3 days before, reappeared 1d after and diminished completely 10 days after the mainshock. Meanwhile, the southwest MBT anomaly firstly occurred 18 days before and peaked 3 days before the mainshock with a maximum of about +20K, and then diminished gradually with aftershocks. It is speculated that the positive MBT anomaly in the Urmia lake was caused by microwave dielectric property change of water body due to gas bubbles leaking from the bottom of the lake disturbed by local crust stress alteration, while the southwest MBT positive anomaly was caused by microwave dielectric constant change of shallow surface due to accumulation of seismically-activated positive charges originated at deep crust. Besides, some accidental abnormal residual stripes existed in line with satellite orbit, which turned out to be periodic data errors of the satellite sensor. High-frequency MBT residual images exhibit some significant negative anomalies, including a narrow stripe pointing to the forthcoming epicenter, which were confirmed to be caused by synchronous altostratus clouds. This study is of guidance meaning for distinguishing non-seismic disturbances and identifying seismic MBT anomaly before, during and after some large earthquakes.

Multi-spectral cloud detection based on a multi-dimensional and multi-grained dense cascade forest

Cloud detection in satellite images is a vital step for cloud/land recognition, cloud/snow discrimination, and cloud shadow removal. Accurate cloud detection plays an important role in land resource management, environmental pollution monitoring, and land target recognition. Deep learning (DL) algorithms have shown great progress in cloud detection. However, as the complexity of the DL-based model increases, cloud detection efficiency decreases. DL-based cloud detection models are unable to successfully balance the performance-efficiency tradeoff. In our study, a multi-dimensional and multi-grained dense cascade forest (MDForest) is proposed for multi-spectral cloud detection. MDForest is a deep forest structure that automatically extracts low-level and high-level features from satellite cloud images end-to-end; a multi-dimensional and multi-grained scanning mechanism is introduced to capture the spectral information of multi-spectral satellite images while enhancing the representation learning ability of cascade forest. The experimental results on the HJ-1A/1B dataset show that MDForest improves the performance of cloud detection and possesses a good inference efficiency compared with DL-based cloud detection methods, which makes the proposed MDForest satisfy the application where good performance and high efficiency are both required.

Superresolution Imaging With a Deep Multipath Network for the Reconstruction of Satellite Cloud Images

AbstractSatellite cloud images play an important role in weather analysis and forecast. High‐resolution satellite images play a significant role in the study of mesoscale weather systems such as typhoons. With the increasing demands of locating and tracking techniques, the resolution of satellite images is no longer satisfactory. Enhancing their resolution with superresolution (SR) methods can help in identifying and locating weather systems. In this paper, we propose a multipath network model, called SRCloudNet, that involves joint training of a back‐projection network and a local residual network. SRCloudNet integrates features extracted from back‐projection units and residual dense blocks to achieve more accurate image reconstruction. We also developed a novel natural‐color cloud and contrail image data set, constituting the first‐ever satellite cloud image data set established for SR research. Because of the special features, contrail images were first used to test the performance of SRCloudNet. Extensive experiments demonstrated that SRCloudNet achieves superior performance.

Analysis on the Mechanism of the Heavy Snow Process in the Northeast of Tarim Basin, Xinjiang, in 2018

An analysis was made on the routine data of meteorological stations, high-resolution satellite cloud images and the NCEP/NCA R 1°×1° reanalysis data during one instance of heavy snow precipitation that took place in northern Bazhou, Xinjiang on October 18, 2018, followed by a diagnostic analysis of physical quantities, including the large-scale circulation anomalies, upper and lower-level circulation conditions and water vapor transport. The following are the results: (1) the heavy snow precipitation process in northern Bazhou, Xinjiang is a heavy precipitation process largely caused by a convergence of severely cold air carried from Siberia by an upper vortex, and a strong, warm and humid air current from southwest China. During this weather process, the upper layer was dry and cold, while the lower layer was warm and humid; an obvious wind shear and convergence center appeared at 700hPa, 850hPa in the lower layer over northern Bazhou; the heavy precipitation area was located in the front of the trough bottom at 500hPa, on the left side of the lower-level shear line. 2) The secondary circulation formed on the right side of the upper-level jet stream entrance area at 200hpa and 300hpa enlarged the ascending motion of the heavy snowfall area, while the low-level easterly jet stream played a critical role in transporting and accumulating water vapor at 850hpa. 3) The variations of temperature at 850hpa and on the ground layer were of certain indicative significance to the forecast of the precipitation phase state, and the temperature field at 850hpa well-indicated the change from rain to snow. When the air temperature fell to 0.5°C or so, and the ground temperature fell below 0°C given T700hpa<-10°C and T850hpa<-2°C, it could be used as a criterion for the judgment of rain changing to snow. 4) The vapor condition, lifting force condition and unstable stratification condition in the numerically-forecasted physical quantity field, all had a good guiding effect on the heavy precipitation process that took place in the Yanqi Reclamation Area.

Classification of satellite cloud images of disaster weather based on adversarial and transfer learning

Classification of satellite cloud images of disaster weather based on adversarial and transfer learning

Short-term solar irradiance forecasting using convolutional neural networks and cloud imagery

Access to accurate, generalizable and scalable solar irradiance prediction is critical for smooth solar-grid integration, especially in the light of the accelerated global adoption of solar energy production. Both physical and statistical prediction models of solar irradiance have been proposed in the literature. Physical models require meteorological forecasts—generated by computationally expensive models—to predict solar irradiance, with limited accuracy in sub-daily predictions. Statistical models leverage in-situ measurements which require expensive equipment and do not account for meso-scale atmospheric dynamics. We address these fundamental gaps by developing a convolutional global horizontal irradiance prediction model, using convolutional neural networks and publicly accessible satellite cloud images. Our proposed model predicts solar irradiance in 12 different locations in the US for various prediction time horizons. Our model yields up to 24% improvement in an hour-ahead predictions and 26% in a day-ahead predictions compared to a persistence forecast. Moreover, using saliency maps and target-location-focused cropping, we demonstrate the benefits of incorporating meso-scale atmospheric dynamics for prediction performance. Our results are critical for energy systems planners, utility managers and electricity market participants to ensure efficient harvesting of the solar energy and reliable operation of the grid.