Ground-based Cloud Image Research Articles

CloudSwinNet: A hybrid CNN-transformer framework for ground-based cloud images fine-grained segmentation

Solar irradiance is the main factor affecting the output of a photovoltaic (PV) power station. The dominant role of ground-based clouds on the variation of direct solar radiation in the whole sky. Ground-based cloud segmentation plays a crucial role in photovoltaic power generation prediction. Despite advancements, current cloud segmentation methods fail to meet the requirements of this application. While convolutional neural networks demonstrate impressive segmentation capabilities in ground-based cloud segmentation tasks, their inherent limitations hinder further performance enhancement. Hence, this paper introduces CloudSwinNet, a hybrid CNN-Transformer framework for ground-based cloud images fine-grained segmentation. CloudSwinNet leverages concepts from convolutional neural networks, including stepwise downsampling, local convolution, and skip connections. To further explore the fine-grained features in the ground-based cloud images, we incorporates the Fine-grained Feature Fusion Module (Fg-FFM) in encoder structure, while the jump connection structure integrates the GloRe spatial graph inference module. These additions enable comprehensive learning of multi-scale and long-range dependencies. We conducted extensive experiments on the fine-grained ground-based cloud dataset, demonstrating that CloudSwinNet outperforms six semantic segmentation networks in both qualitative and quantitative evaluations. The results of ablation experiments prove the effectiveness of the module introduced in this paper.

Research on ground-based cloud image classification combining local and global features

Research on ground-based cloud image classification combining local and global features

Improved RepVGG ground-based cloud image classification with attention convolution

Abstract. Atmospheric clouds greatly impact Earth's radiation, hydrological cycle, and climate change. Accurate automatic recognition of cloud shape based on a ground-based cloud image is helpful for analyzing solar irradiance, water vapor content, and atmospheric motion and then predicting photovoltaic power, weather trends, and severe weather changes. However, the appearance of clouds is changeable and diverse, and their classification is still challenging. In recent years, convolution neural networks (CNNs) have made great progress in ground-based cloud image classification. However, traditional CNNs poorly associate long-distance clouds, making the extraction of global features of cloud images quite problematic. This study attempts to mitigate this problem by elaborating on a ground-based cloud image classification method based on the improved RepVGG convolution neural network and attention mechanism. Firstly, the proposed method increases the RepVGG residual branch and obtains more local detail features of cloud images through small convolution kernels. Secondly, an improved channel attention module is embedded after the residual branch fusion, effectively extracting the global features of cloud images. Finally, the linear classifier is used to classify the ground cloud images. Finally, the warm-up method is applied to optimize the learning rate in the training stage of the proposed method, making it lightweight in the inference stage and thus avoiding overfitting and accelerating the model's convergence. The proposed method is validated on the multimodal ground-based cloud dataset (MGCD) and the ground-based remote sensing cloud database (GRSCD) containing seven cloud categories, with the respective classification accuracy rate values of 98.15 % and 98.07 % outperforming those of the 10 most advanced methods used as the reference. The results obtained are considered instrumental in ground-based cloud image classification.

CloudFU-Net: A Fine-Grained Segmentation Method for Ground-Based Cloud Images Based on an Improved Encoder–Decoder Structure

CloudFU-Net: A Fine-Grained Segmentation Method for Ground-Based Cloud Images Based on an Improved Encoder–Decoder Structure

Photovoltaic Power Forecasting Approach Based on Ground-Based Cloud Images in Hazy Weather

Photovoltaic Power Forecasting Approach Based on Ground-Based Cloud Images in Hazy Weather

An improved SSA-BiLSTM-based short-term irradiance prediction model via sky images feature extraction

The development and utilization of solar energy has become an important strategic decision for the sustainable development of many countries. Short-term variations in solar irradiation have an impact on the safety and stability of photovoltaic and solar thermal power plants, therefore, the development and accuracy of solar irradiance prediction models have received much attention. This paper proposes a short-term irradiance prediction model based on mixed intelligent optimization algorithm and deep learning algorithm that integrates features of various forms of information. First, the sequence containing the picture attributes as well as the color and texture characteristics are recovered from ground-based cloud images, historical irradiance and meteorological feature information is decomposed and reconstructed by singular spectrum analysis (SSA). Secondly, the bidirectional short term to long term memory (BiLSTM) network is optimized for model training and finally assessment using an enhanced chaotic sparrow search method. The technique exceeds benchmark methods and a number of sophisticated individual algorithms in forecasting ultra-short-term global horizontal irradiance (GHI), according to experimental data, while also offering extremely high accuracy and resilience.

CloudY-Net: A Deep Convolutional Neural Network Architecture for Joint Segmentation and Classification of Ground-Based Cloud Images

Ground-based cloud images contain a wealth of cloud information and are an important part of meteorological research. However, in practice, ground cloud images must be segmented and classified to obtain the cloud volume, cloud type and cloud coverage. Existing methods ignore the relationship between cloud segmentation and classification, and usually only one of these is studied. Accordingly, our paper proposes a novel method for the joint classification and segmentation of cloud images, called CloudY-Net. Compared to the basic Y-Net framework, which extracts feature maps from the central layer, we extract feature maps from four different layers to obtain more useful information to improve the classification accuracy. These feature maps are combined to produce a feature vector to train the classifier. Additionally, the multi-head self-attention mechanism is implemented during the fusion process to enhance the information interaction among features further. A new module called Cloud Mixture-of-Experts (C-MoE) is proposed to enable the weights of each feature layer to be automatically learned by the model, thus improving the quality of the fused feature representation. Correspondingly, experiments are conducted on the open multi-modal ground-based cloud dataset (MGCD). The results demonstrate that the proposed model significantly improves the classification accuracy compared to classical networks and state-of-the-art algorithms, with classification accuracy of 88.58%. In addition, we annotate 4000 images in the MGCD for cloud segmentation and produce a cloud segmentation dataset called MGCD-Seg. Then, we obtain a 96.55 mIoU on MGCD-Seg, validating the efficacy of our method in ground-based cloud imagery segmentation and classification.

CloudDeepLabV3+: a lightweight ground-based cloud segmentation method based on multi-scale feature aggregation and multi-level attention feature enhancement

ABSTRACT The segmentation of ground-based cloud images is the basis for obtaining numerous cloud parameters. To achieve high-precision adaptive cloud image segmentation requirements, this study designs a lightweight ground-based cloud image adaptive segmentation method named CloudDeepLabV3+ that integrates multi-scale features aggregation and multi-level attention feature enhancement. Firstly, a novel lightweight EfficientNetV2-S is designed as a feature extraction backbone to reduce network parameters. Secondly, a heterogeneous receptive field splicing atrous spatial pyramid pooling module is designed. It enhances the correlation of information between layers, and realizes multiscale information fusion. The feature enhancement module based on the self-attention mechanism intensifies the representation of local and global features. Thirdly, the feature alignment module based on the attention mechanism is constructed to pull deep and shallow features for alignment. Finally, we implement ablation study on the key components of method and comparison experiment with other advanced methods using five evaluation metrics. Results show that the key components play an important role in multiscale information fusion. It promotes the accuracy of cloud image feature extraction while reducing the loss of detailed features. Generalization performance verification indicates the excellent performance of the proposed model in advanced cloud feature extraction and cloud-mask generation.

Cloud Type Classification for Prediction of Rain Fall using CNN

Abstract: In order to forecast the climate, clouds are essential. Prediction of rainfall is also greatly influenced by the quantity and nature of clouds in the atmosphere. As a result, the most fascinating and important subject in meteorology is cloud identification, which also draws the most researchers from other fields. The transfer learning method used in this article to forecast rainfall using ground-based cloud images is presented. By classifying the type of cloud using cloud image input, it will be able to forecast the expected amount of rainfall. Based on the associated Precipitation that caused the appropriate Rainfall, the cloud images in the dataset are divided into three groups (classes) labeled as no rain to very low rain, low to medium rain, and medium to high rain. In this paper using CNN the classification of rainfall was done

Development of a Machine Learning Forecast Model for Global Horizontal Irradiation Adapted to Tibet Based on Visible All-Sky Imaging

The Qinghai-Tibet Plateau is rich in renewable solar energy resources. Under the background of China’s “dual-carbon” strategy, it is of great significance to develop a global horizontal irradiation (GHI) prediction model suitable for Tibet. In the radiation balance budget process of the Earth-atmosphere system, clouds, aerosols, air molecules, water vapor, ozone, CO2 and other components have a direct influence on the solar radiation flux received at the surface. For the descending solar shortwave radiation flux in Tibet, the attenuation effect of clouds is the key variable of the first order. Previous studies have shown that using Artificial intelligence (AI) models to build GHI prediction models is an advanced and effective research method. However, regional localization optimization of model parameters is required according to radiation characteristics in different regions. This study established a set of AI prediction models suitable for Tibet based on ground-based solar shortwave radiation flux observation and cloud cover observation data of whole sky imaging in the Yangbajing area, with the key parameters sensitively tested and optimized. The results show that using the cloud cover as a model input variable can significantly improve the prediction accuracy, and the RMSE of the prediction accuracy is reduced by more than 20% when the forecast horizon is 1 h compared with a model without the cloud cover input. This conclusion is applicable to a scenario with a forecast horizon of less than 4 h. In addition, when the forecast horizon is 1 h, the RMSE of the random forest and long short-term memory models with a 10-min step decreases by 46.1% and 55.8%, respectively, compared with a 1-h step. These conclusions provide a reference for studying GHI prediction models based on ground-based cloud images and machine learning.

MMST: A Multi-Modal Ground-Based Cloud Image Classification Method.

In recent years, convolutional neural networks have been in the leading position for ground-based cloud image classification tasks. However, this approach introduces too much inductive bias, fails to perform global modeling, and gradually tends to saturate the performance effect of convolutional neural network models as the amount of data increases. In this paper, we propose a novel method for ground-based cloud image recognition based on the multi-modal Swin Transformer (MMST), which discards the idea of using convolution to extract visual features and mainly consists of an attention mechanism module and linear layers. The Swin Transformer, the visual backbone network of MMST, enables the model to achieve better performance in downstream tasks through pre-trained weights obtained from the large-scale dataset ImageNet and can significantly shorten the transfer learning time. At the same time, the multi-modal information fusion network uses multiple linear layers and a residual structure to thoroughly learn multi-modal features, further improving the model's performance. MMST is evaluated on the multi-modal ground-based cloud public data set MGCD. Compared with the state-of-art methods, the classification accuracy rate reaches 91.30%, which verifies its validity in ground-based cloud image classification and proves that in ground-based cloud image recognition, models based on the Transformer architecture can also achieve better results.

Neural Network-Based Identification of Cloud Types from Ground-Based Images of Cloud Layers

Clouds are a significant factor in regional climates and play a crucial role in regulating the Earth’s water cycle through the interaction of sunlight and wind. Meteorological agencies around the world must regularly observe and record cloud data. Unfortunately, the current methods for collecting cloud data mainly rely on manual observation. This paper presents a novel approach to identifying ground-based cloud images to aid in the collection of cloud data. However, there is currently no publicly available dataset that is suitable for this research. To solve this, we built a dataset of surface-shot images of clouds called the SSC, which was overseen by the Macao Meteorological Society. Compared to previous datasets, the SSC dataset offers a more balanced distribution of data samples across various cloud genera and provides a more precise classification of cloud genera. This paper presents a method for identifying cloud genera based on cloud texture, using convolutional neural networks. To extract cloud texture effectively, we apply Gamma Correction to the images. The experiments were conducted on the SSC dataset. The results show that the proposed model performs well in identifying 10 cloud genera, achieving an accuracy rate of 80% for the top three possibilities.

Cloud detection method based on clear sky background under multiple weather conditions

This paper deals with an image processing methodology based on a clear sky library for cloud detection. It is a part of a project which aims at using the detection results of ground-based cloud images in combination with solar irradiation data to complete power forecasts for solar power plants at ultra-short-term horizons (15 min). This paper focuses on the implementation of the cloud detection process. To overcome the problem of detection failure caused by uneven background brightness distribution in the sky, detection work is achieved by building a library of clear sky background images for various weather conditions at different solar zenith angles and combining background elimination methods through image rotation and image matching algorithms. A hybrid algorithm is proposed for detection failure when the sun is blocked by the cloud to achieve secondary processing. The test results show that this method has good detection accuracy at the visual level compared to traditional algorithms, especially for situations with low cloud and sky discrepancies such as thin clouds and hazy weather.

Cloud-MobiNet: An Abridged Mobile-Net Convolutional Neural Network Model for Ground-Based Cloud Classification

More than 60 percent of the global surface is covered by clouds, and they play a vital role in the hydrological circle, climate change, and radiation budgets by modifying shortwaves and longwave. Weather forecast reports are critical to areas such as air and sea transport, energy, agriculture, and the environment. The time has come for artificial intelligence-powered devices to take the place of the current method by which decision-making experts determine cloud types. Convolutional neural network models (CNNs) are starting to be utilized for identifying the types of clouds that are caused by meteorological occurrences. This study uses the publicly available Cirrus Cumulus Stratus Nimbus (CCSN) dataset, which consists of 2543 ground-based cloud images altogether. We propose a model called Cloud-MobiNet for the classification of ground-based clouds. The model is an abridged convolutional neural network based on MobileNet. The architecture of Cloud-MobiNet is divided into two blocks, namely the MobileNet building block and the support MobileNet block (SM block). The MobileNet building block consists of the weights of the depthwise separable convolutions and pointwise separable convolutions of the MobileNet model. The SM block is made up of three dense network layers for feature extraction. This makes the Cloud-MobiNet model very lightweight to be implemented on a smartphone. An overall accuracy success of 97.45% was obtained for the CCSN dataset used for cloud-type classification. Cloud-MobiNet promises to be a significant model in the short term, since automated ground-based cloud classification is anticipated to be a preferred means of cloud observation, not only in meteorological analysis and forecasting but also in the aeronautical and aviation industries.

Integration Transformer for Ground-Based Cloud Image Segmentation

Recently, convolutional neural network (CNN) dominates the ground-based cloud image segmentation task, but disregards the learning of long-range dependencies due to the limited size of filters. Although Transformer-based methods could overcome this limitation, they only learn long-range dependencies at a single scale, hence failing to capture multi-scale information of cloud image. The multi-scale information is beneficial to ground-based cloud image segmentation, because the features from small scales tend to extract detailed information while features from large scales have the ability to learn global information. In this paper, we propose a novel deep network named Integration Transformer (InTransformer), which builds long-range dependencies from different scales. To this end, we propose the Hybrid Multi-head Transformer Block (HMTB) to learn multi-scale long-range dependencies, and hybridize CNN and HMTB as the encoder at different scales. The proposed InTransformer hybridizes CNN and Transformer as the encoder to extract multi-scale representations, which learns both local information and long-range dependencies with different scales. Meanwhile, in order to fuse the patch tokens with different scales, we propose Mutual Cross-Attention Module (MCAM) for the decoder of InTransformer which could adequately interact multi-scale patch tokens in a bidirectional way. We have conducted a series of experiments on large ground-based cloud detection database TLCDD and SWIMSEG. The experimental results show that the performance of our method outperforms other methods, proving the effectiveness of the proposed InTransformer.

Classification of Ground-Based Cloud Images by Contrastive Self-Supervised Learning

Clouds have an enormous influence on the hydrological cycle, Earth’s radiation budget, and climate changes. Accurate automatic recognition of cloud shape based on ground-based cloud images is beneficial to analyze the atmospheric motion state and water vapor content, and then to predict weather trends and identify severe weather processes. Cloud type classification remains challenging due to the variable and diverse appearance of clouds. Deep learning-based methods have improved the feature extraction ability and the accuracy of cloud type classification, but face the problem of lack of labeled samples. In this paper, we proposed a novel classification approach of ground-based cloud images based on contrastive self-supervised learning (CSSL) to reduce the dependence on the number of labeled samples. First, data augmentation is applied to the input data to obtain augmented samples. Then contrastive self-supervised learning is used to pre-train the deep model with a contrastive loss and a momentum update-based optimization. After pre-training, a supervised fine-tuning procedure is adopted on labeled data to classify ground-based cloud images. Experimental results have confirmed the effectiveness of the proposed method. This study can provide inspiration and technical reference for the analysis and processing of other types of meteorological remote sensing data under the scenario of insufficient labeled samples.

A Novel Method for Ground-Based Cloud Image Classification Using Transformer

In recent years, convolutional neural networks (CNNs) have achieved competitive performance in the field of ground-based cloud image (GCI) classification. Proposed CNN-based methods can fully extract the local features of images. However, due to the locality of the convolution operation, they cannot well establish the long-range dependencies between the images, and thus they cannot extract the global features of images. Transformer has been applied to computer vision with great success due to its powerful global modeling capability. Inspired by it, we propose a Transformer-based GCI classification method that combines the advantages of the CNN and Transformer models. Firstly, the CNN model acts as a low-level feature extraction tool to generate local feature sequences of images. Then, the Transformer model is used to learn the global features of the images by efficiently extracting the long-range dependencies between the sequences. Finally, a linear classifier is used for GCI classification. In addition, we introduce a center loss function to address the problem of the simple cross-entropy loss not adequately supervising feature learning. Our method is evaluated on three commonly used datasets: ASGC, CCSN, and GCD. The experimental results show that the method achieves 94.24%, 92.73%, and 93.57% accuracy, respectively, outperforming other state-of-the-art methods. It proves that Transformer has great potential to be applied to GCI classification tasks.

A Novel Ground-Based Cloud Image Segmentation Method Based on a Multibranch Asymmetric Convolution Module and Attention Mechanism

Cloud segmentation is a fundamental step in accurately acquiring cloud cover. However, due to the nonrigid structures of clouds, traditional cloud segmentation methods perform worse than expected. In this paper, a novel deep convolutional neural network (CNN) named MA-SegCloud is proposed for segmenting cloud images based on a multibranch asymmetric convolution module (MACM) and an attention mechanism. The MACM is composed of asymmetric convolution, depth-separable convolution, and a squeeze-and-excitation module (SEM). The MACM not only enables the network to capture more contextual information in a larger area but can also adaptively adjust the feature channel weights. The attention mechanisms SEM and convolutional block attention module (CBAM) in the network can strengthen useful features for cloud image segmentation. As a result, MA-SegCloud achieves a 96.9% accuracy, 97.0% precision, 97.0% recall, 97.0% F-score, 3.1% error rate, and 94.0% mean intersection-over-union (MIoU) on the Singapore Whole-sky Nychthemeron Image Segmentation (SWINySEG) dataset. Extensive evaluations demonstrate that MA-SegCloud performs favorably against state-of-the-art cloud image segmentation methods.

CloudRaednet: residual attention-based encoder–decoder network for ground-based cloud images segmentation in nychthemeron

ABSTRACT Obtaining accurate cloudage information through ground-based cloud observation is of great significance to astronomical telescope observatory site selection. This paper proposes a residual attention-based encoder–decoder network (CloudRAEDNet) for ground-based cloud image segmentation in nychthemeron. CloudRAEDNet uses ImageNet pre-trained ResNet50 as the encoder backbone network, which reduces the number of network training. The network decoder introduces residual modules to solve the problem of network degradation caused by the increase in the number of network layers. CloudRAEDNet connects encoder and decoder through attention gates to suppress the features of irrelevant areas and automatically focus on areas with prominent features. In addition, the segmentation performance of the network is further improved by the ranger optimizer. The comparative experimental results show that CloudRAEDNet can segment the local details of the ground-based cloud images more finely without increasing the time complexity. Compared with CloudSegNet, EFCN, CloudU-Net and CloudU-Netv2, CloudRAEDNet has the best segmentation performance. The results of ablation experiments show that the attention module contributes the most to CloudRAEDNet and the residual module contributes the least to CloudRAEDNet. In addition, the pre-training and Ranger optimizer also contribute to improving the segmentation performance of CloudRAEDNet.

Classification of Ground-Based Cloud Images by Improved Combined Convolutional Network

Changes in clouds can affect the outpower of photovoltaics (PVs). Ground-based cloud images classification is an important prerequisite for PV power prediction. Due to the intra-class difference and inter-class similarity of cloud images, the classical convolutional network is obviously insufficient in distinguishing ability. In this paper, a classification method of ground-based cloud images by improved combined convolutional network is proposed. To solve the problem of sub-network overfitting caused by redundancy of pixel information, overlap pooling kernel is used to enhance the elimination effect of information redundancy in the pooling layer. A new channel attention module, ECA-WS (Efficient Channel Attention–Weight Sharing), is introduced to improve the network’s ability to express channel information. The decision fusion algorithm is employed to fuse the outputs of sub-networks with multi-scales. According to the number of cloud images in each category, different weights are applied to the fusion results, which solves the problem of network scale limitation and dataset imbalance. Experiments are carried out on the open MGCD dataset and the self-built NRELCD dataset. The results show that the proposed model has significantly improved the classification accuracy compared with the classical network and the latest algorithms.