Short-term Solar Irradiance Forecasting Research Articles

Machine learning and deep learning models based grid search cross validation for short-term solar irradiance forecasting

In late 2023, the United Nations conference on climate change (COP28), which was held in Dubai, encouraged a quick move from fossil fuels to renewable energy. Solar energy is one of the most promising forms of energy that is both sustainable and renewable. Generally, photovoltaic systems transform solar irradiance into electricity. Unfortunately, instability and intermittency in solar radiation can lead to interruptions in electricity production. The accurate forecasting of solar irradiance guarantees sustainable power production even when solar irradiance is not present. Batteries can store solar energy to be used during periods of solar absence. Additionally, deterministic models take into account the specification of technical PV systems and may be not accurate for low solar irradiance. This paper presents a comparative study for the most common Deep Learning (DL) and Machine Learning (ML) algorithms employed for short-term solar irradiance forecasting. The dataset was gathered in Islamabad during a five-year period, from 2015 to 2019, at hourly intervals with accurate meteorological sensors. Furthermore, the Grid Search Cross Validation (GSCV) with five folds is introduced to ML and DL models for optimizing the hyperparameters of these models. Several performance metrics are used to assess the algorithms, such as the Adjusted R2 score, Normalized Root Mean Square Error (NRMSE), Mean Absolute Deviation (MAD), Mean Absolute Error (MAE) and Mean Square Error (MSE). The statistical analysis shows that CNN-LSTM outperforms its counterparts of nine well-known DL models with Adjusted R2 score value of 0.984. For ML algorithms, gradient boosting regression is an effective forecasting method with Adjusted R2 score value of 0.962, beating its rivals of six ML models. Furthermore, SHAP and LIME are examples of explainable Artificial Intelligence (XAI) utilized for understanding the reasons behind the obtained results.

Short-term solar irradiance forecasting under data transmission constraints

We report a data-parsimonious machine learning model for short-term forecasting of solar irradiance. The model follows the convolutional neural network – long-short term memory architecture. Its inputs include sky camera images that are reduced to scalar features to meet data transmission constraints. The model focuses on predicting the deviation of irradiance from the persistence of cloudiness (POC) model. Inspired by control theory, a noise signal input is used to capture the presence of unknown and/or unmeasured input variables and is shown to improve model predictions, often considerably. Five years of data from the NREL Solar Radiation Research Laboratory were used to create three rolling train-validate sets and determine the best representations for time, the optimal span of input measurements, and the most impactful model input data (features). For the chosen validation data, the model achieves a mean absolute error of 74.29 W/m2over a time horizon of up to two hours, compared to a baseline 134.35 W/m2 using the POC model.

Advancing short-term solar irradiance forecasting accuracy through a hybrid deep learning approach with Bayesian optimization

The optimization of solar energy integration into the power grid relies heavily on accurate forecasting of solar irradiance. In this study, a new approach for short-term solar irradiance forecasting is introduced. This method combines Bayesian Optimized Attention-Dilated Long Short-Term Memory and Savitzky-Golay filtering. The methodology is implemented to analyze data obtained from a solar irradiance probe situated in Douala, Cameroon. Initially, the unprocessed data is augmented by integrating distinctive solar irradiation variables, and the Savitzky-Golay filter with Bayesian Optimization is used to enhance its quality. Subsequently, multiple deep learning models, including Long Short-Term Memory, Bidirectional Long Short-Term Memory, Artificial Neural Networks, Bidirectional Long Short-Term Memory with Additive Attention Mechanism, and Bidirectional Long Short-Term Memory with Additive Attention Mechanism and Dilated Convolutional layers, are trained and evaluated. Out of all the models considered, the proposed approach, which combines the attention mechanism and dilated convolutional layers, demonstrates exceptional performance with the best convergence and accuracy in forecasting. Bayesian Optimization is further utilized to fine-tune the polynomial and window size of the Savitzky-Golay filter and optimize the hyperparameters of the deep learning models. The results show a Symmetric Mean Absolute Percentage Error of 0.6564, a Normalized Root Mean Square Error of 0.2250, and a Root Mean Square Error of 22.9445, surpassing previous studies in the literature. Empirical findings highlight the effectiveness of the proposed methodology in enhancing the accuracy of short-term solar irradiance forecasting. This research contributes to the field by introducing novel data pre-processing techniques, a hybrid deep learning architecture, and the development of a benchmark dataset. These advancements benefit both researchers and solar plant managers, improving solar irradiance forecasting capabilities.

All sky imaging-based short-term solar irradiance forecasting with Long Short-Term Memory networks

The intermittent nature of solar irradiance, primarily due to cloud movements, leads to rapid short-term fluctuations in the power output of photovoltaic (PV) systems. These fluctuations pose a significant challenge for integrating this renewable energy source into the power grid. Accurate forecasting of solar irradiance is not only crucial but also multi-beneficial. It enables more precise grid management by allowing operators to anticipate power output fluctuations and adjust energy distribution and storage strategies accordingly. This proactive approach reduces the reliance on backup power sources, which are often less sustainable and more expensive. Furthermore, accurate forecasts enhance the overall efficiency and reliability of energy systems by minimizing the impact of power variability on the grid, thereby supporting a more stable and sustainable energy supply.Addressing this need, our study focuses on the development of a forecasting model through innovative feature engineering, systematic design of specific attributes, and optimization of sequence length. The model is tailored to perform efficiently across various weather conditions and offers predictions for a time horizon of 0 to 20 min ahead. Utilizing a Long Short-Term Memory (LSTM) model, we achieve a remarkable ramp Forecast Skill Score of 39% in sunny and 25% in partially cloudy conditions. This work not only contributes to the existing literature but also presents a pioneering methodology for solar energy integration, highlighting the importance and application of accurate short-term solar irradiance forecasting.

Short-Term Solar Irradiance Prediction with a Hybrid Ensemble Model Using EUMETSAT Satellite Images

Accurate short-term solar irradiance forecasting is crucial for the efficient operation of solar energy-driven photovoltaic (PV) power plants. In this research, we introduce a novel hybrid ensemble forecasting model that amalgamates the strengths of machine learning tree-based models and deep learning neuron-based models. The hybrid ensemble model integrates the interpretability of tree-based models with the capacity of neuron-based models to capture complex temporal dependencies within solar irradiance data. Furthermore, stacking and voting ensemble strategies are employed to harness the collective strengths of these models, significantly enhancing the prediction accuracy. This integrated methodology is enhanced by incorporating pixels from satellite images provided by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT). These pixels are converted into structured data arrays and employed as exogenous inputs in the algorithm. The primary objective of this study is to improve the accuracy of short-term solar irradiance predictions, spanning a forecast horizon up to 6 h ahead. The incorporation of EUMETSAT satellite image pixel data enables the model to extract valuable spatial and temporal information, thus enhancing the overall forecasting precision. This research also includes a detailed analysis of the derivation of the GHI using satellite images. The study was carried out and the models tested across three distinct locations in Austria. A detailed comparative analysis was carried out for traditional satellite (SAT) and numerical weather prediction (NWP) models with hybrid models. Our findings demonstrate a higher skill score for all of the approaches compared to a smart persistent model and consistently highlight the superiority of the hybrid ensemble model for a short-term prediction window of 1 to 6 h. This research underscores the potential for enhanced accuracy of the hybrid approach to advance short-term solar irradiance forecasting, emphasizing its effectiveness at understanding the intricate interplay of the meteorological variables affecting solar energy generation worldwide. The results of this investigation carry noteworthy implications for advancing solar energy systems, thereby supporting the sustainable integration of renewable energy sources into the electrical grid.

A Review on Neural Network Based Models for Short Term Solar Irradiance Forecasting

The accuracy of solar energy forecasting is critical for power system planning, management, and operation in the global electric energy grid. Therefore, it is crucial to ensure a constant and sustainable power supply to consumers. However, existing statistical and machine learning algorithms are not reliable for forecasting due to the sporadic nature of solar energy data. Several factors influence the performance of solar irradiance, such as forecasting horizon, weather classification, and performance evaluation metrics. Therefore, we provide a review paper on deep learning-based solar irradiance forecasting models. These models include Long Short-Term Memory (LTSM), Gated Recurrent Unit (GRU), Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), Generative Adversarial Networks (GAN), Attention Mechanism (AM), and other existing hybrid models. Based on our analysis, deep learning models perform better than conventional models in solar forecasting applications, especially in combination with some techniques that enhance the extraction of features. Furthermore, the use of data augmentation techniques to improve deep learning performance is useful, especially for deep networks. Thus, this paper is expected to provide a baseline analysis for future researchers to select the most appropriate approaches for photovoltaic power forecasting, wind power forecasting, and electricity consumption forecasting in the medium term and long term.

Global Horizontal Irradiance in West Africa: Evaluation of the WRF-Solar Model in Convection-Permitting Mode with Ground Measurements

Abstract The number of solar power plants has increased in West Africa in recent years. Reliable reanalysis data and short-term forecasting of solar irradiance from numerical weather prediction models could provide an economic advantage for the planning and operation of solar power plants, especially in data-poor regions such as West Africa. This study presents a detailed assessment of different shortwave (SW) radiation schemes from the Weather Research and Forecasting (WRF) Model option Solar (WRF-Solar), with appropriate configurations for different atmospheric conditions in Ghana and the southern part of Burkina Faso. We applied two 1-way nested domains (D1 = 15 km and D2 = 3 km) to investigate four different SW schemes, namely, the Community Atmosphere Model, Dudhia, RRTMG, Goddard, and RRTMG without aerosol and with aerosol inputs (RRTMG_AERO). The simulation results were validated using hourly measurements from different automatic weather stations established in the study region in recent years. The results show that the RRTMG_AERO_D01 generally outperforms the other SW radiation schemes to simulate global horizontal irradiance under all-sky condition [RMSE = 235 W m−2 (19%); MAE = 172 W m−2 (14%)] and also under cloudy skies. Moreover, RRTMG_AERO_D01 shows the best performance on a seasonal scale. Both the RRTMG_AERO and Dudhia experiments indicate a good performance under clear skies. However, the sensitivity study of different SW radiation schemes in the WRF-Solar model suggests that RRTMG_AERO gives better results. Therefore, it is recommended that it be used for solar irradiance forecasts over Ghana and the southern part of Burkina Faso.

ACGL-TR: A deep learning model for spatio-temporal short-term irradiance forecast

With the vigorous development of renewable energy, the installed capacity of photovoltaic (PV) power plants continuously expands. For distributed PV systems, spatio-temporal short-term solar irradiance forecasting can give forecasts for multiple sites simultaneously and help improve the generation of the PV power. However, it is difficult to explore appropriate spatio-temporal relationship for high-dimensional and long-historical data. Thus, an Attention Graph Convolution Long short term memory neural network with Tensor Regression (AGCL-TR) is proposed. Firstly, the adaptive adjacent matrices, which represents the relationship among PV sites, are obtained directly from the encoded feature of the history meteorological data of multiple PV sites by self-attention mechanism. Secondly, graph feature along with original data are handled by AGCL to obtain spatio-temporal feature. Thirdly, a TR network is used to decode the higher-order spatio-temporal feature into next step irradiance forecasts for each PV site. The proposed method is validated on an open access dataset, the National Solar Radiation Database (NSRDB), with eight sites randomly selected in Nevada, North America. The experiment demonstrates that the proposed method outperforms the existing spatio-temporal prediction methods in each site, with an improvement of RMSE of 11.9%–19.8%. The proposed method can effectively extract and preserve the spatio-temporal features of high-dimensional data, which can improve the prediction performance in distributed PV systems.

Development of a Short-term Solar Irradiance Forecasting using Satellite Image in Combination with Numerical Weather Prediction Model

Development of a Short-term Solar Irradiance Forecasting using Satellite Image in Combination with Numerical Weather Prediction Model

Complex-valued time series based solar irradiance forecast

A new method for short-term probabilistic forecasting of global solar irradiance from complex-valued time series is explored. The measurement defines the real part of the time series while the estimate of the volatility is the imaginary part. A complex autoregressive model (capable to capture quick fluctuations) is then applied with data gathered on the Corsica island (France). Results show that even if this approach is easy to implement and requires very little resource and data, both deterministic and probabilistic forecasts generated by this model are in agreement with experimental data (root mean square error ranging from 0.196 to 0.325 considering all studied horizons). In addition, it exhibits sometimes a better accuracy than classical models such as the Gaussian process, bootstrap methodology, or even more sophisticated models such as quantile regression. Many studies and many fields of physics could benefit from this methodology and from the many models that could result from it.

Infinite hidden Markov model for short-term solar irradiance forecasting

Hidden state models are among the most widely used and efficient schemes for solar irradiance modeling in general and forecasting in particular. However, the complexity of such models – in terms of the number of states – is usually needed to be specified a priori. For solar irradiance data this assumption is very difficult to justify.In this paper, an infinite hidden Markov model (InfHMM) is introduced for short-term probabilistic forecasting of solar irradiance, where the assumption of fixed number of states a priori is relaxed and model complexity is determined during the model training. InfHMM is a non-parametric Bayesian model (NPB) indexed with an infinite dimensional parameter space which allows the automatic adaptation of the model to the “correct” complexity. This facilitates the automatic adaptation of the model to all weather conditions and locations. Posterior inference for InfHMM is performed using the Markov chain Monte Carlo algorithm, namely the beam sampler.Data from 13 different sources are used to validate the proposed model and subsequently it is compared to two well-established models in the literature: Markov-chain mixture distribution (MCM) and complete-history persistence ensemble (CH-PeEn) models. Important results are found, that cannot be derived from the existing finite models, such as the variation of the number of states within and across sites. The comparison of the models shows that the InfHMM is more consistent in term of the forecasting horizon.For reproducibility of the methodology presented in this paper, we have provided an R script for the InfHMM as supplementary material.

Short-Term Forecasting of Solar Irradiance using STL, Wavelet and LSTM

Short-Term Forecasting of Solar Irradiance using STL, Wavelet and LSTM

Multibranch Attentive Gated ResNet for Short-Term Spatio-Temporal Solar Irradiance Forecasting

The increasing penetration of solar generation into power grids has promoted the need for accurate and reliable short-term solar irradiance forecasting. This article introduces a novel multibranch attentive gated recurrent residual network (ResAttGRU) consisting of multiple branches of residual networks, gated recurrent units (GRUs), and the attention mechanism. The proposed multibranch ResAttGRU is capable of modeling data at various resolutions, extracting hierarchical features, and capturing short- and long-term dependencies. Moreover, this network also presents a strong multitimescale representative within the proposed architecture, while GRUs can exploit temporal information at less computational cost than the popular long short-term memory (LSTM). The novelty of the proposed architecture is to employ multiple convolutional-based branches with different filter lengths to learn multitimescale features jointly, accelerate the learning process, and reduce overfitting by leveraging shared representations as the auxiliary information. This study also compares the multibranch ResAttGRU networks with state-of-the-art deep learning methods using 18 years of NSRDB data at 12 solar sites. Finally, the proposed multibranch ResAttGRU requires 7.1% fewer parameters than multibranch residual LSTM while achieving similar average RMSE, MAE, and R-squared values.

Short-term forecasting of global solar irradiance in tropical environments with incomplete data

Electricity access is a common issue around the world. Many countries that face this problem are located in the tropical region and solar energy might be an alternative to mitigate this limitation. Accurate mechanisms for forecasting solar irradiance and insolation provide important information regarding the potential for generating solar energy. Furthermore, this data is relevant for the planning of renewable energy projects, and energy policy formulation. This research introduces a pipeline for the one-day ahead forecasting of solar irradiance and insolation that only requires solar irradiance historical data for training. Our approach includes a data imputation stage to handle missing data. In the prediction stage, we consider four data-driven approaches: Autoregressive Integrated Moving Average, Single Layer Feed Forward Network, Multiple Layer Feed Forward Network, and Long Short-Term Memory. The experiments are performed in a real-world dataset collected by 12 Automatic Weather Stations located in Nariño - Colombia. Our results show that the neural network-based models outperform Autoregressive Integrated Moving Average in most cases, and that Long Short-Term Memory exhibits better performance in cloudy environments (where more randomness is expected).

Short term solar irradiance forecasting using sky images based on a hybrid CNN–MLP model

High penetration of photovoltaics (PV) has been observed in the energy market over the last decade. However, its integration into electrical grids is challenging, as solar energy is highly fluctuating given its dependence on different weather variables. Consequently, short-term forecasting of solar irradiance provides a pivotal solution to ensure optimal use of the produced energy and reduce its uncertainty. This study proposes a hybrid convolutional neural network and Multilayer perceptron (CNN–MLP) model to forecast the global irradiance 15 min ahead. The model uses images from a hemispherical sky imager, time series of GHI, and weather variables collected from a ground meteorological station in Morocco. The evaluation of the proposed model under clear, mixed, and overcast days shows that the proposed model performs better than the persistence model. The root mean square error (RMSE) varies between 13.05 W/m2 and 49.16 W/m2 for CNN–MLP and between 45.76 W/m2 and 114.19 W/m2 for persistence. The coefficient of determination (R2) varies between 0.99 and 0.94 for the MLP–CNN and between 0.98 and 0.79 for persistence. The results show that the proposed model could be an appropriate choice for short-term forecasting even under cloudy conditions.

Multi-view deep forecasting for hourly solar irradiance with error correction

Short-term solar irradiance forecasting is crucial in managing power network operations and solar photovoltaic applications. In this paper, a Multi-view Deep Forecasting method with Error Correction (MvDF_EC) for 1-hour ahead solar forecasting is proposed. MvDF_EC comprises of the Multi-view Deep Forecasting method (MvDF) and a robust Radial Basis Function Neural Network trained via minimizing the Localized Generalization Error for compensating the solar forecasting error of MvDF. MvDF consists of three deep neural networks which learn representations of input data from different views. The three views are 1) the hierarchical local temporal information extracted by the Temporal Convolutional Neural Network (TCN), 2) the key context sequential information captured by the Bi-directional Long Short-Term Memory Neural Network with Temporal Attention (BLSTMattn), and 3) long-term temporal dependencies between local temporal patterns filtered by the Convolutional Gated Recurrent Unit Neural Network (C_GRU). The solar forecasting performance of the proposed MvDF_EC is evaluated with the National Solar Radiation Database. Simulation results show that MvDF_EC yields the most accurate solar prediction compared with the benchmarks including the smart persistence and the state-of-the-art models. The lowest relative Root Mean Square Error values for Maraba and Labelle are 22.08% and 27.40%, respectively in 1-hour ahead solar forecasting.

Automated construction of clear-sky dictionary from all-sky imager data

All-sky imagers (ASIs) have substantial promise as scalable sensors for short-term solar irradiance forecasting. Many of the current computational techniques that use ASIs for this purpose rely on collections of clear-sky images indexed by time of day, solar angle, or both, called clear-sky dictionaries (CSDs). These CSDs act as baselines against which images can be compared to locate and classify clouds within the image frame. CSDs are often compiled by hand, where individuals visually inspect collections of images one at a time to find clear-sky images. This process is not scalable, and it is prone to error. This paper proposes an automated, nonparametric alternative that uses the principles of digital image processing to find clear-sky images within a set of images taken over several days. We use ground-truth measurements of the clearness index to assess the performance of our method, and we show that the images it selects accurately correspond to clear-sky images. We also compare our proposal, which is nonparametric, with a state-of-the-art parametric method. The numerical results indicate that the performance of the method proposed here is superior.

Short-term global horizontal irradiance forecasting based on a hybrid CNN-LSTM model with spatiotemporal correlations

Abstract Accurate short-term solar irradiance forecasting is crucial for ensuring the optimum utilization of photovoltaic power generation sources. This study addresses this issue by proposing a spatiotemporal correlation model based on deep learning. The proposed model first applies a convolutional neural network (CNN) to extract spatial features from a two-dimensional matrix composed of meteorological parameters associated with a target site and its neighboring sites. Then, a long short-term memory (LSTM) network is applied to extract temporal features from historical solar irradiance time-series data associated with the target site. Finally, the spatiotemporal correlations are merged to predict global horizontal irradiance one hour in advance. The prediction performance and generalization ability of the proposed CNN-LSTM model are evaluated within a whole year, under diverse seasons and sky conditions. Three datasets are involved for case studies, which are collected from 34 locations spread across three different climate zones in Texas, USA. Moreover, the performance of the CNN-LSTM model is compared with those obtained using the CNN, LSTM, and other benchmark models based on five evaluation metrics. The results indicate that the proposed model has advantages over the other models considered and provides a good alternative for short-term solar radiation prediction.

Short-Term Solar Irradiance Forecasting and Photovoltaic System Management Using Octonion Neural Networks

In this paper, the octonion neural network is investigated to forecast the short-term solar irradiance. The previous and the next eight values solar irradiance are organized into two octonion values; thereby the network could be constructed. This method not just gives the opportunity to forecast eight values ahead solar irradiance using one octonion input but also takes all the advantages of the octonion domain. The octonion input contains the past values solar irradiance which produces dynamics naturally to the network and decreases the input dimension vector. The octonion training algorithm has eight dimensions rather than one dimension in the real-valued neural networks. Comparison with the real-valued neural networks for forecasting solar irradiance shows that the proposed method is promising to deal with such problem. The optimal structure is used to manage the an autonomous photovoltaic (PV) system that contains the PV modules and the battery bank. The use of the proposed method presents benefits for the number of the used modules and for the battery energy requested as well.

Spatio-Temporal Resolution of Irradiance Samples in Machine Learning Approaches for Irradiance Forecasting

Improving short term solar irradiance forecasting is crucial to increase the market share of the solar energy production. This paper analyzes the impact of using spatially distributed irradiance sensors as inputs to four machine learning algorithms: ARX, NN, RRF and RT. We used data from two different sensor networks for our experiments, the NREL dataset that includes data from 17 sensors that cover a 1 km^2 area and the InfoRiego dataset which includes data from 50 sensors that cover an area of 94 Km^2. Several studies have been published that use these datasets individually, to the author knowledge this is the flrst work that evaluates the influence of the spatially distributed data across a range from 0.5 to 17 sensors per km^2. We show that all of algorithms evaluated are able to take advantage of the data from the surroundings, from the very short forecast horizons of 10s up to a few hours, and that the wind direction and intensity plays an important role in the optimal distribution of the network and its density. We show that these machine learning methods are more effective on the short horizons when data is obtained from a dense enough network to capture the cloud movements in the prediction interval, and that in those cases complex non-linear models give better results. On the other hand, if only a sparse network is available, the simpler linear models give better results. The skills obtained with the models under test range from 13% to 70%, depending on the sensor network density, time resolution and lead time.