Electricity Price Forecasting Research Articles

Forecasting the Spot Market Electricity Price with a Long Short-Term Memory Model Architecture in a Disruptive Economic and Geopolitical Context

In this paper, we perform a short-run Electricity Price Forecast (EPF) with a Recurrent Neural Network (RNN), namely Long Short-Term Memory (LSTM), using an algorithm that selects the variables and optimizes the hyperparameters. The results are compared with one of the standout machine learning algorithms, namely eXtreme Gradient Boosting (XGB). Apart from other EPF solutions, in this paper, we focus on the interval before and after the pandemic and the conflict in Ukraine. Furthermore, compared to the previous papers that mainly approached German, Austrian, Australian, Spanish, Nordic electricity Day Ahead Markets (DAM), we emphasize on the EPF for one of the East-European countries—Romania whose market rules closely align with the rules of the European Union electricity DAM. The contribution of this study consists in creating a data set that spans from January 2019 to August 2022 and providing an algorithm to identify the best stacked LSTM architecture to cope with a challenging short-term EPF. The proposed algorithm identifies the most relevant variables using a correlation threshold and performs a combination of three parameters—hidden layer size, dropout and learning rate generating the best EPF results.

Day-Ahead Electricity Price Probabilistic Forecasting Based on SHAP Feature Selection and LSTNet Quantile Regression

Electricity prices are a central element of the electricity market, and accurate electricity price forecasting is critical for market participants. However, in the context of increasingly integrated economic markets, the complexity of the electricity system has increased. As a result, the number of factors required to consider in electricity price forecasting is growing. In addition, the high percentage of renewable energy penetration has increased the volatility of electricity generation, making it more challenging to predict prices accurately. In this paper, we propose a probabilistic forecasting method based on SHAP (SHapley Additive exPlanation) feature selection and LSTNet (long- and short-term time-series network) quantile regression. First, to reduce feature redundancy and overfitting, we use the SHAP method to perform feature selection in a high-dimensional input feature set, and specifically analyze the magnitude and manner in which features affect electricity prices. Second, we apply the LSTNet quantile regression model to predict the electricity value under different quantiles. Finally, the probability density function and the prediction interval of the predicted electricity prices are obtained by kernel density estimation. The case of the Danish electricity market validates the effectiveness and accuracy of our proposed method. The accuracy of the proposed method is better than that of other methods, and we assess the importance and direction of the impact of features on electricity prices.

Electricity Market Price Prediction Based on Quadratic Hybrid Decomposition and THPO Algorithm

Electricity price forecasting is a crucial aspect of spot trading in the electricity market and optimal scheduling of microgrids. However, the stochastic and periodic nature of electricity price sequences often results in low accuracy in electricity price forecasting. To address this issue, this study proposes a quadratic hybrid decomposition method based on ensemble empirical modal decomposition (EEMD) and wavelet packet decomposition (WPD), along with a deep extreme learning machine (DELM) optimized by a THPO algorithm to enhance the accuracy of electricity price prediction. To overcome the problem of the optimization algorithm falling into local optima, an improved optimization algorithm strategy is proposed to enhance the optimization-seeking ability of HPO. The electricity price series is decomposed into a series of components using EEMD decomposition and WPD decomposition, and the DELM model optimized by the THPO algorithm is built for each component separately. The predicted values of all the series are then superimposed to obtain the final electricity price prediction. The proposed prediction model is evaluated using electricity price data from an Australian electricity market. The results demonstrate that the proposed improved algorithm strategy significantly improves the convergence performance of the algorithm, and the proposed prediction model effectively enhances the accuracy and stability of electricity price prediction, as compared to several other prediction models.

Analyzing and forecasting electricity price using regime‐switching models: The case of New Zealand market

AbstractThis paper aims to study the forecasting capabilities of several models under the Markov regime‐switching (MRS) and the extreme value theory (EVT) frameworks applied to daily electricity prices in the New Zealand electricity market. The MRS models in this study include up to five regimes, with time‐varying transition probabilities and incorporation of external market variables. We apply Hamilton's filter with maximum likelihood estimation for parameter estimation. The EVT peaks‐over‐threshold (EVT‐PoT) framework is also considered, and its relationship to the MRS class of models is discussed. We generate out‐of‐sample forecasts under various market scenarios. The MRS models are able to replicate real price densities under stable market conditions. The EVT‐PoT model performs well despite its lack of complexity compared to the MRS framework. We attribute this to the usage of the generalized Pareto distribution to model price extremities.

Implication of machine learning techniques to forecast the electricity price and carbon emission: Evidence from a hot region

The current study examines the significant determinants of electricity consumption and identifies an appropriate model to forecast the electricity price accurately. The main contribution is focused on eastern region of Saudi Arabia, a relatively hottest geographical area full of energy resources but with different electricity consumption patterns. The relative irrelevance of temperature as predicting factor of electricity consumption is quite surprising and contradicts the previous studies. In the eastern region, electricity price has negative association with electricity consumption. While comparing traditional and machine learning, it is found that machine learning techniques offer better predictability. Amongst the machine learning techniques, the support vector machine has the lowest errors in forecasting the electricity price. Additionally, the support vector machine approach is used to forecast the trend of carbon emissions caused by electricity consumption. The findings have policy implications and offer valuable suggestions to policymakers while addressing the determinants of electricity consumption and forecasting electricity prices.

Multivariate probabilistic forecasting of intraday electricity prices using normalizing flows

Electricity is traded on various markets with different time horizons and regulations. Short-term intraday trading becomes increasingly important due to the higher penetration of renewables. In Germany, the intraday electricity price typically fluctuates around the day-ahead price of the European Power EXchange (EPEX) spot markets in a distinct hourly pattern. This work proposes a probabilistic modeling approach that models the intraday price difference to the day-ahead contracts. The model captures the emerging hourly pattern by considering the four 15min intervals in each day-ahead price interval as a four-dimensional joint probability distribution. The resulting nontrivial, multivariate price difference distribution is learned using a normalizing flow, i.e., a deep generative model that combines conditional multivariate density estimation and probabilistic regression. Furthermore, this work discusses the influence of different external impact factors based on literature insights and impact analysis using explainable artificial intelligence (XAI). The normalizing flow is compared to an informed selection of historical data and probabilistic forecasts using a Gaussian copula and a Gaussian regression model. Among the different models, the normalizing flow identifies the trends with the highest accuracy and has the narrowest prediction intervals. Both the XAI analysis and the empirical experiments highlight that the immediate history of the price difference realization and the increments of the day-ahead price have the most substantial impact on the price difference.

Short-Term Electricity Price Forecasting Using EEMD and GRU-NN

A vital necessity for the reliable operation of the power system as well as for the financial well-being of the consumers is an accurate forecast of the price of electricity. The price of electricity is highly volatile, nonlinear, and subject to seasonal fluctuations. The price series is also affected by electrical market uncertainty and demand, which is strongly reliant on weather and electricity usage time. This study presents a short-term electricity price forecasting method that follows the similar days approach comprising Ensemble Empirical Mode Decomposition technique. The suggested method utilizes a feature selection methodology that combines the Random Forest Regressor and the Gradient Boosting Regressor to determine which features are most important for the machine learning model. To forecast the electricity price, a Gated Recurrent Unit Neural Network (GRU-NN) is employed as the machine learning model. The GRU-NN has the capability of accurately capturing complicated temporal relationships in electricity price time series, which enables it to make correct predictions. To evaluate the validity of the proposed method, data from the PJM electricity market have been used. The simulation results demonstrate that the suggested method is superior to the existing technique, with significantly improved values achieved for both the mean absolute percentage error (MAPE) and the root mean square error (RMSE).

A Predictive Fuzzy Logic Model for Forecasting Electricity Day-Ahead Market Prices for Scheduling Industrial Applications

Electricity price forecasting (EPF) has become an essential part of decision-making for energy companies to participate in power markets. As the energy mix becomes more uncertain and stochastic, this process has also become important for industrial companies, as their production schedules are greatly impacted by energy costs. Although various approaches have been tested with varying degrees of success, this study focuses on predicting day-ahead market (DAM) prices in different European markets and how this directly affects the optimal production scheduling for various industrial loads. We propose a fuzzy-based architecture that incorporates the results of two forecasting algorithms; a random forest (RF) and a long short-term memory (LSTM). To enhance the accuracy of the proposed model for a specific country, electricity market data from neighboring countries are also included. The developed DAM price forecaster can then be utilized by energy-intensive industries to optimize their production processes to reduce energy costs and improve energy-efficiency. Specifically, the tool is important for industries with multi-site production facilities in neighboring countries, which could reschedule the production processes depending on the forecasted electricity market price.

Artificial Intelligence-Based Prediction of Spanish Energy Pricing and Its Impact on Electric Consumption

The energy supply sector faces significant challenges, such as the ongoing COVID-19 pandemic and the ongoing conflict in Ukraine, which affect the stability and efficiency of the energy system. In this study, we highlight the importance of electricity pricing and the need for accurate models to estimate electricity consumption and prices, with a focus on Spain. Using hourly data, we implemented various machine learning models, including linear regression, random forest, XGBoost, LSTM, and GRU, to forecast electricity consumption and prices. Our findings have important policy implications. Firstly, our study demonstrates the potential of using advanced analytics to enhance the accuracy of electricity price and consumption forecasts, helping policymakers anticipate changes in energy demand and supply and ensure grid stability. Secondly, we emphasize the importance of having access to high-quality data for electricity demand and price modeling. Finally, we provide insights into the strengths and weaknesses of different machine learning algorithms for electricity price and consumption modeling. Our results show that the LSTM and GRU artificial neural networks are the best models for price and consumption modeling with no significant difference.

WITHDRAWN: Short term optimal scheduling of a virtual power plants integrated with electrothermal hybrid energy storage system using an effective adaptive multi-objective differential evolution algorithm

virtual power plants (VPPs) in the power distribution grid play an important role in balancing the power and heat generation and consumption in residential, commercial, and industrial sectors. This study proposes VPPs for risk-averse energy exchange of resources and customers in the presence of the hybrid storage system. An appropriate energy management program is presented to make a balance between generators and consumers in VPPs considering generation uncertainties including electricity demand and price forecasting in the electricity market and renewable energy uncertainties. Hence, monetary exchange methods such as day-ahead agreements are proposed for energy exchange among VPPs. The expanded exchange structure of the suggested framework simplifies the renewable energy's sources to the electricity market despite its small-scale and non-dispatchable features. Also, a novel model is presented for short-term programing of virtual power plant considering technical and economic uncertainties. Given that VPPs can enter the electricity market based on time-of-use (TOU) pricing strategy and participation in demand response (DR) program, minimizing the expected cost of energy generation and environmental pollution is considered as the objective function. A multi-objective efficient model of energy management employing the machine speed scaling mechanism is suggested. An effective adaptive multi-objective differential evolution (AMODE) algorithm is presented to solve the multi-objective optimization problem. AMODE usages a novel Fitness Evaluation Mechanism (FEM) using dynamic reference point and fuzzy association randomness study to evaluate solutions in the evolutionary population by implementing the conditional value at risk (CVaR) method for risk management. The simulation studies and sensitivity analysis results are evaluated to confirm the effectiveness of the proposed model on a VPP in the island and grid-connected modes with and without participation in DR program. Finally, the conceptual results are discussed.

Coupling the Empirical Wavelet and the Neural Network Methods in Order to Forecast Electricity Price

This paper aims to evaluate the forecast capability of electricity markets, categorized by numerous major characteristics such as non-stationarity, nonlinearity, highest volatility, high frequency, mean reversion and multiple seasonality, which give multifarious forecasts. To improve it, this investigation proposes a new hybrid approach that links a dual long-memory process (Gegenbauer autoregressive moving average (GARMA) and generalized long-memory GARCH (G-GARCH)) and the empirical wavelet transform (EWT) and local linear wavelet neural network (LLWNN) approaches, forming the k-factor GARMA-EWLLWNN model. The future hybrid model accomplished is assessed via data from the Polish electricity markets, and it is matched with the generalized long-memory k-factor GARMA-G-GARCH process and the hybrid EWLLWNN, to demonstrate the robustness of our approach. The obtained outcomes show that the suggested model presents important results to define the relevance of the modeling approach that offers a remarkable framework to reproduce the inherent characteristics of the electricity prices. Finally, it is presented that the adopted methodology is the most appropriate one for prediction as it realizes a better prediction performance and may be an answer for forecasting electricity prices.

Explainability-based Trust Algorithm for electricity price forecasting models

Advanced machine learning (ML) algorithms have outperformed traditional approaches in various forecasting applications, especially electricity price forecasting (EPF). However, the prediction accuracy of ML reduces substantially if the input data is not similar to the ones seen by the model during training. This is often observed in EPF problems when market dynamics change owing to a rise in fuel prices, an increase in renewable penetration, a change in operational policies, etc. While the dip in model accuracy for unseen data is a cause for concern, what is more, challenging is not knowing when the ML model would respond in such a manner. Such uncertainty makes the power market participants, like bidding agents and retailers, vulnerable to substantial financial loss caused by the prediction errors of EPF models. Therefore, it becomes essential to identify whether or not the model prediction at a given instance is trustworthy. In this light, this paper proposes a trust algorithm for EPF users based on explainable artificial intelligence techniques. The suggested algorithm generates trust scores that reflect the model’s prediction quality for each new input. These scores are formulated in two stages: in the first stage, the coarse version of the score is formed using correlations of local and global explanations, and in the second stage, the score is fine-tuned further by the Shapley additive explanations values of different features. Such score-based explanations are more straightforward than feature-based visual explanations for EPF users like asset managers and traders. A dataset from Italy’s and ERCOT’s electricity market validates the efficacy of the proposed algorithm. Results show that the algorithm has more than 85% accuracy in identifying good predictions when the data distribution is similar to the training dataset. In the case of distribution shift, the algorithm shows the same accuracy level in identifying bad predictions.

Electricity price forecasting using hybrid deep learned networks

AbstractThis paper presents a novel hybrid model integrating maximal overlap discrete wavelet transform (MODWT) denoising and empirical mode decomposition (EMD) with sequence‐to‐sequence (seq2seq) long short‐term memory (LSTM) neural networks for day‐ahead electricity price forecasting. The nonstationary and nonlinear time series electricity price data are first denoised using MODWT. The resulting signal is decomposed into several intrinsic mode functions (IMF) with different resolutions by EMD. The extracted IMF is then introduced into seq2seq LSTM to obtain an aggregated predicted value for electricity price. The proposed method is examined using the Nord pool Elspot energy market data. Empirical results show that the proposed model outperformed the other forecasting models like LSTM and stacked LSTM. The performance measures indicate that data denoising can significantly improve the prediction stability and the generalization ability of the LSTM model.

ELECTRICITY PRICE FORECASTING: A SYSTEMATIC LITERATURE REVIEW INFORMED BY TEXT MINING

Developing forecasting models is a difficult task. Particularly concerning electricity prices, accurately predicting their forthcoming values makes it possible to minimize planning risks. This fact becomes even more relevant in the current geopolitical scenario, represented by the war between Russia and Ukraine. Given the above, this paper presents a systematic review of the literature on electricity price forecasting (EPF) models. It presents a methodology that does a robust search of the literature, obtaining the most relevant papers (n = 554) that addressed this theme. Following that search, we: (i) constructed an attribute matrix of the publications, and (ii) presented a descriptive analysis based on bibliographic data, and network relationships. The sample period comprises the years 1991 to 2019, with an annual growth rate equal to 23.13% and an annual publication rate of 19 papers. Despite the increase in the number of studies on electricity price forecasting, the predominance of papers is produced in only a few countries. This fact reinforces the need to encourage research and development projects related to the energy market. It was also found that research collaboration networks are still weak, highlighting the need for new partnerships between countries, and research institutions. Thus, stimulating global energy security, as well as encouraging cooperation and technology transfer between countries, becomes relevant.

Electricity Price Forecasting via Statistical and Deep Learning Approaches: The German Case

Our research involves analyzing the latest models used for electricity price forecasting, which include both traditional inferential statistical methods and newer deep learning techniques. Through our analysis of historical data and the use of multiple weekday dummies, we have proposed an innovative solution for forecasting electricity spot prices. This solution involves breaking down the spot price series into two components: a seasonal trend component and a stochastic component. By utilizing this approach, we are able to provide highly accurate predictions for all considered time frames.

Deep distributional time series models and the probabilistic forecasting of intraday electricity prices

SummaryRecurrent neural networks (RNNs) with rich feature vectors of past values can provide accurate point forecasts for series that exhibit complex serial dependence. We propose two approaches to constructing deep time series probabilistic models based on a variant of RNN called an echo state network (ESN). The first is where the output layer of the ESN has stochastic disturbances and a Bayesian prior for regularization. The second employs the implicit copula of an ESN with Gaussian disturbances, which is a Gaussian copula process on the feature space. Combining this copula process with a nonparametrically estimated marginal distribution produces a distributional time series model. The resulting probabilistic forecasts are deep functions of the feature vector and marginally calibrated. In both approaches, Markov chain Monte Carlo methods are used to estimate the models and compute forecasts. The proposed models are suitable for the complex task of forecasting intraday electricity prices. Using data from the Australian market, we show that our deep time series models provide accurate short‐term probabilistic price forecasts, with the copula model dominating. Moreover, the models provide a flexible framework for incorporating probabilistic forecasts of electricity demand, which increases upper tail forecast accuracy from the copula model significantly.

Swarm-based hybrid optimization algorithms: an exhaustive analysis and its applications to electricity load and price forecasting.

In this work, we intend to propose multiple hybrid algorithms with the idea of giving a choice to the particles of a swarm to update their position for the next generation. To implement this concept, Cuckoo Search Algorithm (CSA), Grey Wolf Optimization (GWO), Harris Hawks Optimization (HHO), and Whale Optimization Algorithm (WOA) have been utilized. Exhaustive possible combinations of these algorithms are developed and benchmarked against the base algorithms. These hybrid algorithms have been validated on twenty-four well-known unimodal and multimodal benchmarks functions, and detailed analysis with varying dimensions and population size is discussed for the same. Further, the efficacy of these algorithms has been tested on short-term electricity load and price forecasting applications. For this purpose, the algorithms have been combined with Artificial Neural Networks (ANNs) to evaluate their performance on the ISO New Pool England dataset. The results demonstrate that hybrid optimization algorithms perform superior to their base algorithms in most test cases. Furthermore, the results show that the performance of CSA-GWO is significantly better than other algorithms.

From point forecasts to multivariate probabilistic forecasts: The Schaake shuffle for day-ahead electricity price forecasting

Modeling price risks is crucial for economic decision making in energy markets. Besides the risk associated with a single price, the dependence structure of multiple prices is often relevant. We therefore propose a generic and easy-to-implement method for generating multivariate probabilistic forecasts based on univariate point forecasts of day-ahead electricity prices. While each univariate point forecast refers to one of the day’s 24 hours, the multivariate forecast distribution models dependencies across hours. The proposed method is based on simple copula techniques and an optional time series component. We illustrate the method for five benchmark data sets. Furthermore, we demonstrate an example for constructing realistic prediction intervals for the weighted sum of consecutive electricity prices as needed for pricing individual load profiles.

Electricity Load and Price Forecasting Using a Hybrid Method Based Bidirectional Long Short-Term Memory with Attention Mechanism Model

The accuracy of forecasting short- or medium-term electricity loads and prices is critical in the energy market. The amplitude and duration of abnormally high prices and load spikes can be detrimental to retailers and production systems. Therefore, predicting these spikes to effectively manage risk is critical. In this paper, a novel hybrid method that combines ensemble empirical mode decomposition (EEMD) algorithm and a bidirectional long short-term memory with attention mechanism (BiLSTM-AM) model is proposed to predict electricity loads and prices. A simple approach is proposed to determine the number of intrinsic mode functions (IMFs) that decompose raw data using EEMD to avoid overdecomposition, irrelevant components, and high computational cost. Each selected mode is then modeled with BiLSTM-AM to obtain a predicted sequence. These sequences are summed and then reverted to obtain the final predicted value. The proposed method is validated using two datasets (PMJ and Australian Energy Market Operator) with different time intervals to demonstrate the generality and robustness of the forecasts, especially in temporal valley or peak forecasting. The results show that the proposed method outperforms other methods in prediction accuracy and spike-capturing ability, with EEMD reducing the mean absolute percentage error (MAPE) by 53%, 54%, and 60%, respectively. In the three forecast periods, the average MAPE and R ̲ 2 are 0.097 and 0.92, respectively. Furthermore, we use Kolmogorov-Smirnov predictive accuracy (KSPA) test and model confidence set (MCS) test to validate the superiority of the proposed model. The results demonstrate its suitability, reliability, and performance in short- and medium-term forecasting.

Transfer learning for electricity price forecasting

Electricity price forecasting is an essential task in all the deregulated markets of the world. The accurate prediction of day-ahead electricity prices is an active research field and available data from various markets can be used as input for forecasting. A collection of models have been proposed for this task, but the fundamental question on how to use the available big data is often neglected. In this paper, we propose to use transfer learning as a tool for utilizing information from other electricity price markets for forecasting. We pre-train a neural network model on source markets and finally do a fine-tuning for the target market. Moreover, we test different ways to use the rich input data from various electricity price markets to forecast 24 steps ahead in hourly frequency. Our experiments on four different day-ahead markets indicate that transfer learning improves the electricity price forecasting performance in a statistically significant manner. Furthermore, we compare our results with state-of-the-art methods in a rolling window scheme to demonstrate the performance of the transfer learning approach. Our method improves the performance of the state-of-the-art algorithms by 7% for the French market and 3% for the German market.