Day-ahead Electricity Price Research Articles

Novel Custom Loss Functions and Metrics for Reinforced Forecasting of High and Low Day-Ahead Electricity Prices Using Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) and Ensemble Learning

Day-ahead electricity price forecasting (DAEPF) is vital for participants in energy markets, particularly in regions with high integration of renewable energy sources (RESs), where price volatility poses significant challenges. The accurate forecasting of high and low electricity prices is particularly essential, as market participants seek to optimize their strategies by selling electricity when prices are high and purchasing when prices are low to maximize profits and minimize costs. In Japan, the increasing integration of RES has caused day-ahead electricity prices to frequently fall to almost zero JPY/kWh during periods of high RES output, creating significant profitability challenges for electricity retailers. This paper introduces novel custom loss functions and metrics specifically designed to improve the forecasting accuracy of extreme prices (high and low prices) in DAEPF, with a focus on the Japanese wholesale electricity market, addressing the unique challenges posed by the volatility of RES. To implement this, we integrate these custom loss functions into a Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) model, augmented by an ensemble learning approach and multimodal features. The proposed custom loss functions and metrics were rigorously validated, demonstrating their effectiveness in accurately predicting high and low electricity prices, thereby indicating their practical application in enhancing the economic strategies of market participants.

Extension of a low-tech Model Predictive Control (MPC) algorithm for grid-supportive heat pump operation

This paper presents the further development of a low-tech Model Predictive Control (MPC) for the grid-friendly operation of heat pumps. The developed algorithm is already available in the literature [1] and successfully implemented in multiple buildings. The overall goal is, to predictively optimize comfort parameters taking into account weather forecast data for 48 h. A simple mathematical building model and an optimization function are used to calculate the heating (or cooling) requirement and determine the optimum heating (cooling) output curve over time.Based on the initial results, the algorithm is extended to include an energy price forecast. This modification enables the simultaneous optimization of comfort and heat supply costs. This forecast-based cost & comfort function is designed for the use of heat pumps in buildings with thermally activated components (TAB). The extension enables the MPC to process day-ahead electricity prices or other price signals in addition to weather forecast data. This means that the operation of heat pumps can be shifted to periods that are beneficial to the grid.The algorithm is analysed and validated in a Matlab/Simulink building simulation for a heating period of one sample month. A case study with different price scenarios is the main part of this paper. Depending on the assumed price fluctuation, the simulation leads to cost savings between 6.65 % and 12.5 % for the observation period of one month. The simulation results thus show that heat generation can be shifted to times with lower electricity prices, which leads to a significant reduction in heating costs without any significant loss of comfort. In conclusion, the developed low-tech MPC enables grid-supportive operation of heat pumps.

Deep learning-based electricity price forecasting: Findings on price predictability and European electricity markets

Deep learning algorithms have transformed day-ahead electricity price predictions in recent years. This phenomenon has implications for price predictability and perceived risks, which in turn could impact investments. This study aims to unravel the impact of deep learning-based price forecasting (DPF) on price predictability and its correlation with price volatility. This has been done by examining 19 bidding zones across 11 European countries from January 2018 to June 2023. This timeframe captures the unique market dynamics influenced by the COVID-19 pandemic and the Global Energy Crisis. This analysis has been enabled by the development of the Day-ahead European Electricity Price Forecasting Kit (Deepforkit), a forecasting tool developed in-house and open-sourced. Notably, Deepforkit offers a running speed at least 5 times faster than a previously suggested benchmark, enabling large-scale and multi-market analyses such as the one conducted in this study. It is quantitatively proven that if DPF is used, price volatility is just moderately correlated to unpredictability, and using it as a proxy for price unpredictability is not reliable. This is attributed to the capacity of DPFs to adjust their predictions in response to evolving market trends and patterns. This finding has an important implication on the investment risk perception and the cost of capital for new investments in the power sector. Moreover, the predictability behavior of prices is compared and contrasted in 3 markets, namely EPEX SPOT, OMIE, and Nord Pool, and across 11 European countries (Belgium, Denmark, Finland, France, Germany, Netherlands, Norway, Portugal, Sweden, Spain, and Switzerland).

A hybrid forecasting method considering the long-term dependence of day-ahead electricity price series

Day-ahead electricity market is crucial for ensuring the balance of electricity supply and demand. Its electricity price serves as a key guide for market participants and power dispatching agencies, making accurate short-term price forecasting essential. In this paper, a hybrid short-term electricity price forecasting method considering the long-term dependence of wavelet transform (WT) series is proposed. Different from the existing studies that do not distinguish the characteristics of WT series, this paper uses Hurst exponent and autocorrelation function (ACF) and partial autocorrelation function (PACF) plots to analyze the long-term dependence of subseries, and chooses seasonal autoregressive integrated moving average (SARIMA) or long short-term memory (LSTM) method to construct a hybrid forecasting model according to the characteristics of each subseries. Validation using real electricity prices from the NYISO and PJM markets shows that, compared to methods that do not consider long-term dependencies of subseries, the proposed method can improve prediction accuracy by up to 50.98 % and stability by up to 68.09 %.

Impact of Contracts for Differences for non-carbon electricity generation on efficiency of electricity market

In response to the recent energy crisis, the European Commission proposed two-way Contracts for Differences (CfDs) to secure renewable energy investments and protect consumers. The proposed CfDs, however, raised concerns regarding the potential impact on electricity prices. This study examines the effects of various CfD design elements on day-ahead electricity prices, welfare, and renewable energy production. Using a partial equilibrium model of an international electricity market including thirteen European countries, we find a trade-off between stimulating renewable electricity producers to respond to negative prices, and ensuring producer certainty. We also show potential market distortions for CfDs with longer-term reference prices, particularly when strike prices deviate from expectations. Additionally, our study underscores the interactions among various players in the electricity market, in particular storage operators and international traders, and how these interactions can mitigate potential distortions from CfDs. Therefore, policymakers should carefully consider the design elements of CfDs, and weigh the promotion of renewable energy production against the efficiency of markets. Moreover, a proper reference price should be selected, to avoid further increasing electricity prices in periods with high electricity prices.

Enhanced Day-Ahead Electricity Price Forecasting Using a Convolutional Neural Network–Long Short-Term Memory Ensemble Learning Approach with Multimodal Data Integration

Day-ahead electricity price forecasting (DAEPF) holds critical significance for stakeholders in energy markets, particularly in areas with large amounts of renewable energy sources (RES) integration. In Japan, the proliferation of RES has led to instances wherein day-ahead electricity prices drop to nearly zero JPY/kWh during peak RES production periods, substantially affecting transactions between electricity retailers and consumers. This paper introduces an innovative DAEPF framework employing a Convolutional Neural Network–Long Short-Term Memory (CNN–LSTM) model designed to predict day-ahead electricity prices in the Kyushu area of Japan. To mitigate the inherent uncertainties associated with neural networks, a novel ensemble learning approach is implemented to bolster the DAEPF model’s robustness and prediction accuracy. The CNN–LSTM model is verified to outperform a standalone LSTM model in both prediction accuracy and computation time. Additionally, applying a natural logarithm transformation to the target day-ahead electricity price as a pre-processing technique has proven necessary for higher prediction accuracy. A novel “policy-versus-policy” strategy is proposed to address the prediction problem of the zero prices, halving the computation time of the traditional two-stage method. The efficacy of incorporating a suite of multimodal features: areal day-ahead electricity price, day-ahead system electricity price, areal actual power generation, areal meteorological forecasts, calendar forecasts, alongside the rolling features of areal day-ahead electricity price, as explanatory variables to significantly enhance DAEPF accuracy has been validated. With the full integration of the proposed features, the CNN–LSTM ensemble model achieves its highest accuracy, reaching performance metrics of R2, MAE, and RMSE of 0.787, 1.936 JPY/kWh, and 2.630 JPY/kWh, respectively, during the test range from 1 March 2023 to 31 March 2023, underscoring the advantages of a comprehensive, multi-dimensional approach to DAEPF.

A Two-Stage Forecasting Approach for Day-Ahead Electricity Price Based on Improved Wavelet Neural Network With ELM Initialization

A Two-Stage Forecasting Approach for Day-Ahead Electricity Price Based on Improved Wavelet Neural Network With ELM Initialization

Forecasting with an N-dimensional Langevin equation and a neural-ordinary differential equation.

Accurate prediction of electricity day-ahead prices is essential in competitive electricity markets. Although stationary electricity-price forecasting techniques have received considerable attention, research on non-stationary methods is comparatively scarce, despite the common prevalence of non-stationary features in electricity markets. Specifically, existing non-stationary techniques will often aim to address individual non-stationary features in isolation, leaving aside the exploration of concurrent multiple non-stationary effects. Our overarching objective here is the formulation of a framework to systematically model and forecast non-stationary electricity-price time series, encompassing the broader scope of non-stationary behavior. For this purpose, we develop a data-driven model that combines an N-dimensional Langevin equation (LE) with a neural-ordinary differential equation (NODE). The LE captures fine-grained details of the electricity-price behavior in stationary regimes but is inadequate for non-stationary conditions. To overcome this inherent limitation, we adopt a NODE approach to learn, and at the same time predict, the difference between the actual electricity-price time series and the simulated price trajectories generated by the LE. By learning this difference, the NODE reconstructs the non-stationary components of the time series that the LE is not able to capture. We exemplify the effectiveness of our framework using the Spanish electricity day-ahead market as a prototypical case study. Our findings reveal that the NODE nicely complements the LE, providing a comprehensive strategy to tackle both stationary and non-stationary electricity-price behavior. The framework's dependability and robustness is demonstrated through different non-stationary scenarios by comparing it against a range of basic naïve methods.

An Interval Prediction Method for Day-Ahead Electricity Price in Wholesale Market Considering Weather Factors

An Interval Prediction Method for Day-Ahead Electricity Price in Wholesale Market Considering Weather Factors

An optimized deep learning approach for forecasting day-ahead electricity prices

Electricity price forecasting is essential for reliable and cost-effective operations in the power industry. However, the complex and nonlinear structure of the electricity price series presents uncertainties and challenges for energy management. To address this, artificial intelligence models such as SARIMAX, LSTM, and CNN-LSTM have been developed to predict short-term electricity prices. These models were tested using Mean Absolute Error, Root-Mean Squared Error, Mean Absolute Percentage Error, and percentage accuracy to verify their accuracy and to compare forecasting methodologies. The study includes the Diebold–Mariano test to confirm the statistical significance of the difference between the forecast errors of the two models. Ensemble learning was used to optimize a CNN-LSTM model, which automatically selects the best model by using CNN to extract valuable characteristics and LSTM to recognize data dependency in time series. Historical data from the German electrical market were used to validate the models’ prediction performance. The results showed that the LSTM and CNN-LSTM models outperformed the SARIMAX model in terms of accuracy and simplicity, with the CNN-LSTM technique having significant forecasting advantages. These methods can be used for intelligent optimization forecasting of electricity prices.

Machine and deep learning approaches for forecasting electricity price and energy load assessment on real datasets

This article aims to propose short-term forecasting for a combined model of the hourly electricity price and the hourly load. Where both the independent system operators and the market participants must deal with the uncertainty and volatility of the day-ahead electricity price. Therefore, an accurate prediction model has been needed for a combined short-term forecasting model for both the hourly electricity price and the hourly load in the deregulated market. To get a better prediction, real-time data for the day-ahead electricity price and the hourly load are provided from the ISO New England Control Area (ISO-NE-CA) market. This study presents the combined model based on four featured machine and deep learning algorithms: Feed Forward Artificial Neural Network, Adaptive Neuro-Fuzzy Inference System, Long Short-Term Memory, and Gated Recurrent Units. The results are obtained from two scenarios, In the first scenario, the data has been trained and tested without depending on other factors such as temperatures and calendar. In the second scenario, the data has been trained and tested depending on the previously mentioned factors. The obtained results from the four algorithms have been compared to show their performance and their efficiency in the values of Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Normalized Root Mean Squared Error (NRMSE), and Mean Absolute Percentage Error (MAPE). The four featured algorithms prove their good performance with less error in the forecasting model.

Intelligent Building Energy Management Considering Uncertainties and Variation of Day-Ahead and Real-Time Electricity Prices

Intelligent Building Energy Management Considering Uncertainties and Variation of Day-Ahead and Real-Time Electricity Prices

Interpretable Interval Prediction-Based Outlier-Adaptive Day-Ahead Electricity Price Forecasting Involving Cross-Market Features

Interpretable Interval Prediction-Based Outlier-Adaptive Day-Ahead Electricity Price Forecasting Involving Cross-Market Features

Day-Ahead Electricity Price Forecasting in the Contemporary Italian Market

Day-Ahead Electricity Price Forecasting in the Contemporary Italian Market

Toward high-resolution projection of electricity prices: A machine learning approach to quantifying the effects of high fuel and CO2 prices

The growing share of renewables, the retirement of coal generation, and the increasing significance (and price) of carbon emissions continue to reshape electricity market dynamics. Ongoing power sector transformations, along with recurrent energy price shocks, bring new challenges to electricity price prediction and projection. Adept at capturing nonlinear and time-varying dynamics of the power market, machine learning (ML) methods can significantly enhance modeling and projection capabilities. This paper leverages ML modeling to (1) analyze the day-ahead electricity price dynamics and (2) develop future price projections under varying market conditions. First, we offer a methodology for identifying and ranking the factors determining day-ahead power price behavior. Guided by the understanding of power price drivers, we then develop an ensemble ML model for high-resolution (i.e., for each day of the next year) electricity price projection. Hence, the first part of our analysis helps relate fundamental modeling approaches to ML studies, and the second part extends the existing literature, which is limited to high aggregation level (e.g., yearly basis) projections. High-resolution projections provide valuable insights into within-year price dynamics and help answer highly debated questions raised by the growing share of renewables and the current energy market turmoil. We demonstrate our methodology on the German power market, using data on 80 explanatory variables from 2015 to 2021. We identify the most critical power price drivers and develop an ML model to study the within-year impact of high fuel and carbon prices. Our findings indicate that the often-neglected interaction variables may significantly impact price projections and should be included in day-ahead electricity price modeling. We confirm the efficacy of our approach with a forecasting exercise based on day-ahead electricity prices in the first two quarters of 2022.

Forecasting day-ahead electricity prices with spatial dependence

Market integration connects multiple autarkic electricity markets and facilitates the flow of power across areas. More often than not, market integration can increase social welfare for the whole power system with a clear spatial dependence structure among area electricity prices, which motivates a new perspective on electricity price forecasting. In this paper, we construct a model to forecast the day-ahead electricity prices of Nord Pool with spatial dependence. First of all, we convert the electricity prices into graph data. Then, we propose an STGNN (Spatial-Temporal Graph Neural Network) model to exploit spatial and temporal features. In particular, the STGNN model can accurately forecast electricity prices for multiple areas, where the adjacency matrix representing the spatial dependence structure is pre-captured by the R-vine (regular vine) copula. Our results show that the spatial dependence structure described by the R-vine copula can perfectly reflect the physical characteristics of the electricity system; moreover, the forecasting performance of the proposed STGNN model is significantly better than the existing models in terms of overall accuracy and hourly accuracy within a day.

A smart predict-and-optimize framework for microgrid's bidding strategy in a day-ahead electricity market

This paper deals with the microgrid's bidding strategy (MGBS) problem in a day-ahead (DA) electricity market. To this end, the DA electricity prices, demand, and renewable energy uncertainties are modeled by different scenarios via generating several representative daily curves. However, the real-time (RT) electricity prices are modeled using robust optimization (RO) via predicting appropriate intervals. The existing paradigm to predict the RT intervals is to minimize the statistical errors, while this accuracy-oriented method may not necessarily yield the best MGBS plan against the actual realization of RT prices. The novelty of this research is to present a smart predict-and-optimize (SPO) methodology in which a cost-oriented prediction model replaces the existing accuracy-oriented methods. Trilevel mathematical programming is established to construct a cost-oriented model where the first level maximizes/minimizes the actual profit/cost by adjusting the predicted interval bounds of the RT prices. Knowing these trained interval bounds, the second level runs a scenario-based DA-MGBS. Then, a rescheduling problem is solved at the third level based on the actual realization of uncertainties. To solve this trilevel problem, a reformulation-and-decomposition (R&D) algorithm is utilized. The numerical experiment illustrates the advantages of the SPO method in terms of the microgrid (MG) cost.

A TWO STAGE MODEL FOR DAY-AHEAD ELECTRICITY PRICE FORECASTING: INTEGRATING EMPIRICAL MODE DECOMPOSITION AND CATBOOST ALGORITHM

Electricity price forecasting is crucial for the secure and cost-effective operation of electrical power systems. However, the uncertain and volatile nature of electricity prices makes the electricity price forecasting process more challenging. In this study, a two-stage forecasting model was proposed in order to accurately predict day-ahead electricity prices. Historical natural gas prices, electricity load forecasts, and historical electricity price values were used as the forecasting model inputs. The historical electricity and natural gas price data were decomposed in the first stage to extract more deep features. The empirical mode decomposition (EMD) algorithm was employed for the efficient decomposition process. In the second stage, the categorical boosting (CatBoost) algorithm was proposed to forecast day-ahead electricity prices accurately. To validate the effectiveness of the proposed forecasting model, a case study was conducted using the dataset from the Turkish electricity market. The proposed model results were compared with benchmark machine learning algorithms. The results of this study indicated that the proposed model outperformed the benchmark models with the lowest root mean squared error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and correlation coefficient (R) values of 8.3282%, 5.2210%, 6.9675%, and 86.2256%, respectively.

Forecasting Day-Ahead Electricity Prices for the Italian Electricity Market Using a New Decomposition—Combination Technique

Over the last 30 years, day-ahead electricity price forecasts have been critical to public and private decision-making. This importance has increased since the global wave of deregulation and liberalization in the energy sector at the end of the 1990s. Given these facts, this work presents a new decomposition–combination technique that employs several nonparametric regression methods and various time-series models to enhance the accuracy and efficiency of day-ahead electricity price forecasting. For this purpose, first, the time-series of the original electricity prices deals with the treatment of extreme values. Second, the filtered series of the electricity prices is decomposed into three new subseries, namely the long-term trend, a seasonal series, and a residual series, using two new proposed decomposition methods. Third, we forecast each subseries using different univariate and multivariate time-series models and all possible combinations. Finally, the individual forecasting models are combined directly to obtain the final one-day-ahead price forecast. The proposed decomposition–combination forecasting technique is applied to hourly spot electricity prices from the Italian electricity-market data from 1 January 2014 to 31 December 2019. Hence, four different accuracy mean errors—mean absolute error, mean squared absolute percent error, root mean squared error, and mean absolute percent error; a statistical test, the Diebold–Marino test; and graphical analysis—are determined to check the performance of the proposed decomposition–combination forecasting method. The experimental findings (mean errors, statistical test, and graphical analysis) show that the proposed forecasting method is effective and accurate in day-ahead electricity price forecasting. Additionally, our forecasting outcomes are comparable to those described in the literature and are regarded as standard benchmark models. Finally, the authors recommended that the proposed decomposition–combination forecasting technique in this research work be applied to other complicated energy market forecasting challenges.

Dense Skip Attention Based Deep Learning for Day-Ahead Electricity Price Forecasting

The forecasting of the day-ahead electricity price (DAEP) has become more of interest to decision makers in the liberalized market, as it can help optimize bidding strategies and maximize profits with the gradual market expansion. Deep learning (DL) is a promising method for its strong nonlinear approximation capabilities. However, it is challenging for traditional DL models to obtain a high forecasting precision for the DAEP, due to its internal temporal and feature-wise variabilities. To address the issue, this paper proposes a dense skip attention based DL model. In this model, to tackle the feature-wise variability, a mechanism of dense skip attention is proposed to efficiently assign learnable weights on the features for training. In terms of the temporal variability, a drop-connected structure based on the advanced residual unshared convolutional neural network (ARUCNN) and gate recurrent units (GRUs) is further proposed. In this structure, the ARUCNN is developed by embedding advanced activations to deal with the short-term dependencies and degradation while GRUs addressing the long-term ones, and they are integrated via a drop connection to reduce the overfitting. Through validating on real DAEP data in day-ahead markets of Sweden, Denmark, Norway and Finland, the results verify our proposed approach outperforms the existing methods in the deterministic and interval forecasting of DAEP.