Day-ahead Hourly Research Articles

DWT-BILSTM-based models for day-ahead hourly global horizontal solar irradiance forecasting

Accurate forecasting of electricity generation from renewable energy sources is crucial for the operation, planning and management of smart grids. For reliable planning and operation of photovoltaic (PV) systems in grid-connected or islanded utilities, an hourly day-ahead forecast of PV output is critical. The forecast of PV power can be done indirectly by estimating solar irradiance. For forecasting day-ahead hourly global horizontal irradiance (GHI), two forecasting models with different multivariate inputs are proposed in this paper, and the results are compared. These models use a hybrid algorithm of discrete wavelet decomposition and bidirectional long short-term memory (BILSTM). The inputs of the first model contain GHI and weather type data. The other model allows for observation of the effect of meteorological values including GHI, temperature, humidity, wind speed, and weather type data. The forecasting performance of deep learning algorithms which contain recurrent neural network (RNN), long short-term memory (LSTM), and BILSTM algorithms for day ahead hourly solar irradiance forecasting problems are also compared. To evaluate the performance of proposed models, two datasets are used for Model 1 and one dataset is used for Model 2. An experiment is also done to demonstrate that the proposed Model 1 is applicable in datasets collected in the vicinity of the city of Trabzon. On the other hand, BILSTM algorithm outperforms RNN and LSTM algorithms. It is seen that the test successes of both proposed models are better than the results given in the literature.

Total and thermal load forecasting in residential communities through probabilistic methods and causal machine learning

Indoor heating and cooling systems largely influence the power demand of residential buildings and can play a significant role in the Demand Side Management for energy communities. We propose a novel method for probabilistic forecasting of the total load of a residential community and its base and thermal components, combining conformalized quantile regression and causal machine learning techniques, using only aggregate consumption and environmental conditions data. We applied the proposed methods to the dataset of a residential community in Germany, which includes separate measurements of the total electricity demand and of domestic heating system consumption. The results show that the proposed method produces probabilistic day-ahead hourly forecasts of total energy demand that outperform benchmarks and forecasts of electricity consumption components that are not only more accurate than benchmarks but also close to the accuracy achievable with models trained directly on individual load component data. The T-learner method resulted the most effective among the causal methods for load disaggregation in terms of accuracy, simplicity, and potential for extension.

Influencing residential electricity consumption with tailored messages: long-term usage patterns and effects on user experience

BackgroundTo transition our energy system toward sustainable production and consumption, it is important to successfully engage consumers to become active participants in this process. One form this can take is manual demand response, where end users respond to fluctuations in energy production and help balance the grid through adjustment of their consumption. This paper presents a trial of such a system that took place with tenants in subsidized housing in Catalonia, Spain. The aim of the trial was to motivate the load shifting behavior of the participants by forecasting expected consumption curves and tailoring suggestions for optimized behavior. The forecasts and suggestions were based on the users’ past consumption patterns and the hourly day-ahead electricity prices. This information was made available to the users on a web-based platform, and participants were actively informed with text messages sent to their mobile phones in case of attractive saving potentials for the following day. The trial was carried out in 2 phases from November 2019 to May 2020 (Phase 1) and from August to October 2020 (Phase 2). Data were collected on interaction with the platform, the perceived user experience of the platform and text messages, and the perceived energy saving success.ResultsOur results showed that there is a general interest of the participants in the concept, but that there are also important barriers to integrating load shifting behavior into everyday life. The biggest barriers here are limitations in the flexibility potential of households and limited perceived benefits. Feedback from our participants also suggests high acceptance and relevance of more automated demand-side management (DSM) concepts.ConclusionsBased on this, we recommend paying special attention to the accommodation of varying flexibility potential in manual demand response (DR) programs, ensuring that communicated benefits are sufficiently attractive to motivate behavior change, and consideration of a phase of manual DR as an entry point to automated DSM.

Two-Stage Participation of Grid-Connected Energy Hubs in Day-Ahead and Real-Time Energy Markets Based on Robust Modeling of Uncertainties

Optimal management of sources, storage, and responsive loads in the energy network leads to improvement of economic and technical situation of the network. However, note that single-layer energy management requires to provide the network operator with more amounts of information. This results in the increase in the processing speed for the network operator. Also, coordination of sources, storage, and responsive loads in the form of an aggregator such as an energy hub can play an important role for them in promoting the economic situation and energy efficiency. So, this paper presents the participation of grid-connected energy hubs (EHs) in the day-ahead (DA) and real-time (RT) energy markets. Scheme has a two-stage optimization framework. In the first stage, participation of EHs in the hourly DA energy market is executed based on a two-layer energy management system that is modeled as bilevel problem. EH model in DA market is formulated in upper-level problem aimed at maximizing their profit subject to meeting the linearized constraints of the EH’s operation. Lower-level problem deals with the operation of energy networks to minimize cost of energy purchase from the upstream network constrained to linearized optimal power flow equations. The second stage of participation of EHs in a 5 min RT market has an objective function to minimize the unbalance between the EHs’ profits in the mentioned two markets. This stage is subject to the upper-level constraints and the lower-level formulation in the first stage. Robust optimization models the uncertainties of load, renewable power, energy price, and energy demand of mobile storage to achieve an optimal robust solution. Benders decomposition algorithm solves bilevel formulation in two-stage scheme. Numerical results confirm capabilities of the scheme in enhancing operation status of the networks and economic situation of EHs. Hence, the Benders decomposition algorithm is able to extract the optimal solution in shorter computation time. Two-layer energy management needs shorter computation time than the single-layer management. Simultaneous modeling of two real-time and day-ahead energy markets reduces the unbalance cost between these markets. The proposed scheme can enhance the operating and economic indicators of the network by up to around 18% and 26% compared to power flow studies of energy networks.

Short-term probabilistic forecasting models using Beta distributions for photovoltaic plants

This article presents original probabilistic forecasting models for day-ahead hourly energy generation forecasts for a photovoltaic (PV) plant, based on a semi-parametric approach using three deterministic forecasts. Input information of these new models consists of data of hourly weather forecasts obtained from a Numerical Weather Prediction model and variables related to the sun position for future instants. The proposed models were satisfactorily applied to the case study of a real-life PV plant in Portugal. Probabilistic benchmark models were also applied to the same case study and their forecasting results compared with the ones of the proposed models. The computer results obtained with these proposed models achieve better point and probabilistic forecasting evaluation indexes values than the ones obtained with the benchmark models.

From DevOps to MLOps: Overview and Application to Electricity Market Forecasting

In the Software Development Life Cycle (SDLC), Development and Operations (DevOps) has been proven to deliver reliable, scalable software within a shorter time. Due to the explosion of Machine Learning (ML) applications, the term Machine Learning Operations (MLOps) has gained significant interest among ML practitioners. This paper explains the DevOps and MLOps processes relevant to the implementation of MLOps. The contribution of this paper towards the MLOps framework is threefold: First, we review the state of the art in MLOps by analyzing the related work in MLOps. Second, we present an overview of the leading DevOps principles relevant to MLOps. Third, we derive an MLOps framework from the MLOps theory and apply it to a time-series forecasting application in the hourly day-ahead electricity market. The paper concludes with how MLOps could be generalized and applied to two more use cases with minor changes.

Demand response modeling in a day-ahead wholesale electricity market in Japan, considering the impact of customer risk aversion and dynamic price elasticity of demand

This study aims to develop a mathematical modeling approach to maximize the welfare of the price-responsive customers (CUs) in a wholesale electricity market in Tokyo, Japan. The contributions made by this paper in the quest to determine the role of CUs in a demand response market are twofold. First, the aversion of the CUs to the risk of choosing the Demand Response Programs (DPRs) is taken into account by considering their expected utility from consuming electricity. The proposed model is founded on the customer theory in microeconomics, using the concept of the expected utility function to model the behavior of the risk-averse CUs in response to different DPRs. Second, it introduces an hourly-based model for the short-term price elasticity of demand, considering the day-ahead price mechanism defined in the Japan Electric Power Exchange (JEPX) market. The estimated price elasticities are used in a price elasticity matrix of demand (PEMD) to precisely reflect the different response strategies, including flexible, in-flexible forward-shifting, backward shifting, and optimizing responses. An accurate day-ahead hourly load forecasting is performed, using the Seasonal Autoregressive Integrated Moving Average (SARIMA) model, which is trained on four years of data provided by the Tokyo Electric Power Company (TEPCO), with a mean absolute percentage error (MAPE) of 0.94%. The developed model is used to analyze the CUs’ behaviors with different response strategies in the JEPX market in Tokyo. By applying the Time-of-Use (TOU) and Real-Time-Pricing (RTP) programs, the results reveal a peak reduction potential of 10.7% and 7.3%, respectively, for the flexible CUs. Applying the RTP program to the curtailable loads can achieve a 7.7% reduction in daily peak demand and a 1.6% reduction in daily electricity consumption.

A Comparative Study of Time Series Forecasting of Solar Energy Based on Irradiance Classification

Sustainable energy systems rely on energy yield from renewable resources such as solar radiation and wind, which are typically not on-demand and need to be stored or immediately consumed. Solar irradiance is a highly stochastic phenomenon depending on fluctuating atmospheric conditions, in particular clouds and aerosols. The complexity of weather conditions in terms of many variable parameters and their inherent unpredictability limit the performance and accuracy of solar power forecasting models. As renewable power penetration in electricity grids increases due to the rapid increase in the installation of photovoltaics (PV) systems, the resulting challenges are amplified. A regional PV power prediction system is presented and evaluated by providing forecasts up to 72 h ahead with an hourly time resolution. The proposed approach is based on a local radiation forecast model developed by Blue Sky. In this paper, we propose a novel method of deriving forecast equations by using an irradiance classification approach to cluster the dataset. A separate equation is derived using the GEKKO optimization tool, and an algorithm is assigned for each cluster. Several other linear regressions, time series and machine learning (ML) models are applied and compared. A feature selection process is used to select the most important weather parameters for solar power generation. Finally, considering the prediction errors in each cluster, a weighted average and an average ensemble model are also developed. The focus of this paper is the comparison of the capability and performance of statistical and ML methods for producing a reliable hourly day-ahead forecast of PV power by applying different skill scores. The proposed models are evaluated, results are compared for different models and the probabilistic time series forecast is presented. Results show that the irradiance classification approach reduces the forecasting error by a considerable margin, and the proposed GEKKO optimized model outperforms other machine learning and ensemble models. These findings also emphasize the potential of ML-based methods, which perform better in low-power and high-cloud conditions, as well as the need to build an ensemble or hybrid model based on different ML algorithms to achieve improved projections.

Optimal bidding in hourly and quarter-hourly electricity price auctions: Trading large volumes of power with market impact and transaction costs

This paper addresses the question of how much to bid to maximize the profit when trading in two electricity markets: the hourly Day-Ahead Auction and the quarter-hourly Intraday Auction. For optimal coordinated bidding many price scenarios are examined, the own non-linear market impact is estimated by considering empirical supply and demand curves, and a number of trading strategies is used. Additionally, we provide theoretical results for risk neutral agents. The application study is conducted using the German market data, but the presented methods can be easily utilized with other two consecutive auctions. This paper contributes to the existing literature by evaluating the costs of electricity trading, i.e. the price impact and the transaction costs. The empirical results for the German EPEX market show that it is far more profitable to minimize the price impact rather than maximize the arbitrage.

Forecasting electricity prices with expert, linear, and nonlinear models

This paper compares several models for forecasting regional hourly day-ahead electricity prices, while accounting for fundamental drivers. Forecasts of demand, in-feed from renewable energy sources, fossil fuel prices, and physical flows are all included in linear and nonlinear specifications, ranging in the class of ARFIMA-GARCH models—hence including parsimonious autoregressive specifications (known as expert-type models). The results support the adoption of a simple structure that is able to adapt to market conditions. Indeed, we include forecasted demand, wind and solar power, actual generation from hydro, biomass, and waste, weighted imports, and traditional fossil fuels. The inclusion of these exogenous regressors, in both the conditional mean and variance equations, outperforms in point and, especially, in density forecasting when the superior set of models is considered. Indeed, using the model confidence set and considering northern Italian prices, predictions indicate the strong predictive power of regressors, in particular in an expert model augmented for GARCH-type time-varying volatility. Finally, we find that using professional and more timely predictions of consumption and renewable energy sources improves the forecast accuracy of electricity prices more than using predictions publicly available to researchers.

Complexity and Persistence of Price Time Series of the European Electricity Spot Market

The large variability of renewable power sources is a central challenge in the transition to a sustainable energy system. Electricity markets are central for the coordination of electric power generation. These markets rely evermore on short-term trading to facilitate the balancing of power generation and demand and to enable systems integration of small producers. Electricity prices in these spot markets show pronounced fluctuations, featuring extreme peaks as well as occasional negative prices. In this article, we analyze electricity price time series from the European Power Exchange market, in particular the hourly day-ahead, hourly intraday, and 15-min intraday market prices. We quantify the fluctuations, correlations, and extreme events and reveal different time scales in the dynamics of the market. The short-term fluctuations show remarkably different characteristics for time scales below and above 12 h. Fluctuations are strongly correlated and persistent below 12 h, which contributes to extreme price events and a strong multifractal behavior. On longer time scales, they get anticorrelated and price time series revert to their mean, witnessed by a stark decrease of the Hurst coefficient after 12 h. The long-term behavior is strongly influenced by the evolution of a large-scale weather pattern with a typical time scale of four days. We elucidate this dependence in detail using a classification into circulation weather types. The separation in time scales enables a superstatistical treatment, which confirms the characteristic time scale of four days, and motivates the use of q-Gaussian distributions as the best fit to the empiric distribution of electricity prices.3 MoreReceived 6 December 2021Revised 21 February 2022Accepted 28 February 2022DOI:https://doi.org/10.1103/PRXEnergy.1.013002Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasComplex systemsEconophysicsEnergy sourcesFractional Brownian motionPatterns in complex systemsSolar energyStochastic processesWind energyPhysical SystemsFractalsMultiple time scale dynamicsTechniquesNon-Markovian processesTime series analysisStatistical PhysicsInterdisciplinary PhysicsGeneral Physics

Engaging domestic users on demand response for heating cost reduction with a recommendation tool: Case study in Belgrade

The European Union has established a legislative framework that aims to enable consumers and businesses to take information-based decisions to save energy and money. Additionally, the increase of Distributed Energy Resources (both on generation and consumption) requires additional efforts to maintain the reliability and stability of the electric grid and the need of flexibility from residential buildings. The present study introduces a domestic decision support tool for reducing heating costs. This app provides detailed recommendations to end-users based on the day-ahead hourly weather forecast, electric and district heating tariffs predictions, heating demand, and heating systems dynamic performance. The tool was tested in 6 dwellings of a neighborhood of Belgrade during the last months of 2021 heating season (March–May). Energetic results suggest that 40% of participants followed the given recommendations and changed their heating pattern. Additionally, survey results show that end-users found the lack of information and knowledge as the main barrier to actively participate in the energy market, also preferring to have automatic control in their heating system. Authors conclude that recommendation tools are key elements in user-engagement, but they should be supported by additional information and training.

Impacts of data preprocessing and selection on energy consumption prediction model of HVAC systems based on deep learning

Accurate energy consumption prediction is the basis of predictive control for heating, ventilation and air conditioning (HVAC) systems. Data-driven models are widely used for energy consumption prediction. The prediction accuracy can be affected by data preprocessing or selection, which are not well studied yet. This paper attempts to study the impacts of different data processing methods and feature selection on the HVAC energy consumption prediction based on data-driven models. Long and short-term memory models are developed based on historical data to predict the day-ahead hourly energy consumption of HVAC systems. Two data smoothing methods, Gaussian kernel density estimation and Savitzky-Golay filter, are selected and compared. The impacts of feature selection, training set volumes and update frequency of models are analyzed and compared. To obtain general results of the above problems, three office buildings are selected. Results show that the smoothing methods can not ensure the improvement in accuracy by removing the exceptional data in the raw data which arise reasonably in practice. The inputs with raw 1-day historical energy consumption, the dry-bulb temperature and dew-point temperature in the next day, holiday type and day type is recommended to predict day-ahead HVAC energy consumption. It is recommended to use a larger training set if computing cost is acceptable. The model should be updated after being used for over 7 weeks. This study would guide the development of prediction models in practice.

Analysis of data errors in the solar photovoltaic monitoring system database: An overview of nationwide power plants in Korea

Management services for solar photovoltaic power plants (PVs) have become an important issue in today's era of increasing PV installations. The existence of error-free data is necessary for data-driven PV management; however, real-world data contain an unignorable portion of errors, thereby reducing the reliability of corresponding analyses. A report on the plausible errors within the data is necessary to help PV administrators configure proper data cleaning algorithms. This study provides an error case report derived from an analysis of nearly 2000 PVs distributed throughout Korea. Real error cases are categorized according to their causes and symptoms. The effect of typical errors on the data-driven PV analysis is particularly evaluated for the day-ahead hourly PV power forecast. Errors in the system specification data significantly decrease the forecast accuracy, thereby addressing the impact of human errors. Regression imputation in the time domain demonstrates acceptable results as a simple ad-hoc method in most error situations.

Day-Ahead Residential Electricity Demand Response Model Based on Deep Neural Networks for Peak Demand Reduction in the Jordanian Power Sector

In this paper, a comprehensive demand response model for the residential sector in the Jordanian electricity market is introduced, considering the interaction between the power generators (PGs), grid operators (GOs), and service providers (SPs). An accurate day-ahead hourly short-term load forecasting is conducted, using deep neural networks (DNNs) trained on four-year data collected from the National Electric Power Company (NEPCO) in Jordan. The customer behavior is modeled by developing a precise price elasticity matrix of demand (PEMD) based on recent research on the short-term price elasticity of Jordan’s residential and the analysis of the different types of electrical appliances and their daily operational hours according to the latest surveys. First, the DNNs are fine-tuned with a detailed feature analysis to predict the day-ahead hourly electrical demand and achieved a mean absolute percentage error (MAPE) of 1.365% and 1.411% on the validation and test datasets receptively. Then the predictions are used as input to a detailed model of the Jordanian power grid market, where a day-ahead peak-time demand response policy for the residential sector is applied to the three distribution power companies in Jordan. Based on different PEMD analyses for the Jordanian residential sector, the results suggest a reduction potential of 5.4% in peak demand accompanied by a cost reduction of USD 154,505 per day for the Jordanian power sector.

Short-Term Net Load Forecasting with Singular Spectrum Analysis and LSTM Neural Networks

Short-term electricity load forecasting is key to the safe, reliable, and economical operation of power systems. An important challenge that arises with high-frequency load series, e.g., hourly load, is how to deal with the complex seasonal patterns that are present. Standard approaches suggest either removing seasonality prior to modeling or applying time series decomposition. This work proposes a hybrid approach that combines Singular Spectrum Analysis (SSA)-based decomposition and Artificial Neural Networks (ANNs) for day-ahead hourly load forecasting. First, the trajectory matrix of the time series is constructed and decomposed into trend, oscillating, and noise components. Next, the extracted components are employed as exogenous regressors in a global forecasting model, comprising either a Multilayer Perceptron (MLP) or a Long Short-Term Memory (LSTM) predictive layer. The model is further extended to include exogenous features, e.g., weather forecasts, transformed via parallel dense layers. The predictive performance is evaluated on two real-world datasets, controlling for the effect of exogenous features on predictive accuracy. The results showcase that the decomposition step improves the relative performance for ANN models, with the combination of LSTM and SAA providing the best overall performance.

Flexible Continuous-Time Modeling for Multi-Objective Day-Ahead Scheduling of CHP Units

Increasing applications of CHP units have turned the problem of finding the best optimization model into a significant subject for scholars. In this respect, this paper is aimed at driving a novel formulation to the multi-objective day-ahead scheduling of CHP units using Bernstein polynomials, which more optimally schedules power and heat generations as well as ramping trajectories. This procedure includes yielding an affine function that closely approximates real-time net-load and generation trajectories, which is demonstrated to have a superior performance to the conventional hourly day-ahead scheduling of CHP units based on discrete-time approximation. The problem of how to handle various objective functions by function space method is also addressed. The simulations conducted on the sample test systems, which consist of CHP systems, thermal and heat-only units, as well as thermal and electrical loads, show that the suggested multi-objective model can perfectly cover the total heat and electrical loads in terms of economic and environmental criteria. More importantly, the results indicate that the accuracy of the proposed approach renders cost saving of 1.67% and emission saving of 1.46% in comparison with the conventional hourly-based model, apart from leading to fewer ramping scarcities in real-time operations.

Day-Ahead Hourly Operation Planning of Distribution Networks with Photovoltaic Generation Integrated by Smart Inverters via Mixed-Integer Volt/Var Optimization Problems.

A penetração da geração fotovoltaica distribuída nas redes de distribuição tornou-se uma realidade; entretanto, ainda impõe desafios a serem superados, como o funcionamento dessas redes com desvios mínimos nos perfis de magnitude de tensão. Assim, este trabalho propõe o planejamento diário da operação horária de redes de distribuição visando minimizar o desvio nos perfis de magnitude de tensão por meio de ajustes de tap ótimos de transformadores em subestações, ajustes ótimos de bancos de capacitores chaveados remotamente ao longo dos alimentadores primários e injeções de potência ativa e reativa da geração fotovoltaica distribuída por inversores inteligentes. Em tal problema, as variáveis de controle discretas associadas aos taps de transformadores e susceptâncias equivalentes de bancos de capacitores chaveados remotamente são tratadas por função penalidade senoidal que modificam o problema de programação não linear inteira mista original em um problema de programação não linear com apenas variáveis de decisão contínuas. Os resultados obtidos para redes de distribuição com 69 e 135 nós mostram que o modelo OVV proposto e sua metodologia de resolução são eficientes na redução do desvio nos perfis de magnitude de tensão e, ao mesmo tempo, na redução da demanda geral de potência reativa da rede de distribuição.

Accurate Prediction of Hourly Energy Consumption in a Residential Building Based on the Occupancy Rate Using Machine Learning Approaches

In this paper, a novel deep neural network-based energy prediction algorithm for accurately forecasting the day-ahead hourly energy consumption profile of a residential building considering occupancy rate is proposed. Accurate estimation of residential load profiles helps energy providers and utility companies develop an optimal generation schedule to address the demand. Initially, a comprehensive multi-criteria analysis of different machine learning approaches used in energy consumption predictions was carried out. Later, a predictive micro-grid model was formulated to synthetically generate the stochastic load profiles considering occupancy rate as the critical input. Finally, the synthetically generated data were used to train the proposed eight-layer deep neural network-based model and evaluated using root mean square error and coefficient of determination as metrics. Observations from the results indicated that the proposed energy prediction algorithm yielded a coefficient of determination of 97.5% and a significantly low root mean square error of 111 Watts, thereby outperforming the other baseline approaches, such as extreme gradient boost, multiple linear regression, and simple/shallow artificial neural network.

Deep Learning-Based Short-Term Load Forecasting Approach in Smart Grid With Clustering and Consumption Pattern Recognition

Different aggregation levels of the electric grid’s big data can be helpful to develop highly accurate deep learning models for Short-term Load Forecasting (STLF) in electrical networks. Whilst different models are proposed for STLF, they are based on small historical datasets and are not scalable to process large amounts of big data as energy consumption data grow exponentially in large electric distribution networks. This paper proposes a novel hybrid clustering-based deep learning approach for STLF at the distribution transformers’ level with enhanced scalability. It investigates the gain in training time and the performance in terms of accuracy when clustering-based deep learning modeling is employed for STLF. A k-Medoid based algorithm is employed for clustering whereas the forecasting models are generated for different clusters of load profiles. The clustering of the distribution transformers is based on the similarity in energy consumption profile. This approach reduces the training time since it minimizes the number of models required for many distribution transformers. The developed deep neural network consists of six layers and employs Adam optimization using the TensorFlow framework. The STLF is a day-ahead hourly horizon forecasting. The accuracy of the proposed modeling is tested on a 1,000-transformer substation subset of the Spanish distribution electrical network data containing more than 24 million load records. The results reveal that the proposed model has superior performance when compared to the state-of-the-art STLF methodologies. The proposed approach delivers an improvement of around 44% in training time while maintaining accuracy using single-core processing as compared to non-clustering models.