ISO New Research Articles

Machine and deep learning approaches for forecasting electricity price and energy load assessment on real datasets

This article aims to propose short-term forecasting for a combined model of the hourly electricity price and the hourly load. Where both the independent system operators and the market participants must deal with the uncertainty and volatility of the day-ahead electricity price. Therefore, an accurate prediction model has been needed for a combined short-term forecasting model for both the hourly electricity price and the hourly load in the deregulated market. To get a better prediction, real-time data for the day-ahead electricity price and the hourly load are provided from the ISO New England Control Area (ISO-NE-CA) market. This study presents the combined model based on four featured machine and deep learning algorithms: Feed Forward Artificial Neural Network, Adaptive Neuro-Fuzzy Inference System, Long Short-Term Memory, and Gated Recurrent Units. The results are obtained from two scenarios, In the first scenario, the data has been trained and tested without depending on other factors such as temperatures and calendar. In the second scenario, the data has been trained and tested depending on the previously mentioned factors. The obtained results from the four algorithms have been compared to show their performance and their efficiency in the values of Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Normalized Root Mean Squared Error (NRMSE), and Mean Absolute Percentage Error (MAPE). The four featured algorithms prove their good performance with less error in the forecasting model.

A Sparse Model for Electricity Price Forecasting with LASSO-GRBFN and Brain Storm Optimization

This paper proposes a new method for predicting electricity prices. The method is based on the Generalized Radial Basis Function Network (GRBFN), an extension of the RBFN of Artificial Neural Network (ANN) used for one-step-ahead forecasting. The GRBFN is superior to the RBFN because it evaluates the Gaussian function parameters during learning. Conventional ANN methods consider overfitting with the weight decay method, based on the L2 norm of weights between neurons, but there is still room for improvement. Therefore, the paper proposes using the Least Absolute Shrinkage and Selection Operator (LASSO) to improve the model’s performance. Furthermore, the paper presents the Brain Storm Optimization (BSO) of evolutionary computation to evaluate the cost function with the term of the L1 norm. Conventional methods with the gradient are not suitable for the L1 norm. The proposed method is successfully applied to actual data of ISO New England in the USA.

Effect of Real-Time Price on Short-Term Load Forecasting

This paper presents a short-term load forecasting model using the back-propagation neural network (BPNN) model. The proposed model is based on data on loads and factors that directly affect electricity demand, such as temperature, humidity, load over time in the past, etc., collected from the electricity market ISO New England. In addition to the common factors, the article also considers a new factor: real-time price. The data used for training and forecasting are real-time data for three years from 2019 to 2021. The paper has shown that real-time price (RTP) significantly influences forecasting. The proof is that the Mean Absolute Percentage Error (MAPE) value of the predictive model without RTP data is 2.08%, and that of the model with RTP data is 1.44%. The paper also compares the performance of the training algorithms with each other to come up with an optimal algorithm compared to the proposed model. At the same time, the model is also applied to forecast a more extensive period, such as a week or a month, and has had positive results.

Nodal Electricity Price Forecasting using Exponential Smoothing and Holt’s Exponential Smoothing

The prediction of nodal electricity price (NEP) is a primary step to be done before the bidding process starts in the actual market environment. NEP plays a significant role for the efficient working of the electrical system. NEP follows a common trend as during peak hours when the load is high the price will also be high similarly during off-peak-load times the price will be lower and common to all the node. Thus, accurate forecasting of the NEP can help electricity generation companies to be more proactive in the wholesale electricity market to maximize its overall benefits. In this paper, exponential smoothing (ES), and holt’s exponential smoothing (HES) have been utilized for forecasting the NEP. Furthermore, a comparative analysis between ES and HES has been done considering several alpha values and several trends. The model evaluation and the forecasting performance have been tested using different parameters of ES, and HES techniques such as Akaike Information Criterion (AIC), Akaike Information Criterion Corrected (AICc), Bayesian Information Criteria (BIC). The performance of the proposed technique has been authenticated efficaciously on average nodal real-time price data collected from ISO New England (BOSTON Zone).

An Online Data-Driven Method to Locate Forced Oscillation Sources From Power Plants Based on Sparse Identification of Nonlinear Dynamics (SINDy)

Forced oscillations may jeopardize the secure operation of power systems. To mitigate forced oscillations, locating the sources is critical. In this paper, leveraging on Sparse Identification of Nonlinear Dynamics (SINDy), an online purely data-driven method to locate the forced oscillation is developed. Validations in all simulated cases (in the WECC 179-bus system) and actual oscillation events (in ISO New England system) in the IEEE Task Force test cases library are carried out, which demonstrate that the proposed algorithm, requiring no model information, can accurately locate sources in most cases, even under resonance condition and when the natural modes are poorly damped. The little tuning requirement and low computational cost make the proposed method viable for online application.

Short-term load and price forecasting using artificial neural network with enhanced Markov chain for ISO New England

Nowadays, forecasting methods have gained significant attention, particularly with the design and development of energy systems. In fact, accurate load and price forecasting is crucial for effective planning, controlling, and operation of power systems, especially with renewable energy sources (RES). This paper implemented an improved Markov Chain Artificial Neural network (ANN-MC) for load forecasting. The proposed design involved a two-step implementation process, considering various statistical factors such as daily and weekly load, date/time of the year, environmental factors (e.g., dry bulb temperature and dew point), and user behaviour on weekdays and weekends. The test cases were conducted using historical data from ISO New England spanning the years 2004 to 2020. Moreover, the validation of the proposed model has been confirmed through comparing the results with those of Gaussian Process Regression (GPR), Regression Decision Tree (RDT), deep learning Bi-Long Short Memory (bi-LSTM), MLP, and conventional ANN. This article discusses the use of various performance indices such as MAPE, MPE, skewness, kurtosis, and risk indices for evaluating model performance. The performance of a developed model is compared with a conventional ANN model, and its performance is studied for both yearly and seasonal variations. In addition to existing indices, the article proposes two risk indices. The first is based on evaluating the standard deviation of load increment for each time, while the second is based on MC-ANN, the error between forecasted and actual loads. The risk assessment is compared between different cases such as actual load, load forecasting with ANN, and enhanced ANN-MC. Finally, the result confirms that the enhanced ANN-MC provides a higher yearly MPE value compared to other methods. In addition, it has a higher computational time than the conventional ANN-MC model, which is approximately 180.7s and 221.8s, respectively.

Swarm-based hybrid optimization algorithms: an exhaustive analysis and its applications to electricity load and price forecasting.

In this work, we intend to propose multiple hybrid algorithms with the idea of giving a choice to the particles of a swarm to update their position for the next generation. To implement this concept, Cuckoo Search Algorithm (CSA), Grey Wolf Optimization (GWO), Harris Hawks Optimization (HHO), and Whale Optimization Algorithm (WOA) have been utilized. Exhaustive possible combinations of these algorithms are developed and benchmarked against the base algorithms. These hybrid algorithms have been validated on twenty-four well-known unimodal and multimodal benchmarks functions, and detailed analysis with varying dimensions and population size is discussed for the same. Further, the efficacy of these algorithms has been tested on short-term electricity load and price forecasting applications. For this purpose, the algorithms have been combined with Artificial Neural Networks (ANNs) to evaluate their performance on the ISO New Pool England dataset. The results demonstrate that hybrid optimization algorithms perform superior to their base algorithms in most test cases. Furthermore, the results show that the performance of CSA-GWO is significantly better than other algorithms.

A novel combined model for probabilistic load forecasting based on deep learning and improved optimizer

As the transitions of the power industry to decarburization and distributed energy systems, the future uncertainty information of electric load is becoming essential in power systems planning and operation. However, a great number of studies focus on point forecasting, which only provides the expected value at each time step and it cannot provide uncertainty information. This paper proposed a novel probabilistic load forecasting model by combining quantile regression (QR) with a hybrid model to improve smart grid reliability. In addition, to further improve accuracy and solve the problem that the optimal model is not unique, we propose a new combined probabilistic forecasting model (CPFM). The CPFM employs the traditional statistical models and QR-machine learning models as alternative models; several alternative models with the best performance are combined through the improved multi-objective optimizer to obtain the final forecasting results. The ISO New England data is modeled as a case study to verify the effectiveness of the proposed CPFM. The comparative study includes 13 models, and the results show that the proposed CPFM has better performance in reliability, resolution, and sharpness.

A robust support vector regression model for electric load forecasting

Electric load forecasting is a crucial part of business operations in the energy industry. Various load forecasting methods and techniques have been proposed and tested. With growing concerns about cybersecurity and malicious data manipulations, an emerging topic is to develop robust load forecasting models. In this paper, we propose a robust support vector regression (SVR) model to forecast the electricity demand under data integrity attacks. We first introduce a weight function to calculate the relative importance of each observation in the load history. We then construct a weighted quadratic surface SVR model. Some theoretical properties of the proposed model are derived. Extensive computational experiments are based on the publicly available data from Global Energy Forecasting Competition 2012 and ISO New England. To imitate data integrity attacks, we have deliberately increased or decreased the historical load data. Finally, the computational results demonstrate better accuracy of the proposed robust model over other recently proposed robust models in the load forecasting literature.

Deep learning with regularized robust long‐ and short‐term memory network for probabilistic short‐term load forecasting

AbstractShort‐term electricity load forecasting plays an essential role in power system operation and energy trading. However, the strong volatility, seasonality, and uncertainly of the load series pose challenges to accurate short‐term load forecasting using existing forecasting models. To tackle these challenges, in this paper, we investigate a novel deep learning‐based probabilistic short‐term load forecasting model to predict the hourly ahead electricity load accurately. The proposed deep learning forecasting model embeds nonconvex minimax concave penalty in the robust long‐ and short‐term memory (LSTM) network to simplify its complex structure while extracting the important information. The model parameters in the proposed regularized and robust LSTM network are fine‐tuned using whale optimization algorithm which is a meta‐heuristic approach. For empirical applications, electricity load data collected from New York Independent System Operator and ISO New England are considered, and numerical results show that the proposed method outperforms other state‐of‐the‐art competitors in terms of both point and interval forecasting accuracy.

Using Interim Recommitment to Reduce the Operational-cost Impacts of Wind Uncertainty

Using wind-availability forecasts in day-ahead unit commitment can require expensive real-time operational adjustments. We examine the benefit of conducting interim recommitment between day-ahead unit commitment and real-time dispatch. Using a simple stylized example and a case study that is based on ISO New England, we compare system-operation costs with and without interim recommitment. We find an important tradeoff-later recommitment provides better wind-availability forecasts, but the system has less flexibility due to operating constraints. Of the time windows that we examine., hour-20 recommitment provides the greatest operational-cost reduction.

Dynamic simulation of a novel nuclear hybrid energy system with large-scale hydrogen storage in an underground salt cavern

In this study, a nuclear hybrid energy system (NHES) with large-scale hydrogen storage integrated with a gas turbine cycle is proposed as a flexible system for load following. The proposed system consists of a nuclear reactor, a steam Rankine cycle, a hydrogen electrolyzer, a storage system for hydrogen in an underground salt cavern, and a Brayton cycle that uses hydrogen as fuel to generate additional electricity to meet peak demand. A dynamic mathematical model is developed for each subsystem of the NHES. To evaluate the potential benefits of the system, a one-year study is conducted, using scaled grid demand data from ISO New England. The dynamic simulation results show that the system is capable of meeting the demand of the grid without additional electricity from outside sources for 93% of the year, while decreasing the number of ramping cycles of the nuclear reactor by 92.7%. There is also potential for economic benefits as the system only had to ramp up and down 7.4% of the year, which increased the nuclear capacity factor from 86.3% to 98.3%. The simulation results show that the proposed hybrid system improves the flexibility of nuclear power plants, provides more electricity, and reduces greenhouse gas emissions.

An enterprise control assessment case study of the energy–water nexus for the ISO New England system

The generation mix of Independent System Operator in New England (ISO-NE) is fundamentally changing. Nuclear, coal, and oil generation facilities are retiring and are replaced with natural gas, solar, and wind generation. Variable renewable energy resources (VREs) such as solar and wind present multiple operational challenges that require new and innovative ways to manage and control the grid. This paper studies how water supply systems (water and wastewater treatment), and water-dependent electricity generating resources (hydro, and thermal power plants) can be operated flexibly to enhance the reliability of the grid. The study’s methodology employs the novel Electric Power Enterprise Control System (EPECS) simulator to study power systems operation, and the System-Level Generic Model (SGEM) to study water consumption and withdrawals. This work considers six potential 2040 scenarios for the ISO-NE energy–water nexus (EWN). It presents a holistic analysis that quantifies grid imbalances, normal operating reserves, energy market production costs, and water withdrawals and consumption. For scenarios with high amounts of VREs, the study shows great potential of water resources to enhance grid flexibility through improvements in load-following (up to 12.66%), and ramping (up to 18.35%) reserves. Flexible operation also results in up to 10.90% reduction in the total time VREs are curtailed. Additionally, flexible operation reduces water withdrawals by up to 25.58%, water consumption by up to 5.30%, and carbon dioxide emissions by up to 3.46%. In general, this work provides significant insights into how to jointly control the water and energy supply systems to aid in their synergistic integration.

Identification of Low-Frequency Oscillation Modes Using PMU Based Data-Driven Dynamic Mode Decomposition Algorithm

Power system inter-area oscillations curtail the power transferring capabilities of the transmission lines in a large interconnected power system. Accurate identification of dominant modes and associated contributing generators is important to avoid power system failures by taking appropriate remedial measures. This paper proposes a multi-channel Improved Dynamic Mode Decomposition (IDMD) algorithm-based modal analysis technique using Synchrophasors measurement. First, a reduced-order dynamic power system model is estimated and using this model dominant oscillation modes, corresponding modes shapes, damping ratio, coherent group of generators, participation factors are determined. To improve the accuracy data stacking technique is used to capture detailed information of the system. An optimal hard threshold technique is utilized to select the most optimal model order to avoid uncertainties due to the presence of high level of measurement noise. The study results show that the proposed algorithm gives an accurate and robust solution even in systems having high level of noise in the measurement data. The performance of the proposed technique is tested on simulated data from two-area four-machine system and wNAPS 41-bus 16-generator system with PMU measurements corrupted with different levels of measurement noise. To further strengthen the viewpoint, the proposed method is validated on real-time PMU measurement from ISO New England data to validate the accuracy of the proposed work.

Robust Data Predictive Control Framework for Smart Multi-Microgrid Energy Dispatch Considering Electricity Market Uncertainty

With the emerging technologies for Energy Intent (EI) and data-driven applications, the conventional power grid network is undergoing a radical modernization. An efficient energy management and electricity price forecasting remains a challenging task. In this paper, a new Robust Data Predictive Control framework for Energy Management System (RDPC-EMS) is developed to overcome the uncertainty of the electricity retail price market and minimize the total operating costs for the multi-microgrids (MMG) system. The proposed framework solves the economic energy dispatch based on an accurate Electricity Price Forecasting (EPF) by an Outlier-Robust Extreme Learning Machine (OR-ELM) algorithm and a two layers cooperative Distributed Model Predictive Control (DMPC). The First level provides an optimal energy scheduling between the Distribution System Operator (DSO) and cooperative microgrids systems to minimize the operating cost based on the forecasted electricity price. In contrast, second level maintains the supply-demand balance by applying the optimal energy scheduling from the first layer through an adjustment of the distributed energy resources (DER). The electricity retail price prediction is assessed using real dataset from the Iso New England electricity market. The OR-ELM regression method shows a significant forecasting performance in terms of error metrics. For instance, the mean absolute error in the training stage 2.05% for OR-ELM with a comparison of 4.17% and 6.29% for Support Vector Regression (SVR), and Artificial Neural Network (ANN) models respectively. Finally, simulation results demonstrate the efficiency RDPC-EMS for daily operating cost reduction, with decrease of 15% for MG 1 and 16% for MG 2.

A novel method for locating the source of sustained oscillation in power system using synchrophasors data

Large interconnected power systems are usually subjected to natural oscillation (NO) and forced oscillation (FO). NO occurs due to system transient response and is characterized by several oscillation modes, while FO occurs due to external perturbations driving generation sources. Compared to NO, FO is considered a more severe threat to the safe and reliable operation of power systems. Therefore, it is important to locate the source of FO so corrective actions can be taken to ensure stable power system operation. In this paper, a novel approach based on two-step signal processing is proposed to characterize FO in terms of its frequency components, duration, nature, and the location of the source. Data recorded by the Phasor Measurement Units (PMUs) in a Wide Area Monitoring System (WAMS) is utilized for analysis. As PMU data usually contains white noise and appears as multi-frequency oscillatory signal, the first step is to de-noise the raw PMU data by decomposing it into a series of intrinsic mode functions (IMF) using Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) technique. The most appropriate IMF containing the vital information is selected using the correlation technique. The second step involves various signal processing and statistical analysis tools such as segmented Power Spectrum Density (PSD), excess kurtosis, cross PSD etc. to achieve the desired objectives. The analysis performed on the simulated two-area four-machine system, reduced WECC-179 bus 29 machine system, and the real-time power system PMU data set from ISO New England, demonstrates the accuracy of the proposed method. The proposed approach is independent of complex network topologies and their characteristics, and is also robust against measurement noise usually contained in PMU data.

A Novel Event Detection and Classification Scheme Using Wide-Area Frequency Measurements

Automated event detection and classification are vital to power system monitoring. This article proposes a novel event detection and classification scheme based on wide area frequency measurement system (WAFMS). Raw frequency measurements obtained from WAFMS are used as the only input to the event detection and classification algorithm (EDCA). Wavelet-based signal pre-processing is used to denoise the data. Afterward, the rate of change of frequency (ROCOF) is estimated from the frequency measurements using the Kalman filter (KF). In the same step, phase angle difference (PAD) across different stations is estimated using WAFMS. Thus, the overall algorithm uses three features such as frequency, ROCOF, and PAD to detect and classify events in the power system. In the first step, an event is detected based on standard deviation (SD) of estimated ROCOF and PAD. In the second step, four types of events are classified using wide area frequency measurements. The suggested algorithms have been validated with real WAFMS data from the Indian Power System, recent 9th August 2019 U.K. blackout data collected from the U.K. power system, and 20th July 2017 oscillation event data obtained from ISO New England (ISO-NE) power system. As a promising tool for power system monitoring, the suggested scheme requires less input measurement for decision making and is having a low computational complexity, which is suitable for practical application.

Computing a Strategic Decarbonization Pathway: A Chance-Constrained Equilibrium Problem

US transmission systems and wholesale electricity markets, albeit federally regulated, often span across multiple state jurisdictions. In this environment, state regulators can strategically exploit this techno-economic coupling to advance their clean energy policy goals at the expense of neighboring jurisdictions. This paper investigates strategic regulatory competition to understand its effect on achieving Renewable Portfolio Standards (RPS). We formulate a chance-constrained equilibrium problem with equilibrium constraints (CC-EPEC), which considers multiple state regulators, acting in coordination with in-state power companies, to implement RPS goals in the least-cost manner. To solve this CC-EPEC, we customize a Progressive Hedging (PH) algorithm. The case study uses the CC-EPEC and PH algorithm to analyze the effects of state regulatory competition in the ISO New England system.

PLS-CNN-BiLSTM: An End-to-End Algorithm-Based Savitzky–Golay Smoothing and Evolution Strategy for Load Forecasting

This paper proposes an effective deep learning framework for Short-Term Load Forecasting (STLF) of multivariate time series. The proposed model consists of a hybrid Convolutional neural network-Bidirectional Long Short-Term Memory (CBiLSTM) based on the Evolution Strategy (ES) method and the Savitzky–Golay (SG) filter (SG-CBiLSTM). The adopted methodology incorporates the virtue of different prepossessing blocks to enhance the performance of the CBiLSTM model. In particular, a data-augmentation strategy is employed to synthetically improve the feature representation of the CBiLSTM model. The augmented data is forwarded to the Partial Least Square (PLS) method to select the most informative features above the predefined threshold. Next, the SG algorithm is computed for smoothing the load to enhance the learning capabilities of the underlying system. The structure of the SG-CBiLSTM for the ISO New England dataset is optimized using the ES technique. Finally, the CBiLSTM model generates output forecasts. The proposed approach demonstrates a remarkable improvement in the performance of the original CBiLSTM model. Furthermore, the experimental results strongly confirm the high effectiveness of the proposed SG-CBiLSTM model compared to the state-of-the-art techniques.

Serverless computing for cloud-based power grid emergency generation dispatch

Operating a modern power grid reliably in case of SCADA/EMS failure or amid difficult times like COVID-19 pandemic is a challenging task for grid operators. In [11], a PMU-based emergency generation dispatch scheme has been proposed to help the system operators with the supply and demand balancing; however, its realization highly relies on the control center infrastructure for computing and communication. This work, rather than using the on-premises server and dispatch communication system, proposes and implements a cloud-centric serverless architecture to ensure the operation continuity regardless of local infrastructure’s availability and accessibility. Through its prototype implementation and evaluation at ISO New England, the solution has demonstrated two major advantages. Firstly, the cloud infrastructure is independent and fault-tolerant, providing grid monitoring and control capability even when EMS loses the corresponding functionality or when operators need to work remotely away from the control center. Secondly, the overall design is event-driven using serverless cloud services in response to the SCADA/EMS failure event. Thanks to “serverless”, the burden of the server provisioning and maintenance can be avoided from the user side. The cost of using public cloud services for this solution is extremely low since it is architected and implemented based on the event-driven Function-as-a-Service (FaaS) model. This work also develops a comprehensive cyber security mechanism to comply with critical infrastructure requirements for the power grid, which can serve as an exemplary framework for other grid operators to secure their cloud services.