California Independent System Operator Research Articles

Aggregating Distributed Energy Resources: Efficiency and Market Power

Problem definition: The rapid expansion of distributed energy resources (DERs) is one of the most significant changes to electricity systems around the world. Examples of DERs include solar panels, electric storage, thermal storage, and combined heat and power plants. Because of the small supply capacities of these DERs, it is impractical for them to participate directly in the wholesale electricity market. We study in this paper the question of how to integrate these DER supplies into the electricity market, with the objective of achieving full market efficiency. Methodology/results: We study four aggregation models, where there is an aggregator who, with the knowledge of DERs’ utility functions and generations, procures electricity from DERs, and sells them in the wholesale market. In the first aggregation model, a profit-maximizing aggregator announces a differential two-part pricing policy to the DER owners. We show that this model preserves full market efficiency, that is, the social welfare achieved by this model is the same as that when DERs participate directly in the wholesale market. In the second aggregation model, the profit-seeking aggregator is forced to impose a uniform two-part pricing policy to prosumers from the same location, and we numerically show that there can be large efficiency loss. In the third (fourth) aggregation model, a uniform (semiuniform) two-part pricing policy is applied to DER owners, whereas the aggregator becomes fully regulated but is guaranteed nonnegative (positive) profit. It is shown that these models again achieve full market efficiency. Furthermore, we show that DER aggregation also leads to a reduction in the market power of conventional generators. Managerial implications: DER aggregation via profit-seeking and/or regulated aggregators has been investigated by California Independent System Operator and New York Independent System Operator, among others, and the recent Federal Energy Regulatory Commission Order No. 2222 paved the way for aggregators to bid in the wholesale market. Our four aggregation models may shed light on how DERs should be included in the wholesale electricity market. Funding: This work was supported by the King Fahd University of Petroleum and Minerals [Grant INML2106] and the National Science Foundation [Grant 1832230]. Supplemental Material: The online appendices are available at https://doi.org/10.1287/msom.2021.0539 .

Locational Role Analysis of Energy Storage Systems Based on Optimal Capacity Needs and Operations under High Penetration of Renewable Energy

As the need for energy storage systems (ESSs) capacity is increasing due to high accommodation of renewable resources, it is crucial to analyze in which location and for what purpose the ESSs are required to achieve the highest efficiency. Investors and system operators can place and operate the ESSs as expected based on this analysis. Therefore, this study assesses the specific roles of ESSs in a grid system based on their optimal capacity needs, locations, and operations. A long-term simulation model using mixed-integer programming is proposed to obtain these optimal solutions, such as ESS capacity and operational schedules for energy and reserves. Four-week operational simulations are performed for each month using data from the California Independent System Operator. ESSs are placed at sites with solar photovoltaic (PV) systems or wind farms, at baseload generator buses, and at load buses to verify the role of ESSs, depending on the locational differences. The detailed roles are analyzed from the aspects of flexible capacity supply, reserve deployments, time-shifting renewable and thermal energy generation, and costs. The results show that the ESSs on the baseload generation side provide flexibility by time-shifting baseload generation and turn on baseload generators, even when the net load is small. For instance, the required capacity of the flexible thermal generators, such as natural gas turbine generators, is about 3004 MW without the ESS operations in May. When 450 MW ESSs colocated with solar PVs are operated, the required flexible capacity of the thermal generators is lowered to 2404 MW. Moreover, ESSs are highly utilized as a downward reserve provider, although their costs for reserves are higher than thermal generators.

As Grids Reach 100% Renewable at Peak, Growing Curtailment of 8 Gigawatts Looms as a Challenge to Decarbonization

In April 2022, the California Independent System Operator (CAISO) power grid reached momentary peaks of 100% renewable energy for the first time. After a year, momentary 100% supply from renewables isn't a news any more, and CAISO reported record wind and solar renewable curtailment of 606 GWh (March 2023) and 686 GWh (April 2023). In addition, peak renewable coincided with curtailment rates of 8 GW in April 2023. Our prior studies documented monthly-peak renewable curtailment growth of 40% annually (23 GWh in March 2015 growing to 82 GWh in March 2017). Updating for 2018-2023 shows that 40% annual growth has continued through April 2023 and with it, CAISO has reached an average sunlight hours curtailment rate of 2 GW. If this 9-year trend continues unchecked for the next 5 years, monthly curtailment is projected to reach 3.3 TWh in both March and April of 2028, an average sunlight curtailment rate of nearly 11 GW. We analyze causes of increased curtailment, and discuss its likely future trajectory (growth). We also discuss the challenges its growth represents for grid decarbonization. Finally we outline the difficulties in reducing curtailment growth and highlight several new opportunities for power grids and computing systems.

Congestion Relief Services by Vehicle-to-Grid Enabled Electric Vehicles Considering Battery Degradation

Battery electric vehicles (BEVs) offer substantial potential to enhance the electric grid through bi-directional charging technologies. In essence, BEVs, functioning as portable battery energy storage systems, play a pivotal role in enabling the seamless integration of renewable energy, grid optimization, and ancillary services. This article sets out to explore the value of BEVs equipped with Vehicle-to-Grid (V2G) for grid operators, particularly in the context of alleviating congestion. This valuable service, though not accompanied by direct monetary compensation for users, holds significant promise in minimizing congestion and renewable energy curtailment. This study utilizes the Day-Ahead Locational Marginal Price (LMP) data obtained from various locations within California Independent System Operator (CAISO) to ascertain the financial benefits to BEVs located on either side of congestion at different grid nodes, across various months. Similar analysis is performed on some of the largest solar energy plants in California. Mixed-integer linear programs are used to optimize the charging/discharging decisions for the BEV for maximizing revenue from LMP arbitrage and for minimizing the congestion component of LMP. Additionally, we take into account the impact of battery degradation, quantified as a cost per kilowatt-hour ($/kWh), and integrate this factor into our assessment to understand the evolving discharging behavior of BEVs. The article compares the benefits from the BEVs towards congestion minimization for the two different optimization scenarios, discusses seasonality, and addresses the importance of adequately compensating BEV users and incentivizing them to prioritize congestion relief during specific time intervals.

Residential electricity demand on CAISO Flex Alert days: a case study of voluntary emergency demand response programs

Abstract The California Independent System Operator (CAISO) utilizes a system-wide, voluntary demand response (DR) tool, called the Flex Alert program, designed to reduce energy usage during peak hours, particularly on hot summer afternoons when surges in electricity demand threaten to exceed available generation resources. However, the few analyses on the efficacy of CAISO Flex Alerts have produced inconsistent results and do not investigate how participation varies across sectors, regions, population demographics, or time. Evaluating the efficacy of DR tools is difficult as there is no ground truth in terms of what demand would have been in the absence of the DR event. Thus, we first define two metrics that to evaluate how responsive customers were to Flex Alerts, including the Flex Period Response, which estimates how much demand was shifted away from the Flex Alert period, and the Ramping Response, which estimates changes in demand during the first hour of the Flex Alert period. We then analyze the hourly load response of the residential sector, based on ∼200 000 unique homes, on 17 Flex Alert days during the period spanning 2015–2020 across the Southern California Edison (SCE) utility’s territory and compare it to total SCE load. We find that the Flex Period Response varied across Flex Alert days for both the residential (−18% to +3%) and total SCE load (−7% to +4%) and is more dependent on but less correlated with temperature for the residential load than total SCE load. We also find that responsiveness varied across subpopulations (e.g. high-income, high-demand customers are more responsive) and census tracts, implying that some households have more load flexibility during Flex Alerts than others. The variability in customer engagement suggests that customer participation in this type of program is not reliable, particularly on extreme heat days, highlighting a shortcoming in unincentivized, voluntary DR programs.

Using neural networks to forecast marginal emissions factors: A CAISO case study

Marginal Emissions Factors (MEFs) quantify the time-dependent changes in CO2 emissions resulting from changes in electricity consumption. Accurate MEFs are critical for calculating the emissions impact of demand-side management (DSM) activities and programs, but current methods of calculating MEFs are limited by their temporal resolution, accuracy (particularly in grids with high penetrations of variable renewable energy), and ability to predict MEFs ahead of time, reducing their utility for DSM. We improve upon existing techniques by introducing a novel multi-layer perceptron to linear composite model that uses publicly available grid data to calculate historical MEFs and predict day-ahead MEFs. We test our model on publicly-available data from the California Independent System Operator over the period of 2019–2021, a grid with high daytime VRE generation. Results indicate that our model produces more accurate and more granular demand-based MEF estimations than comparable regression techniques and maintains high accuracy when use to forecast future MEFs. Our MEF framework can be applied to other regional grids to evaluate and design DSM strategies that leverage CO2 emissions-reductions as motivation for altering electricity consuming behaviors.

Principal component analysis of day‐ahead electricity price forecasting in CAISO and its implications for highly integrated renewable energy markets

AbstractElectricity price forecasting is crucial for grid management, renewable energy integration, power system planning, and price volatility management. However, poor accuracy due to complex generation mix data and heteroskedasticity poses a challenge for utilities and grid operators. This paper evaluates advanced analytics methods that utilize principal component analysis (PCA) to improve forecasting accuracy amidst heteroskedastic noise. Drawing on the experience of the California Independent System Operator (CAISO), a leading producer of renewable electricity, the study analyzes hourly electricity prices and demand data from 2016 to 2021 to assess the impact of day‐ahead forecasting on California's evolving generation mix. To enhance data quality, traditional outlier analysis using the interquartile range (IQR) method is first applied, followed by a novel supervised PCA technique called robust PCA (RPCA) for more effective outlier detection and elimination. The combined approach significantly improves data symmetry and reduces skewness. Multiple linear regression models are then constructed to forecast electricity prices using both raw and transformed features obtained through PCA. Results demonstrate that the model utilizing transformed features, after outlier removal using the traditional method and SAS Sparse Matrix method, achieves the highest forecasting performance. Notably, the SAS Sparse Matrix outlier removal method, implemented via proc RPCA, greatly contributes to improved model accuracy. This study highlights that PCA methods enhance electricity price forecasting accuracy, facilitating the integration of renewables like solar and wind, thereby aiding grid management and promoting renewable growth in day‐ahead markets.This article is categorized under: Energy and Power Systems > Energy Management Energy and Power Systems > Distributed Generation Emerging Technologies > Digitalization

A tri-level programming-based frequency regulation market equilibrium under cyber attacks

Owing to their flexibility and rapid response, grid-connected distributed energy resources (DERs) are wielding growing influence in frequency regulation markets (FRMs). Nevertheless, compared with conventional large-scale generators, small-scale DERs are usually weakly shielded by cyber security measures. This offers attackers the opportunity of disrupting the clearing and settlement of FRMs by manipulating the bid information of DERs. In this paper, the frequency regulation market equilibrium is studied considering the dynamic gaming between attackers and defenders, both of which need the knowledge of FRMs for their interventions. Specifically, a tri-level programming model characterizing the attacker–defender–operator (ADO) interdiction problem in FRMs is developed and then analyzed using a column and constraint generation algorithm, thereby achieving market equilibrium representing the defender's best response to the attacker. The defense strategy in the market equilibrium can attenuate the negative influence of cyber attacks upon the FRMs to the maximum extent. Finally, based on the operating rules of the California Independent System Operator, the FRM process considering the ADO interdiction is simulated and the numerical equilibrium results are presented.

Predicting the Gap in the Day-Ahead and Real-Time Market Prices Leveraging Exogenous Weather Data

Predicting the price gap between the day-ahead Market (DAM) and the real-time Market (RTM) plays a vital role in the convergence bidding mechanism of Independent System Operators (ISOs) in wholesale electricity markets. This paper presents a model to predict the values of the price gap between the DAM and RTM using statistical machine learning algorithms and deep neural networks. In this paper, we seek to answer these questions: What will be the impact of predicting the DAM and RTM price gap directly on the prediction performance of learning methods? How can exogenous weather data affect the price gap prediction? In this paper, several exogenous features are collected, and the impacts of these features are examined to capture the best relations between the features and the target variable. An ensemble learning algorithm, namely the Random Forest (RF), is used to select the most important features. A Long Short-Term Memory (LSTM) network is used to capture long-term dependencies in predicting direct gap values between the markets stated. Moreover, the advantages of directly predicting the gap price rather than subtracting the price predictions of the DAM and RTM are shown. The presented results are based on the California Independent System Operator (CAISO)’s electricity market data for two years. The results show that direct gap prediction using exogenous weather features decreases the error of learning methods by 46%. Therefore, the presented method mitigates the prediction error of the price gap between the DAM and RTM. Thus, the convergence bidders can increase their profit, and the ISOs can tune their mechanism accordingly.

Multi-prediction of electric load and photovoltaic solar power in grid-connected photovoltaic system using state transition method

In the grid-connected photovoltaic system (GPVS), due to characteristics of fluctuation and intermittency for photovoltaic solar power, and high randomness for electric load, it is of great difficulty for integrating photovoltaic solar power into power grid. Therefore, an accurate prediction of short-term electric load and photovoltaic solar power is of great importance for balancing supply and demand. Currently, numerous isolated models about the forecasting of electric load and photovoltaic solar power have emerged, while the coupling effect between them has been hardly considered and lower stability of existing methods brings great difficulty in providing reliable predictions at practical applications. To address this gap, this paper proposes an interpretable multi-prediction model for short-term (day-ahead) electric load and photovoltaic solar power forecasting. In the framework, a non-parametric functional principal component analysis (FPCA) is constructed to extract the overall trend and identify dominant modes of variation in the daily electric load and photovoltaic solar power data. Furthermore, state transition matrix is proposed to comprehensively interpret the coupling effect, with which a novel multi-prediction strategy that takes advantage of coupling effect is further introduced, where the Maximum Likelihood Estimation (MLE) is employed to estimate unknown parameters. Moreover, data from California Independent System Operator (ISO) is utilized to investigate the performance of proposed method, and its results are compared with those from other widely-used techniques. Results show that the proposed method can increase prediction accuracy of electric load and photovoltaic solar power by 16.84% and 10.57%, respectively, with narrow fluctuations and reasonable computational cost (211.11 s), demonstrating that it can provide better predictions in terms of prediction accuracy, stability and applicability.

Residential precooling on a high-solar grid: impacts on CO2 emissions, peak period demand, and electricity costs across California

Abstract As regional grids increase penetrations of variable renewable electricity (VRE) sources, demand-side management (DSM) presents an opportunity to reduce electricity-related emissions by shifting consumption patterns in a way that leverages the large diurnal fluctuations in the emissions intensity of the electricity fleet. Here we explore residential precooling, a type of DSM designed to shift the timing of air-conditioning (AC) loads from high-demand periods to periods earlier in the day, as a strategy to reduce peak period demand, CO2 emissions, and residential electricity costs in the grid operated by the California Independent System Operator (CAISO). CAISO provides an interesting case study because it generally has high solar generation during the day that is replaced by fast-ramping natural gas generators when it drops off suddenly in the early evening. Hence, CAISO moves from a fleet of generators that are primarily clean and cheap to a generation fleet that is disproportionately emissions-intensive and expensive over a short period of time, creating an attractive opportunity for precooling. We use EnergyPlus to simulate 480 distinct precooling schedules for four single-family homes across California’s 16 building climate zones. We find that precooling a house during summer months in the climate zone characterizing Downtown Los Angeles can reduce peak period electricity consumption by 1–4 kWh d−1 and cooling-related CO2 emissions by as much as 0.3 kg CO2 d−1 depending on single-family home design. We report results across climate zone and single-family home design and show that precooling can be used to achieve simultaneous reductions in emissions, residential electricity costs, and peak period electricity consumption for a variety of single-family homes and locations across California.

Market analysis for the integration of new power technologies: A case study of the deployment of hybrid fossil-based generator plus energy storage (ES-FE)

This study examines the national landscape of hybridized fossil energy (FE) power plants with energy storage (ES) technologies (“ES-FE”) and presents the compilation of an ES-FE dataset, which includes over 65 ES-FE projects and concepts in the United States, comprising approximately 500 MWh of co-located ES capacity with FE power plants. This study also estimates the economic feasibility of adding ES to existing FE power plants by characterizing the potential revenues that can be generated by the ES component through flexibility and capacity value. The analysis focuses on ES technologies with 2- to 10-h. durations located in four U.S. independent system operators (ISOs): Midcontinent ISO (MISO), Electric Reliability Council of Texas (ERCOT), PJM Interconnection (PJM), and California ISO (CAISO), which have +70,000 MW of combined FE power capacity that could add ES. Annual revenues are estimated for the ES component using a what-if-analysis approach, for capacity value, price arbitrage, or ancillary services provision. The results show that annual revenues depend on the end-use storage service, wholesale electricity and capacity market prices, and ES technology operation parameters such as discharging duration and cycling frequency. When performing a sensitivity analysis, ES accrues $7–178/kW-yr. via price arbitrage and ancillary services provision in the four ISOs, and $13–92/kW-yr. when providing capacity value only in MISO and PJM. A cash flow analysis is performed to estimate the net present value (NPV) of the ES addition using a range of ES costs. The study finds that for most ES technologies considered, these revenues alone are insufficient to achieve economic feasibility. Of the 1645 total runs analyzed, 115 had positive NPVs (7%). Therefore, other revenue streams or monetizable benefits are necessary to achieve the break-even point.

Flattening Energy-Consumption Curves by Monthly Constrained Direct Load Control Contracts

California Utility Firm Implements Innovative Model, Reducing Costs by 4% A California utility firm has successfully implemented a pioneering model to balance electricity demand and supply while minimizing costs. By utilizing direct load control contracts (DLCCs), the firm can reduce energy consumption during peak hours. Researchers developed an integer stochastic dynamic optimization problem that considers monthly and annual constraints, allowing for effective execution of DLCCs. Incorporating a “reduce-to-threshold” policy to flatten energy-consumption curves during high demand, the model was verified using real data from the California Independent System Operator. When implemented, the utility firm achieved an impressive cost reduction of approximately 4%. Sensitivity analysis was conducted to enhance customer experience and improve DLCC contract features. The success of this innovative model highlights the potential of DLCCs and advanced optimization techniques in the energy sector, offering a blueprint for other utility companies seeking to optimize grid stability and reduce costs.

Analysis and prediction of the penetration of renewable energy in power systems using artificial neural network

The curtailment of renewable energy worsens with increasing penetration in power systems, so it is necessary to explore the upper limit value of the penetration of renewable energy (PRE). This paper uses artificial neural networks (ANN) to study the historical data of California independent system operator (CAISO), analyze the fluctuation balance strategy of wind and solar power, and predict the upper limit value of the PRE, which will peak at 40.5% in 2025. In addition, this paper also simulates the inclusion of energy storage unit (ESU) with an installed capacity of 3 GWh in the grid to reduce curtailments, and analyzes the grid operating conditions and the upper limit value of the PRE, showing that the storage units with an installed capacity of only 3% of the average daily power output of solar energy recover 58% of the annual curtailments (2021), and the maximum PRE is 41.2%.

The role of electricity market design for energy storage in cost-efficient decarbonization

The role of electricity market design for energy storage in cost-efficient decarbonization

A hybrid framework for short term load forecasting with a navel feature engineering and adaptive grasshopper optimization in smart grid

Short-term load forecasting (STLF) enables distribution system operators to perform efficient energy management by flexibly engaging energy consumers under the intelligent demand-response program in the smart grid (SG). This paper develops a fast and accurate hybrid load forecasting model with this motivation. The proposed model integrates a locally weighted support vector regression (LWSVR) based forecaster with two modules. These modules are feature engineering (FE) and adaptive grasshopper optimization (AGO) based optimizers. In the FE module, firstly, the proposed hybrid feature selector (HFS) is developed using wrapper and filter techniques to determine an optimal subset of features. Furthermore, the instance-based Relief-F (REF) and information theoretic-based mutual information (MI) filters are used to decrease the curse of feature dimensionality by finding and eradicating extraneous features. In addition, to overcome the overfitting problem, the HFS module is further optimized using the recursive feature elimination (RFE) wrapper feature selection technique. The essential features are then extracted using a radial basis Kernel-based principal component analysis (RBF-KPCA) algorithm to eliminate the dimensionality reduction problem. The AGO algorithm tunes the LWSVR model’s appropriate parameters to effectively evade entrapping into local optimum and yield accurate prediction results. However, the efficacy and productiveness of the forecasting model are differentiated equally by its convergence rate and stability. Actual hourly load data of two states of Australia (New South Wales (NSW) and Victoria (VIC)) and California Independent System Operator United States (CAISO-US) are employed as a case study to estimate the effectiveness and applicability of the designed model. Empirical results show that the devised model surpasses benchmark models (single and hybrid) in terms of stability, accuracy, and convergence rate.

A data-driven framework for quantifying consumption-based monthly and hourly marginal emissions factors

The fleet of power plants supplying electricity to a power grid varies diurnally and seasonally, creating large time-dependent differences in the emissions associated with consuming electricity, particularly in grids with high penetrations of renewable electricity generators. In addition, modern grids are incorporating more demand-side interventions that incentivize electricity end users to temporarily modify their electricity consuming behavior in efforts to change the shape and magnitude of electricity consumption over a period of time. Current methods for quantifying the emissions associated with marginal shifts in electricity consumption are not sufficient given the changing dynamics of supply-side generation resources. This study introduces a novel multiple linear regression model that utilizes historical demand, variable renewable generation, and CO2 emissions data to quantify hourly marginal emissions factors for the years of 2019 and 2020. The developed consumption-based CO2 accounting method includes the emissions embedded in net electricity imports in addition to emissions from in-region generators. The proposed framework is applied to the case study of California Independent System Operator (CAISO), revealing a wide range of hourly-level marginal emissions factors (89–503 kgCO2/MWh) during the period of study. The proposed method improves upon the existing literature by proposing a consumption-based method that is well suited for estimating emissions avoided through demand-side changes in load, particularly in electric grids, like CAISO, with high renewable energy penetrations.

The value of concentrating solar power in ancillary services markets

Ancillary services, such as spinning reserves, can provide grid reliability and contribute to profitability of an energy resource. We exercise an existing dispatch optimization model to estimate the profitability of a concentrating solar power plant by incorporating the sale of spinning reserves in the ancillary service market using the National Renewable Energy Laboratory’s System Advisor Model to simulate operations within a 72-h rolling horizon framework. Assuming a price-taker approach with day-ahead energy and spinning reserve prices from both the California Independent System Operator and the Electricity Reliability Council of Texas, we find that selling spinning reserves in addition to electric energy increases plant profitability by up to 7% with perfect knowledge of day-ahead pricing and solar resource availability. This finding suggests that spinning reserve markets provide significant value streams to concentrating solar power plants that can leverage thermal energy storage to offer reliable production in the short-to-medium term.

Operating Dynamic Reserve Dimensioning Using Probabilistic Forecasts

The rapid integration of variable renewable energy sources (VRES) into power grids increases variability and uncertainty of the net demand, making the power system operation challenging. Operating reserve is used by system operators to manage and hedge against such variability and uncertainty. Traditionally, reserve requirements are determined by rules-of-thumb (static reserve requirements, e.g., NERC Reliability Standards), and more recently, dynamic reserve requirements from tools and methods which are in the adoption process such as Dynamic Assessment and Determination of Operating Reserve ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DynADOR</uri> ), Dynamic Reserves Dimensioning ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DRD</uri> ), and <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RESERVE</uri> , among others. While these methods/tools significantly improve the static rule-of-thumb approaches, they rely exclusively on deterministic data (i.e., best guess only). Consequently, these methods disregard the probabilistic uncertainty thresholds associated with specific days and their weather conditions (i.e., best guess plus probabilistic uncertainty). This work presents practical approaches to determine the operating reserve requirements leveraging the wealth information from probabilistic forecasts. Proposed approaches are validated and tested using actual data from the California Independent System Operator (CAISO) system. Results show the benefits in terms of risk reduction of considering the probabilistic forecast information into the dimensioning process of operating reserve requirements.

Probability density function forecasting of electricity price: Deep gabor convolutional mixture network

This paper establishes a framework for probabilistic electricity price forecasting in the electrical markets using an end-to-end and direct approach. In the proposed method, firstly, a deep Gabor filter-oriented convolutional network is designed as the strong deep structure. Then, the designed deep Gabor network is developed as a deep mixture network to predict a set of probability density functions (PDFs) in the next hours. To do so, a probabilistic loss function is formulated. The results are validated by the actual electricity market data in California independent system operator (CAISO) and show high level of accuracy and reliability in comparison with other state-of-the-arts methods