Stochastic Dual Dynamic Programming Research Articles

Uncertainty dynamics in energy planning models: An autoregressive and Markov chain modeling approach

This paper deals with the modeling of stochastic processes in long-term multi-stage energy planning problems when limited information is available about the distributions of such processes. Starting from simple estimates of variation intervals for uncertain parameters, such as energy demands and costs, we model the temporal correlation of these parameters through carefully constructed autoregressive (AR) models that respect the intervals defined in each period. We introduce a coefficient for “zigzag” effects in the evolution of uncertain processes, which controls the correlation across periods and also the likelihood of extreme scenarios. To preserve the convexity of the stochastic problem, we discretize the AR models associated with the cost parameters involved in the objective function by Markov chains. The resulting formulation is then solved using a Stochastic Dual Dynamic Programming (SDDP) algorithm available in the literature that handles finite-state Markov chains. Our numerical experiments, carried out on the Swiss energy system, show a very desirable adaptation strategy of investment decisions to uncertainty scenarios, a behavior that is not observed when the temporal correlation is ignored. Moreover, the solutions lead to better out-of-sample cost performances, especially on extreme scenario realizations, than the non-correlated ones which usually result in overcapacities to protect against high, but unlikely, parameter variations over time.

Dual Stochastic Dual Dynamic Programming for Multi-Stage Economic Dispatch With Renewable Energy and Thermal Energy Storage

Dual Stochastic Dual Dynamic Programming for Multi-Stage Economic Dispatch With Renewable Energy and Thermal Energy Storage

Approximation of multistage stochastic programming problems by smoothed quantization

AbstractWe present an approximation technique for solving multistage stochastic programming problems with an underlying Markov stochastic process. This process is approximated by a discrete skeleton process, which is consequently smoothed down by means of the original unconditional distribution. Approximated in this way, the problem is solvable by means of Markov Stochastic Dual Dynamic Programming. We state an upper bound for the nested distance between the exact process and its approximation and discuss its convergence in the one-dimensional case. We further propose an adjustment of the approximation, which guarantees that the approximate problem is bounded. Finally, we apply our technique to a real-life production-emission trading problem and demonstrate the performance of its approximation given the “true” distribution of the random parameters.

Medium-term stochastic hydrothermal scheduling with short-term operational effects for large-scale power and water networks

The high integration of variable renewable sources in electric power systems entails a series of challenges inherent to their intrinsic variability. A critical challenge is to correctly value the water available in reservoirs in hydrothermal systems, considering the flexibility that it provides. In this context, this paper proposes a medium-term multistage stochastic optimization model for the hydrothermal scheduling problem solved with the stochastic dual dynamic programming algorithm. The proposed model includes operational constraints and simplified mathematical expressions of relevant operational effects that allow more informed assessment of the water value by considering, among others, the flexibility necessary for the operation of the system. In addition, the hydrological uncertainty in the model is represented by a vector autoregressive process, which allows capturing spatio-temporal correlations between the different hydro inflows. A calibration method for the simplified mathematical expressions of operational effects is also proposed, which allows a detailed short-term operational model to be correctly linked to the proposed medium-term linear model. Through extensive experiments for the Chilean power system, the results show that the difference between the expected operating costs of the proposed medium-term model, and the costs obtained through a detailed short-term operational model was only 0.1%, in contrast to the 9.3% difference obtained when a simpler base model is employed. This shows the effectiveness of the proposed approach. Further, this difference is also reflected in the estimation of the water value, which is critical in water shortage situations.

Batch Learning SDDP for Long-Term Hydrothermal Planning

We consider the stochastic dual dynamic programming (SDDP) algorithm - a widely employed algorithm applied to multistage stochastic programming - and propose a variant using experience replay - a batch learning technique from reinforcement learning. To connect SDDP with reinforcement learning, we cast SDDP as a Q-learning algorithm and describe its application in both risk-neutral and risk-averse settings. We demonstrate the superiority of the algorithm over conventional SDDP by benchmarking it against PSR's SDDP software using a large-scale instance of the long-term planning problem of inter-connected hydropower plants in Colombia. We find that SDDP with batch learning is able to produce tighter optimality gaps in a shorter amount of time than conventional SDDP. We also find that batch learning improves the parallel efficiency of SDDP backward passes.

Retrofit Planning and Flexible Operation of Coal-Fired Units Using Stochastic Dual Dynamic Integer Programming

The continuous growth of wind power requires greater power system operational flexibility owing to the variability and uncertainty of wind power generation. Retrofitting existing coal-fired units is a demonstrated cost-effective method for improving system flexibility. This study proposes a decision support tool that provides optimal investment decisions to enhance power system operational flexibility. A novel flexibility retrofit planning (FRP) model is proposed in which the embedded operational problem is a stochastic mixed-integer nonlinear programming (SMINLP) unit commitment problem. In the operational unit commitment model, novel mixed-integer nonlinear formulations are proposed to represent the coal-fired unit attribute (minimum power output, startup and shutdown times, and maximum ramp rate) modifications after retrofitting. A new solution method based on the Benders decomposition (BD) and stochastic dual dynamic integer programming (SDDIP) algorithms is proposed to solve the proposed FRP model. The modified IEEE 39-bus, 57-bus, and 118-bus test systems were used to verify the effectiveness of the proposed model and solution method. The results demonstrate that the proposed FRP model can balance the retrofitting and operational costs to minimize the total cost and accommodate more wind generation. The proposed solution method is computationally efficient for solving FRP models.

Unit Commitment Problem with Energy Storage Under Correlated Renewables Uncertainty

Unit Commitment Problem with Energy Storage Under Correlated Renewables Uncertainty” introduces a novel approach to address the challenges of renewable integration. The study acknowledges the growing variability and correlation in power availability due to renewable generation and proposes a day-ahead unit commitment (UC) problem formulation that incorporates energy storage and considers multistage correlated uncertainty. Using a variant of the stochastic dual dynamic programming (SDDP) method, which can handle temporal correlations effectively, the researchers solve the complex UC problem. Results obtained from the IEEE 118-bus system demonstrate the significant advantages of considering multistage uncertainty and correlations. Applying their approach to the Chilean power system, the researchers present superior UC solutions that adapt generation to changing uncertainty at a lower cost. Additionally, they propose a more efficient deterministic UC solution that outperforms current industry practices. These advancements promise to enhance the integration of renewable energy sources, enabling a more sustainable and environmentally friendly future.

A multistage distributionally robust optimization approach for generation dispatch with demand response under endogenous and exogenous uncertainties

AbstractDecision‐dependent (endogenous) uncertainties (DDUs), as a new type of uncertainties revealed recently, couple dispatch decisions with uncertainty parameters and thus render power system dispatch more challenging. However, most previous works handled various DDUs via stochastic programming (SP) or robust optimization (RO) in a two‐stage framework, which undoubtedly introduces the drawbacks of SP and RO, and cannot meet the nonanticipativity requirements in power scheduling. In this paper, a multistage distributionally robust optimization (DRO) method for generation dispatch with demand response (DR) is proposed considering the DDUs of deferrable loads and the decision‐independent (exogenous) uncertainties (DIUs) of wind power and regular loads. By analyzing the structure of decision‐dependency parameters, a novel data‐driven decision‐dependent ambiguity set is proposed, which provides a generic framework for formulating DDUs and DIUs simultaneously. Then a multistage DRO model with nested max‐min structure is developed to integrate the merits of DRO and nonanticipativity into generation dispatch. The proposed model is solved by tailored reformulation method and improved stochastic dual dynamic integer programming (SDDiP). Case studies illustrate the effectiveness of the proposed approach by comparing with the multistage SP, RO, and decision‐independent DRO methods.

Multi‐stage stochastic dual dynamic programming to low‐carbon economic dispatch for power systems with flexible carbon capture and storage devices

AbstractThe carbon capture and storage (CCS) technique gains much attention due to its role in reducing CO2 emissions. By introducing flexible CCS devices into conventional power plants, the low‐carbon economic operation of the power system can be achieved. However, the uncertainty of renewable energy makes it hard to obtain an optimal operation policy. First, the detailed model of CCS consisting of bypass venting stacks, solvent storage tanks, absorber, and stripper is described. Then, a low‐carbon economic dispatch model of the power system with CCS is proposed to minimize the total operation and renewable energy abandonment penalty costs. Next, the problem is reformulated as multi‐stage stochastic programming, and the stochastic dual dynamic programming (SDDP) method is applied to relieve the computation burden. To control the probability of stopping the algorithm prematurely, a new stopping criterion based on a two‐sided hypothesis test is proposed. Finally, the effectiveness of the proposed model and the new stopping criterion is demonstrated by case studies on a practical power system.

Optimal portfolio choice of couples with tax-deferred accounts and survival-contingent products

Financial products for retirement planning generally have complex taxation structures and death conditions. In particular, tax-deferred accounts (TDAs) can provide tax-sheltered wealth accumulation by deferring taxes, even with the same financial products. Additionally, various survival-contingent products (SCPs), such as annuity products and life insurance contracts, have different payouts for policyholders. In this study, considering both the TDA and SCPs, we formulate and solve a couple’s lifetime portfolio choice problem using a multistage stochastic programming model. Owing to its high-dimensional state space and lifelong planning periods, stochastic dual dynamic programming (SDDP) was used to solve this problem. We find some interesting results; when both the TDA and SCPs are available, the portfolio is less concentrated in annuity holdings than when the TDA is unavailable. Moreover, the couple ends their contribution to the TA earlier than when SCPs are unavailable.

Medium-Term Hydrothermal Scheduling of the Infiernillo Reservoir Using Stochastic Dual Dynamic Programming (SDDP): A Case Study in Mexico

This article aims to obtain and evaluate medium-term operating policies for the hydrothermal scheduling problem by using the stochastic dual dynamic programming (SDDP) approach. To this end, to feed the mathematical model and build the probability distribution functions that best fit each month of the actual inflow volume, monthly inflow data recorded from 1938 to 2018 for the Infiernillo reservoir located in Mexico were employed. Moreover, we simulated inflow volume scenarios using the Monte Carlo method for each month of a one-year planning period. The SDDP approach to solving the optimization problem consisted of the simulation of one forward scenario per iteration and the stabilization of the total operating cost as a convergence criterion, which results in an operating policy. We then assessed its quality by estimating the one-sided optimality gap. It is worth mentioning that the best operation policy required scenario trees of up to 17,000 inflow realizations per stage. Additionally, to study the evolution of the expected value along the planning horizon of the main variables involved in the medium-term hydrothermal scheduling problem, we simulated the best operation policy over 10,000 inflow scenarios. Finally, to show the practical value of the proposed approach, we report its computational complexity.

A stochastic policy algorithm for seasonal hydropower planning

AbstractHydropower producers need to plan several months or years ahead to estimate the opportunity value of water stored in their reservoirs. The resulting large-scale optimization problem is computationally intensive, and model simplifications are often needed to allow for efficient solving. Alternatively, one can look for near-optimal policies using heuristics that can tackle non-convexities in the production function and a wide range of modelling approaches for the price- and inflow dynamics. We undertake an extensive numerical comparison between the state-of-the-art algorithm stochastic dual dynamic programming (SDDP) and rolling forecast-based algorithms, including a novel algorithm that we develop in this paper. We name it Scenario-based Two-stage ReOptimization abbreviated as STRO. The numerical experiments are based on convex stochastic dynamic programs with discretized exogenous state space, which makes the SDDP algorithm applicable for comparisons. We demonstrate that our algorithm can handle inflow risk better than traditional forecast-based algorithms, by reducing the optimality gap from 2.5 to 1.3% compared to the SDDP bound.

Multistage stochastic fractionated intensity modulated radiation therapy planning

Intensity modulated radiation therapy (IMRT) is a widely used cancer treatment technique designed to target malignant cells. To enhance its effectiveness on tumors and reduce side effects, radiotherapy plans are usually divided into consecutive treatments, or fractions, that are delivered over multiple weeks. However, typical planning approaches have focused on finding the full sequence of radiation intensities prior to the treatment, or were restricted to a single treatment session. In this work, we investigate the fractionated IMRT planning problem that accounts for geometric motion-related uncertainty during treatment. We propose a novel multistage stochastic programming (MSP) modeling framework that incorporates the sequential decision-making nature of the problem and prevailing stochasticity in cancer treatment. We model the uncertainty in the form of a scenario tree following the multiple instance of geometry approximation, and solve the MSP model using stochastic dual dynamic programming considering a variety of risk measures. We conduct computational experiments on five test cases that are generated based on clinical data. Through extensive simulations, we show that our MSP model generates higher quality treatment plans compared to deterministic and two-stage program counterparts based on multiple performance measures. In particular, our model leads to a higher rate of tumor coverage and a lower rate of radiation exposure for healthy tissues. Accordingly, the proposed MSP framework can greatly contribute to the clinical practice in fractionated IMRT.

Large-scale financial planning via a partially observable stochastic dual dynamic programming framework

The multi-stage stochastic programming (MSP) approach is widely used to solve financial planning problems owing to its flexibility. However, the size of an MSP problem grows exponentially with the number of stages, and such problem can easily become computationally intractable. Financial planning problems often consider planning horizons of several decades, and thus, the curse of dimensionality can become a critical issue. Stochastic dual dynamic programming (SDDP), a sampling-based decomposition algorithm, has emerged to resolve this issue. While SDDP has been successfully implemented in the energy domain, few applications of SDDP are found in the finance domain. In this study, we identify the major obstacle in using SDDP to solve financial planning problems to be the stagewise independence assumption and propose a partially observable SDDP (PO-SDDP) framework to overcome such limitations. We argue that the PO-SDDP framework, which models uncertainties using discrete-valued partially observable Markov states and introduces feasibility cuts, can properly address large-scale financial planning problems.

Spatiotemporal decomposed dispatch of integrated electricity-gas system via stochastic dual dynamic programming-based value function approximation

This paper proposes a novel stochastic dual dynamic programming-based value function approximation approach for the spatiotemporal decomposed dispatch of integrated electricity-gas systems with uncertainties. Stochastic dual dynamic programming is employed to decompose the optimal dispatch problem into several subproblems in both spatial and temporal dimensions. Then, Benders cuts are used in the real-time dispatch process to describe the interaction among these subproblems, according to which each subsystem can make decentralized and real-time decisions. In this way, both the information privacy and decision independence of each subsystem can be guaranteed, while the future variability of wind power and loads can be firmed. Moreover, the historical information can be used to obtain the Benders cuts offline, which helps to omit the time-consuming iterations in the real-time decision stage, while the near-optimal solutions can still be obtained. The effectiveness of the proposed approach is verified by case studies on a 4-bus-4-node system and a 118-bus-20-node system. Numerical results demonstrate that the proposed method reduces the average error of the operation cost by 1 order of magnitude compared with the traditional real-time methods. Moreover, the computational time is significantly reduced by up to 4 orders of magnitude compared with the traditional decentralized algorithms.

Medium-term multi-stage distributionally robust scheduling of hydro–wind–solar complementary systems in electricity markets considering multiple time-scale uncertainties

Joint trading of hydro–wind–solar complementary systems (HWSCSs) in the electricity market (EM) helps to reduce the imbalance cost and increase profits. However, multiple energy resources and market price uncertainties affect the trading strategies. Existing medium-term (MT) scheduling approaches assume that the probability distribution of the random variable is perfectly known. Short-term variations were also ignored, which led to revenue loss and trading risk. To address the above issues, this paper proposes an MT multi-stage distributionally robust optimization (MDRO) scheduling approach for a price-taking HWSCS in the EM. Firstly, hourly unit commitment (HUC) constraints are incorporated into the MT scheduling model to accurately capture short-term variations. A novel ambiguity set is designed based on the modified chi-square distance to address probability distribution uncertainties at two different time scales. Subsequently, an MDRO scheduling model is proposed to optimize the trading strategy. Finally, the proposed MDRO model is converted to a large-scale multi-stage integer programming problem based on linearization and reformation. The stochastic dual dynamic integer programming algorithm is modified to ensure computational tractability. Xiluodu-Xiangjiaba HWSCS, located in the Jinsha River in China, was selected as a case study. The results show that: 1) the MDRO model is more robust to distributional uncertainties than the multi-stage stochastic programming (MSSP) model. When the probability distribution of the random variable changes, the MDRO model yields a higher expected revenue (+2.43%) and a lower standard deviation (-60.8%) of revenue, which illustrates lower trading risk. 2) Compared with MSSP, deterministic, two-stage stochastic programming, and distributionally robust optimization models, the MDRO model exhibits the best out-of-sample performance in terms of the highest expected revenue and lowest trading risk. 3) Incorporating HUC constraints into the MDRO model helps to increase the total revenue (+3.53%) and energy generation (+3.31%) at the expense of increasing the computational burden.

Multi‐stage stochastic long‐term planning of grid‐connected hydrogen‐based energy system based on improved SDDIP

AbstractHydrogen‐based energy systems (HESs) have shown great potential to promote the process of decarbonization. Conventional studies mainly focus on the sizing and operation of HESs in a determined static situation, and the dynamic planning model of HESs considering large‐scale uncertain scenarios of future developments should be considered. This paper proposes a multi‐stage stochastic programming (MSP) long‐term planning model to find the optimal sequential planning results of the grid‐connected HES. The planning model considers the long‐term uncertainties of the investment cost decrease and the load increase. Additionally, the short‐term uncertainties of renewable energies are also considered to obtain robust results in each stage. The improved stochastic dual dynamic integer programming (SDDIP) is then employed to solve the MSP long‐term planning model with consideration of the realized uncertainties. Specifically, the sequential planning order is developed to improve the efficiency of the SDDIP. Numerical case studies are constructed to show the convergence process of the improved SDDIP and the planning results of the HES. Moreover, the improved SDDIP shows greater efficiency compared with the traditional SDDIP and the method which solves the model directly.

Combining monthly wind and inflow uncertainties in the stochastic dual dynamic programming

AbstractFollowing a global trend, intermittent sources, especially wind, have been experiencing accelerated growth in Brazil—in the last decade, wind power grew 13 times and became the second largest source in the electricity mix (12%), just behind hydropower (60%). Currently, although following regulatory guidelines, the representation of wind power in the long-term operation planning model is done in a simplified way, based on the monthly average of the last five years of aggregated generation, thus demanding improvements. The objective of this work is to describe an approach to be used by the Brazilian power industry to represent the uncertainties of monthly wind power production in the SDDP algorithm applied in the long-term operation planning model, keeping the large-scale stochastic problem still computationally viable, when applied to large interconnected systems, especially with hydroelectric predominance. The proposed methodology comprises statistical clustering of wind regimes and definition of equivalent wind farms; evaluation of monthly transfer functions between wind speed and power production; integrated generation of monthly multivariate synthetic scenarios of inflows and winds, considering associated cross-correlations; and representing monthly wind power in the SDDP algorithm. The application to real configurations of the Brazilian interconnected system, including case studies related to the monthly operation program and the calculation of the maximum amount of energy that can be traded in long-term power purchase agreements, points to its effectiveness and the relevance of modeling the wind uncertainties in the long-term operation planning of large hydro-dominated systems.

Optimal scheduling of the energy storage system in a hybrid micro‐grid considering uncertainties, using the stochastic quasi‐gradient method

AbstractEnergy storage and renewable sources play a unique role in the future advances of smart grids. In this article, the optimal scheduling of the energy storage system in a hybrid microgrid is presented considering the uncertainties of the renewable generations and the load. The optimisation problem in this article is non‐linear and non‐convex, therefore conventional optimisation methods such as linear programming (LP) are unable to solve this problem. On the other hand, because of parameters uncertainty, special considerations are required to simulate these parameters. In this regard, a new optimisation algorithm that can solve the non‐linearity and non‐convexity of the objective function is proposed based on the Stochastic Quasi‐Gradient optimisation Method (SQGM). Moreover, the uncertainties of the wind, PV generation, and the load are modelled. Different optimisation algorithms: the conventional Stochastic Dynamic Programming (SDP), the Stochastic Dual Dynamic Programming (SDDP) and the proposed SQGM are compared. A 9‐bus benchmark system with distributed generation units is used to evaluate the optimisation strategies.

A Nonconvex Regularization Scheme for the Stochastic Dual Dynamic Programming Algorithm

We propose a new nonconvex regularization scheme to improve the performance of the stochastic dual dynamic programming (SDDP) algorithm for solving large-scale multistage stochastic programs. Specifically, we use a class of nonconvex regularization functions, namely folded concave penalty functions, to improve solution quality and the convergence rate of the SDDP procedure. We develop a strategy based on mixed-integer programming to guarantee global optimality of the nonconvex regularization problem. Moreover, we establish provable convergence guarantees for our customized SDDP algorithm. The benefits of our regularization scheme are demonstrated by solving large-scale instances of two multistage stochastic optimization problems. History: Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2021.0255 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0255 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .