Stochastic Dual Dynamic Programming Algorithm Research Articles

Time-consistent risk-constrained dynamic portfolio optimization with transactional costs and time-dependent returns

Dynamic portfolio optimization has a vast literature exploring different simplifications by virtue of computational tractability of the problem. Previous works provide solution methods considering unrealistic assumptions, such as no transactional costs, small number of assets, specific choices of utility functions and oversimplified price dynamics. Other more realistic strategies use heuristic solution approaches to obtain suitable investment policies. In this work, we propose a time-consistent risk-constrained dynamic portfolio optimization model with transactional costs and Markovian time-dependence. The proposed model is efficiently solved using a Markov chained stochastic dual dynamic programming algorithm. We impose one-period conditional value-at-risk constraints, arguing that it is reasonable to assume that an investor knows how much he is willing to lose in a given period. In contrast to dynamic risk measures as the objective function, our time-consistent model has relatively complete recourse and a straightforward lower bound, considering a maximization problem. We use the proposed model for approximately solving: (i) an illustrative problem with 3 assets and 1 factor with an autoregressive dynamic; (ii) a high-dimensional problem with 100 assets and 5 factors following a discrete Markov chain. In both cases, we empirically show that our approximate solution is near-optimal for the original problem and significantly outperforms selected (heuristic) benchmarks. To the best of our knowledge, this is the first systematic approach for solving realistic time-consistent risk-constrained dynamic asset allocation problems.

Assessing solution quality and computational performance in the long-term generation scheduling problem considering different hydro production function approaches

Abstract The long-term generation scheduling (LTGS) problem aims at finding a generation policy that minimizes an objective function over a multi-year planning horizon. A crucial aspect of this problem is the Hydropower Production Function (HPF), which relates power with head, turbined outflow, and efficiency of the generating units. Given that the LTGS is a large-scale stochastic optimization problem, the HPF is modeled in a simplified manner. However, considering the high-performance computers currently available and the recent advances in stochastic optimization algorithms, it is possible to enhance the HPF modeling to use the energy resources more efficiently. This paper proposes a piecewise linear model of HPF that considers the plant generation as a function of the volume and the total outflow. Unlike previous works, the HPF also considers the (nonlinear) efficiency function of each generating unit. The paper also presents a comparison between the proposed HPF and a one-dimensional HPF known as constant productivity. The generation policy and the computational burden are analyzed using an optimization-simulation process based on Stochastic Dual Dynamic Programming algorithm. The computational tests use data of a large-scale electrical power system, which corresponds to about 90% of the Brazilian system.

Optimization of Hydropower Dam Design Parameters for Adaptation to Climate Change using stochastic dual dynamic programming algorithm (SDDP)

Optimization of Hydropower Dam Design Parameters for Adaptation to Climate Change using stochastic dual dynamic programming algorithm (SDDP)

Optimization of Hydropower Dam Design Parameters for Adaptation to Climate Change using stochastic dual dynamic programming algorithm (SDDP)

Optimization of Hydropower Dam Design Parameters for Adaptation to Climate Change using stochastic dual dynamic programming algorithm (SDDP)

Distributionally robust SDDP

We study a version of stochastic dual dynamic programming (SDDP) with a distributionally robust objective. The classical SDDP algorithm uses a finite (nominal) probability distribution for the random outcomes at each stage. We modify this by defining a distributional uncertainty set in each stage to be a Euclidean neighbourhood of the nominal probability distribution. We derive a formula for the worst-case expectation of future costs over this set that can be applied in the backward pass of SDDP. We verify the correctness of this algorithm, show its almost sure convergence under standard assumptions, and illustrate it by applying it to a model of the New Zealand hydrothermal electricity system.

Multistage Stochastic Programs with a Random Number of Stages: Dynamic Programming Equations, Solution Methods, and Application to Portfolio Selection

We introduce the class of multistage stochastic optimization problems with a random number of stages. For such problems, we show how to write dynamic programming equations and detail the Stochastic Dual Dynamic Programming algorithm to solve these equations. Finally, we consider a portfolio selection problem over an optimization period of random duration. For several instances of this problem, we show the gain obtained using of a policy that takes the random duration of the number of stages into account over a policy built taking a fixed number of stages (namely the maximal possible number of stages).

Managing Energy Storage in Microgrids: A Multistage Stochastic Programming Approach

A microgrid is a small-scale version of a centralized power grid that generates, distributes and regulates electricity flow to local entities using distributed generation and the main grid. Distributed energy storage systems can be used to mitigate adverse effects of intermittent renewable sources in a microgrid in which operators dynamically adjust electricity procurement and storage decisions in response to randomly-evolving demand, renewable supply and pricing information. We formulate a multistage stochastic programming (SP) model whose objective is to minimize the expected total energy costs incurred within a microgrid over a finite planning horizon. The model prescribes the amount of energy to procure, store and discharge in each decision stage of the horizon. However, for even a moderate number of stages, the model is computationally intractable; therefore, we customize the stochastic dual dynamic programming (SDDP) algorithm to obtain high-quality approximate solutions. Computation times and optimization gaps are significantly reduced by implementing a dynamic cut selection procedure and a lower bound improvement scheme within the SDDP framework. An extensive computational study reveals significant cost savings as compared to myopic and non-storage policies, as well as policies obtained using a two-stage SP model. The study also demonstrates the scalability of our solution procedure.

Co-Optimization of Energy and Ancillary Services for Hydrothermal Operation Planning Under a General Security Criterion

One of the most used methods for long-term hydrothermal operation planning is the stochastic dual dynamic programming (SDDP). Using this method, the immediate and future water opportunity cost can be balanced and an economic-dispatch policy can be defined for multiple reservoirs under inflow uncertainty. In this framework, equipment outages and reserve deliverability are generally disregarded, despite their strong impact on the operative plan. However, recent advances in robust optimization have shown how to endogenously account for security criteria in scheduling models with reduced computational burden. Within this framework, reserve deliverability is ensured across the network via the co-optimization of energy and reserves (ancillary services). In this paper, we propose a new multistage model for planning hydrothermal coordination that co-optimizes the nominal energy dispatch and individual up and down reserve allocations. The main goal of this paper is to address a general $n-K$ security criterion, such that, for each inflow scenario, the system is capable of withstanding the loss of up to $K$ components, i.e., generation or transmission assets. The proposed methodology uses the column-and-constraint generation algorithm to efficiently incorporate a compound umbrella set of contingencies in the SDDP algorithm. Results for the Brazilian power system data corroborate the effectiveness of the proposed model.

Assessing the Cost of Time-Inconsistent Operation Policies in Hydrothermal Power Systems

The current state-of-the-art method used for medium- and long-term planning studies of hydrothermal power system operation is the stochastic dual dynamic programming (SDDP) algorithm. The computational savings provided by this method notwithstanding, it still relies on major system simplifications to achieve acceptable performances in practical applications. In contrast with its actual implementation, simplifications in the planning stage may induce time-inconsistent policies, and consequently, a suboptimality gap. In this paper, we extend the concept of time inconsistency to measure the effects of modeling simplifications in the SDDP framework for hydrothermal operation planning. Case studies involving simplifications in transmission lines modeling and in security criteria indicate that these source of time inconsistency may result in unexpected reservoir depletion and spikes in energy market spot prices.

Improving operating policies of large‐scale surface‐groundwater systems through stochastic programming

AbstractThe management of large‐scale water resource systems with surface and groundwater resources requires considering stream‐aquifer interactions. Optimization models applied to large‐scale systems have either employed deterministic optimization (with perfect foreknowledge of future inflows, which hinders their applicability to real‐life operations) or stochastic programming (in which stream‐aquifer interaction is often neglected due to the computational burden associated with these methods). In this paper, stream‐aquifer interaction is integrated in a stochastic programming framework by combining the Stochastic Dual Dynamic Programming (SDDP) optimization algorithm with the Embedded Multireservoir Model (EMM). The resulting extension of the SDDP algorithm, named Combined Surface‐Groundwater SDDP (CSG‐SDDP), is able to properly represent the stream‐aquifer interaction within stochastic optimization models of large‐scale surface‐groundwater resource systems. The algorithm is applied to build a hydroeconomic model for the Jucar River Basin (Spain), in which stream‐aquifer interactions are essential to the characterization of water resources. Besides the uncertainties regarding the economic characterization of the demand functions, the results show that the economic efficiency of the operating policies under the current system can be improved by better management of groundwater and surface resources.

Regularized Decomposition Methods for Deterministic and Stochastic Convex Optimization and Application to Portfolio Selection with Direct Transaction and Market Impact Costs

We define a regularized variant of the Dual Dynamic Programming algorithm called REDDP (REgularized Dual Dynamic Programming) to solve nonlinear dynamic programming equations. We extend the algorithm to solve nonlinear stochastic dynamic programming equations. The corresponding algorithm, called SDDP-REG, can be seen as an extension of a regularization of the Stochastic Dual Dynamic Programming (SDDP) algorithm recently introduced which was studied for linear problems only and with less general prox-centers. We show the convergence of REDDP and SDDP-REG. We assess the performance of REDDP and SDDP-REG on portfolio models with direct transaction and market impact costs. In particular, we propose a risk-neutral portfolio selection model which can be cast as a multistage stochastic second-order cone program. The formulation is motivated by the impact of market impact costs on large portfolio rebalancing operations. Numerical simulations show that REDDP is much quicker than DDP on all problem instances considered (up to 184 times quicker than DDP) and that SDDP-REG is quicker on the instances of portfolio selection problems with market impact costs tested and much faster on the instance of risk-neutral multistage stochastic linear program implemented (8.2 times faster).

Spatio-temporal hydro forecasting of multireservoir inflows for hydro-thermal scheduling

Hydro-thermal scheduling is the problem of finding an optimal dispatch of power plants in a system containing both hydro and thermal plants. Since hydro plants are able to store water over long time periods, and since future inflows are uncertain due to precipitation, the resulting multi-stage stochastic optimization problem becomes challenging to solve. Several solution methods have been developed over the past few decades to compute practically useful operation policies. One of these methods is stochastic dual dynamic programming (SDDP). SDDP poses strong restrictions on the forecasting method generating the necessary inflow scenarios. In this context, the current state-of-the-art in forecasting are periodic autoregressive (PAR) models. We present a new forecasting model for hydro inflows that incorporates spatial information, i.e., inflow information from neighboring reservoirs of the system, and that also satisfies the restrictions posed by SDDP. We benchmark our model against a PAR model that is similar to the one currently used in Brazil. Three multi-reservoir basins in Brazil serve as a case study for the comparison. We show that our approach outperforms the benchmark PAR model and present the root mean squared error (RMSE) as well as the seasonally-adjusted coefficient of efficiency (SACE) for each reservoir modeled. The overall decrease in RMSE is 8.29 percent using our approach for one month-ahead forecasts. The decrease in RMSE is achieved without additional data collection while only adding 11.8 percent more state variables for the SDDP algorithm.

Assessing policy quality in a multistage stochastic program for long-term hydrothermal scheduling

We consider a multistage stochastic linear program in which we aim to assess the quality of an operational policy computed by means of a stochastic dual dynamic programming algorithm. We perform policy assessment by considering two strategies to compute a confidence interval on the optimality gap: (i) using multiple scenario trees and (ii) using a single scenario tree. The first approach has already been considered in several applications, while the second approach has been discussed previously only in a two-stage framework. The second approach is useful in practical applications in order to more quickly assess the quality of a policy. We present these ideas in the context of a multistage stochastic program for Brazilian long-term hydrothermal scheduling, and use numerical instances to compare the confidence intervals on the optimality gap computed via both strategies. We further consider the relative merits of using naive Monte Carlo sampling, randomized quasi Monte Carlo sampling, and Latin hypercube sampling within our framework for assessing the quality of a policy.

Efficient Parallelization of the Stochastic Dual Dynamic Programming Algorithm Applied to Hydropower Scheduling

Stochastic dual dynamic programming (SDDP) has become a popular algorithm used in practical long-term scheduling of hydropower systems. The SDDP algorithm is computationally demanding, but can be designed to take advantage of parallel processing. This paper presents a novel parallel scheme for the SDDP algorithm, where the stage-wise synchronization point traditionally used in the backward iteration of the SDDP algorithm is partially relaxed. The proposed scheme was tested on a realistic model of a Norwegian water course, proving that the synchronization point relaxation significantly improves parallel efficiency.

Improving the performance of Stochastic Dual Dynamic Programming

This paper is concerned with tuning the Stochastic Dual Dynamic Programming algorithm to make it more computationally efficient. We report the results of some computational experiments on a large-scale hydrothermal scheduling model developed for Brazil. We find that the best improvements in computation time are obtained from an implementation that increases the number of scenarios in the forward pass with each iteration and selects cuts to be included in the stage problems in each iteration. This gives an order of magnitude decrease in computation time with little change in solution quality.

Stochastic inflow modeling for hydropower scheduling problems

We introduce a new stochastic model for inflow time series that is designed with the requirements of hydropower scheduling problems in mind. The model is an “iterated function system’’: it models inflow as continuous, but the random innovation at each time step has a discrete distribution. With this inflow model, hydro-scheduling problems can be solved by the stochastic dual dynamic programming (SDDP) algorithm exactly as posed, without the additional sampling error introduced by sample average approximations. The model is fitted to univariate inflow time series by quantile regression. We consider various goodness-of-fit metrics for the new model and some alternatives to it, including performance in an actual hydro-scheduling problem. The numerical data used are for inflows to New Zealand hydropower reservoirs.

Combining sampling-based and scenario-based nested Benders decomposition methods: application to stochastic dual dynamic programming

Nested Benders decomposition is a widely used and accepted solution methodology for multi-stage stochastic linear programming problems. Motivated by large-scale applications in the context of hydro-thermal scheduling, in 1991, Pereira and Pinto introduced a sampling-based variant of the Benders decomposition method, known as stochastic dual dynamic programming (SDDP). In this paper, we embed the SDDP algorithm into the scenario tree framework, essentially combining the nested Benders decomposition method on trees with the sampling procedure of SDDP. This allows for the incorporation of different types of uncertainties in multi-stage stochastic optimization while still maintaining an efficient solution algorithm. We provide an illustration of the applicability of our method towards a least-cost hydro-thermal scheduling problem by examining an illustrative example combining both fuel cost with inflow uncertainty and by studying the Panama power system incorporating both electricity demand and inflow uncertainties.

On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs

We prove the almost-sure convergence of a class of sampling-based nested decomposition algorithms for multistage stochastic convex programs in which the stage costs are general convex functions of the decisions and uncertainty is modelled by a scenario tree. As special cases, our results imply the almost-sure convergence of stochastic dual dynamic programming, cutting-plane and partial-sampling (CUPPS) algorithm, and dynamic outer-approximation sampling algorithms when applied to problems with general convex cost functions.

On variance reduction of mean-CVaR Monte Carlo estimators

We formulate an objective as a convex combination of expectation and risk, measured by the \(\mathrm{CVaR }\) risk measure. The poor performance of standard Monte Carlo estimators applied on functions of this form is discussed and a variance reduction scheme based on importance sampling is proposed. We provide analytical solution for random variables based on normal distribution and outline the way for the other distributions, either by analytical computation or by sampling. Our results are applied in the framework of stochastic dual dynamic programming algorithm. Computational results which validate the previous analysis are given.

Structure of risk-averse multistage stochastic programs

We deal with risk-averse multistage stochastic programs with coherent risk measures such as multiperiod extensions of conditional value at risk or polyhedral risk measures. Their basic properties are discussed and applied to scenario-based input data. Using the contamination technique we quantify the influence of changes in the structure of the scenario-based approximation to the optimal value of the problem. Stochastic dual dynamic programming algorithm is used to provide illustrative numerical comparisons for different choices of risk measures and changes of input data for a simple multistage risk-averse stock allocation problem with scenario trees based on log-normal distribution of the asset returns.