Stochastic Dual Dynamic Programming Algorithm Research Articles

Combining Polyhedral Approaches and Stochastic Dual Dynamic Integer Programming for Solving the Uncapacitated Lot-Sizing Problem Under Uncertainty

We study the uncapacitated lot-sizing problem with uncertain demand and costs. The problem is modeled as a multistage stochastic mixed-integer linear program in which the evolution of the uncertain parameters is represented by a scenario tree. To solve this problem, we propose a new extension of the stochastic dual dynamic integer programming algorithm (SDDiP). This extension aims at being more computationally efficient in the management of the expected cost-to-go functions involved in the model, in particular by reducing their number and by exploiting the current knowledge on the polyhedral structure of the stochastic uncapacitated lot-sizing problem. The algorithm is based on a partial decomposition of the problem into a set of stochastic subproblems, each one involving a subset of nodes forming a subtree of the initial scenario tree. We then introduce a cutting plane–generation procedure that iteratively strengthens the linear relaxation of these subproblems and enables the generation of an additional strengthened Benders’ cut, which improves the convergence of the method. We carry out extensive computational experiments on randomly generated large-size instances. Our numerical results show that the proposed algorithm significantly outperforms the SDDiP algorithm at providing good-quality solutions within the computation time limit. Summary of Contribution: This paper investigates a combinatorial optimization problem called the uncapacitated lot-sizing problem. This problem has been widely studied in the operations research literature as it appears as a core subproblem in many industrial production planning problems. We consider a stochastic extension in which the input parameters are subject to uncertainty and model the resulting stochastic optimization problem as a multistage stochastic integer program. To solve this stochastic problem, we propose a novel extension of the recently published stochastic dual dynamic integer programming (SDDiP) algorithm. The proposed extension relies on two main ideas: the use of a partial decomposition of the scenario tree and the exploitation of existing knowledge on the polyhedral structure of the stochastic uncapacitated lot-sizing problem. We provide the results of extensive computational experiments carried out on large-size randomly generated instances. These results show that the proposed extended algorithm significantly outperforms the SDDiP at providing good-quality solutions for the stochastic uncapacitated lot-sizing problem. Although the paper focuses on a basic lot-sizing problem, the proposed algorithmic framework may be useful to solve more complex practical production planning problems.

Parallel and distributed computing for stochastic dual dynamic programming

We study different parallelization schemes for the stochastic dual dynamic programming (SDDP) algorithm. We propose a taxonomy for these parallel algorithms, which is based on the concept of parallelizing by scenario and parallelizing by node of the underlying stochastic process. We develop a synchronous and asynchronous version for each configuration. The parallelization strategy in the parallelscenario configuration aims at parallelizing the Monte Carlo sampling procedure in the forward pass of the SDDP algorithm, and thus generates a large number of supporting hyperplanes in parallel. On the other hand, the parallel-node strategy aims at building a single hyperplane of the dynamic programming value function in parallel. The considered algorithms are implemented using Julia and JuMP on a high performance computing cluster. We study the effectiveness of the methods in terms of achieving tight optimality gaps, as well as the scalability properties of the algorithms with respect to an increasing number of CPUs. In particular, we study the effects of the different parallelization strategies on performance when increasing the number of Monte Carlo samples in the forward pass, and demonstrate through numerical experiments that such an increase may be harmful. Our results indicate that a parallel-node strategy presents certain benefits as compared to a parallel-scenario configuration.

Mature offshore oil field development: Solving a real options problem using stochastic dual dynamic integer programming

Oil and gas companies are facing low output prices and are forced to focus on the development of mature fields. Relevant investment decisions for operators include lifetime-enhancing activities, such as drilling new wells or permanent shutdown. We study the problem of optimal timing of investments in mature oil and gas fields in the presence of price uncertainty, which is an example of a complex real options problem consisting of a portfolio of interdependent options. We formulate a multistage stochastic integer programming model that incorporates a detailed representation of the uncertain oil price, and demonstrate how such a complex real options problem can be efficiently solved using the Stochastic Dual Dynamic Integer Programming algorithm. The paper presents a numerical example based on realistic data and discusses our computational results. We find that only a small number of Markov states are required to represent the uncertain price process, while obtaining convergence of the lower and upper bounds of the objective function. The value of stochastic solution of 11% is considerable in this example. It is concluded that the shutdown decision tends to be postponed as a result of high decommissioning costs, high discount rates, high price uncertainty and low operational expenditures, while it generally is accelerated if the decommissioning costs increase over time.

A Stochastic Dual Dynamic Integer Programming for the Uncapacitated Lot-Sizing Problem with Uncertain Demand and Costs

We study the uncapacitated lot-sizing problem with uncertain demand and costs. We consider a multi-stage decision process and rely on a scenario tree to represent the uncertainty. We propose to solve this stochastic combinatorial optimization problem thanks to a new extension of the stochastic dual dynamic integer programming algorithm. Our results show that this approach can provide good quality solutions in a reasonable time for large-size instances.

Stochastic Dynamic Cutting Plane for Multistage Stochastic Convex Programs

We introduce Stochastic Dynamic Cutting Plane (StoDCuP), an extension of the Stochastic Dual Dynamic Programming (SDDP) algorithm to solve multistage stochastic convex optimization problems. At each iteration, the algorithm builds lower bounding affine functions not only for the cost-to-go functions, as SDDP does, but also for some or all nonlinear cost and constraint functions. We show the almost sure convergence of StoDCuP. We also introduce an inexact variant of StoDCuP where all subproblems are solved approximately (with bounded errors) and show the almost sure convergence of this variant for vanishing errors. Finally, numerical experiments are presented on nondifferentiable multistage stochastic programs where Inexact StoDCuP computes a good approximate policy quicker than StoDCuP while SDDP and the previous inexact variant of SDDP combined with Mosek library to solve subproblems were not able to solve the differentiable reformulation of the problem.

Global warming impact to River Basin of Blue Nile and the optimum operation of its multi-reservoir system for hydropower production and irrigation

The water resource of the Blue Nile River basin (BNRB) has been under pressure due to growing demands from many users, and the climate change impact. Potential impact of climate change for the maximum, median and minimum projected changes in the simulated streamflow of BNRB by a hydrologic model, VIC, driven by Representative Concentration Pathways climate scenarios, RCP4.5 and RCP8.5, of 4 GCMs (global climate models) downscaled dynamically by a regional climate model, WRF (Weather Research Forecasting) using a one-domain framework that covers the entire NRB for 2041–2070 and 2071–2100. These projected changes in streamflow were used to assess its future water allocations using a stochastic Dual Dynamic Programming (SDDP) algorithm and a hydro-economic model to optimize hydropower production and irrigated agriculture. Overall, it seems the Grand Ethiopian Renaissance Dam (GERD) reservoir will likely not operate at full storage level because the streamflow of BNRB is assumed to be regulated by three upstream reservoirs. The outflow from the reservoir of GERD or BNRB's annual flow at Khartoum is projected to increase under maximum, but is expected to decrease under minimum and median projected changes in streamflow for 2041–2070 and 2071–2100, respectively. Given the annual net benefit obtained from hydropower production and irrigated agriculture of the reservoir is projected to increase (decrease) under the maximum (median and minimum) projected changes in streamflow, the potential climate change impact should be considered in designing and developing the future water resources of BNRB.

Experimental Study of Methods of Scenario Lattice Construction for Stochastic Dual Dynamic Programming

The stochastic dual dynamic programming (SDDP) algorithm is becoming increasingly used. In this paper we present analysis of different methods of lattice construction for SDDP exemplifying a realistic variant of the newsvendor problem, incorporating storage of production. We model several days of work and compare the profits realized using different methods of the lattice construction and the corresponding computer time spent in lattice construction. Our case differs from the known one because we consider not only a multidimensional but also a multistage case with stage dependence. We construct scenario lattice for different Markov processes which play a crucial role in stochastic modeling. The novelty of our work is comparing different methods of scenario lattice construction. We considered a realistic variant of the newsvendor problem. The results presented in this article show that the Voronoi method slightly outperforms others, but the k-means method is much faster overall.

Two-stage and multi-stage decompositions for the medium-term hydrothermal scheduling problem: A computational comparison of solution techniques

In hydro-based power systems, the Generation Scheduling (GS) problem is usually divided into a series of coupled models aiming to obtain an optimal operating policy over a multi-year planning horizon. The medium-term GS (MTGS) problem belongs to these models, linking the long-term GS (LTGS) with the short-term GS (STGS) problem. To carry out a proper coupling, we propose a stochastic MTGS model with a multi-period scenario tree with non-common samples, and hourly discretization in the first week to include the nonconvex thermal unit-commitment constraints. These considerations lead to solving a very challenging large-scale stochastic optimization problem with mixed-binary variables in the first week. In this context, efficient solution techniques must be explored to ensure a suitable trade-off between solution accuracy and computational performance. In this paper, we propose a two-stage decomposition for the MTGS that is solved by the extended level bundle method. Every second-stage subproblem is itself a multi-period stochastic problem involving only continuous variables. Numerical assessments on the large-scale Brazilian MTGS problem indicate that our approach significantly outperforms the Nested decomposition and Stochastic Dual Dynamic Programming (SDDP) algorithm in terms of computational burden, providing CPU time reductions in the order of 19%.

Impact of operating uncertainty on the performance of distributed, hybrid, renewable power plants

This paper studies the performance of distributed, on-grid and off-grid, hybrid renewable power plants with short time scale operating uncertainty in solar irradiation and consumer demand. For the same, a multi-stage model using the Stochastic Dual Dynamic Programming algorithm with generation expansion is developed. The impact of operating uncertainty is quantified using this approach in two ways. First, operating and total costs for stochastically optimised hybrid plants are compared with equivalent, deterministically optimised hybrid plants. When accounted for in the generation expansion problem, operating uncertainty results in larger plant capacities in order to guarantee energy delivery and thus higher total costs. The value of having a dispatchable generator as part of the hybrid plant is then examined. This shows that hybrids that feature dispatchable generation have lower operating and total costs because the dispatchable generator can be relied upon to satisfy demand when either the intermittent generator or the energy storage is unable to do so. This highlights the robustness of hybrids that feature dispatchable generators and not just intermittent renewable plant and energy storage.

SDDP.jl: A Julia Package for Stochastic Dual Dynamic Programming

We present SDDP.jl, an open-source library for solving multistage stochastic programming problems using the stochastic dual dynamic programming algorithm. SDDP.jl is built on JuMP, an algebraic modeling language in Julia. JuMP provides SDDP.jl with a solver-agnostic, user-friendly interface. In addition, we leverage unique features of Julia, such as multiple dispatch, to provide an extensible framework for practitioners to build on our work. SDDP.jl is well tested, and accessible documentation is available at https://github.com/odow/SDDP.jl .

Improving the performance of the stochastic dual dynamic programming algorithm using Chebyshev centers

In hydro predominant systems, the long-term hydrothermal scheduling problem (LTHS) is formulated as a multistage stochastic programming model. A classical optimization technique to obtain an operational policy is the stochastic dual dynamic programming (SDDP), which employs a forward step, for generating trial state variables, and a backward step to construct Benders-like cuts. To assess the quality of the obtained policy (the cuts obtained over the iterations), a confidence interval is computed on the optimality gap. As the SDDP is a cutting-plane based method, it exhibits slow convergence in large-scale problems. To improve computational efficiency, we explore different regions in which the cuts are usually constructed by the classical algorithm. For that, the cuts in the forward step are translated using ideas related to the definition of Chebyshev centers of certain polyhedrons. Essentially, the cuts are lifted by a parameter that vanishes along with iterations without harming the convergence analysis. The proposed technique is assessed on an instance of the Brazilian LTHS problem with individualized monthly decisions per plant, indicating a higher policy quality in comparison with the classical approach, since it computes lower optimality gaps throughout the iterative process.

Multistage stochastic programs with a random number of stages: dynamic programming equations, solution methods, and application to portfolio selection

We introduce the class of multistage stochastic optimization problems with a random number of stages. For such problems, we show how to write dynamic programming equations and how to solve these equations using the Stochastic Dual Dynamic Programming algorithm. Finally, we consider a portfolio selection problem over an optimization period of random duration. For several instances of this problem, we show the gain obtained using a policy that takes the randomness of the number of stages into account over a policy built taking a fixed number of stages (namely the maximal possible number of stages).

Assessing the value of natural gas underground storage in the Brazilian system via stochastic dual dynamic programming

The Brazilian natural gas sector is currently characterized by low maturity and dynamism of the market. The stochastic behavior of the demand for natural gas added to its associated market price volatility motivates the usage of underground storage to provide supply flexibility and protection against price fluctuations. However, the existing literature lacks a proper analytical tool to assess the benefits of underground natural gas storage (UNGS) activity. In this work, it is proposed a stochastic dynamic programming model for long/medium-term operation planning to determine the optimal gas supply and storage policies. A markovian model characterizes the uncertainty over the thermoelectric demand and market price. The proposed model is efficiently solved using the stochastic dual dynamic programming algorithm for the Brazilian case study considering realistic data for the actual gas network and electric power system. For an exogenous but meaningful choice of underground storage location and size, it is observed the operational and economic benefits of the provided storage flexibility. Finally, our numerical simulations show that the economic benefit for the system surpasses the operational and capital expenses for the storage infrastructure in depleted fields and salt caverns.

Regularized stochastic dual dynamic programming for convex nonlinear optimization problems

We define a regularized variant of the dual dynamic programming algorithm called DDP-REG to solve nonlinear dynamic programming equations. We extend the algorithm to solve nonlinear stochastic dynamic programming equations. The corresponding algorithm, called SDDP-REG, can be seen as an extension of a regularization of the stochastic dual dynamic programming (SDDP) algorithm recently introduced which was studied for linear problems only and with less general prox-centers. We show the convergence of DDP-REG and SDDP-REG. We assess the performance of DDP-REG and SDDP-REG on portfolio models with direct transaction and market impact costs. In particular, we propose a risk-neutral portfolio selection model which can be cast as a multistage stochastic second-order cone program. The formulation is motivated by the impact of market impact costs on large portfolio rebalancing operations. Numerical simulations show that DDP-REG is much quicker than DDP on all problem instances considered (up to 184 times quicker than DDP) and that SDDP-REG is quicker on the instances of portfolio selection problems with market impact costs tested and much faster on the instance of risk-neutral multistage stochastic linear program implemented (8.2 times faster).

Real-Time Stochastic Optimization of Energy Storage Management Using Deep Learning-Based Forecasts for Residential PV Applications

A computationally proficient real-time energy management method with stochastic optimization is presented for a residential photovoltaic (PV)-storage hybrid system comprised of a solar PV generation and a battery energy storage (BES). Existing offline energy management approaches for day-ahead scheduling of BES suffer from energy loss in real time due to the stochastic nature of load and solar generation. On the other hand, typical online algorithms do not offer optimal solutions for minimizing electricity purchase costs to the owners. To overcome these limitations, we propose an integrated energy management framework consisting of an offline optimization model concurrent with a real-time rule-based controller. The optimization is performed in receding horizon with load and solar generation forecast profiles using deep learning-based long short term memory method in rolling horizon to reduce the daily electricity purchase costs. The optimization model is formulated as a multistage stochastic program where we use the stochastic dual dynamic programming algorithm in the receding horizon to update the optimal set point for BES dispatch at a fixed interval. To prevent loss of energy during optimal solution update intervals, we introduce a rule-based controller underneath the optimization layer in finer time resolution at the power electronics converter control level. The proposed framework is evaluated using a real-time controller-hardware-in-the-loop test platform in an OPAL-RT simulator. The proposed real-time method is effective in reducing the net electricity purchase cost compared to other existing energy management methods.

Exact Converging Bounds for Stochastic Dual Dynamic Programming via Fenchel Duality

The Stochastic Dual Dynamic Programming (SDDP) algorithm has become one of the main tools to address convex multistage stochastic optimal control problem. Recently a large amount of work has been devoted to improve the convergence speed of the algorithm through cut-selection and regularization, or to extend the field of applications to non-linear, integer or risk-averse problems. However one of the main downside of the algorithm remains the difficulty to give an upper bound of the optimal value, usually estimated through Monte Carlo methods and therefore difficult to use in the algorithm stopping criterion. In this paper we present a dual SDDP algorithm that yields a converging exact upper bound for the optimal value of the optimization problem. Incidently we show how to compute an alternative control policy based on an inner approximation of Bellman value functions instead of the outer approximation given by the standard SDDP algorithm. We illustrate the approach on an energy production problem involving zones of production and transportation links between the zones. The numerical experiments we carry out on this example show the effectiveness of the method.

Coordinated Control of PEV and PV-Based Storages in Residential Systems Under Generation and Load Uncertainties

Energy storage deployment in residential and commercial applications is an attractive proposition for ensuring proper utilization of solar photovoltaic (PV) power generation. Energy storage can be controlled and coordinated with PV generation to satisfy electricity demand and minimize electricity purchases from the grid. For optimal energy management, PV generation and load demand uncertainties need to be considered when designing a control method for the PV-based storage system. Another resource available at the residential level is the plug-in electric vehicle (PEV) which also has bi-directional power flow capability. The charging and discharging routines of the PEV can be controlled to help reduce the energy drawn from the power grid during peak hours. In this paper, a method of coordinated optimal control between PV-based storage and PEV storage is proposed considering the stochastic nature of solar PV generation and load demand. The stochastic dual dynamic programming algorithm is employed to optimize the charge/discharge profiles of PV-based storage and PEV storage to minimize the daily household electricity purchase cost from the grid. Simulation analysis shows the advantage of the coordinated control compared to other control strategies.

A combined SDDP/Benders decomposition approach with a risk-averse surface concept for reservoir operation in long term power generation planning

Power generation planning in hydrothermal systems is a complex optimization task, specially due to the high uncertainty in the inflows to hydro plants. Since it is impossible to traverse the huge scenario tree of the multistage problem, stochastic dual dynamic programming (SDDP) is the leading technique to solve it, originally from an expected-cost minimization perspective. However, there is a growing need to apply risk-averse/robust formulations to protect the system from critical hydrological scenarios. This is particularly important for predominantly hydro systems, because environmental issues prevent the construction of large reservoirs, thus reducing their water regulating capability. This paper proposes a two-level SDDP/Benders decomposition approach to include a new risk averse surface (RAS) concept for reservoir operation in power generation planning. The upper level problem is a SDDP solving strategy with expected-cost minimization criterion, where recourse functions for each time step are built through forward/backward passes. The second level consists in multi-period deterministic optimization subproblems for each node of the scenario tree, which are solved to ensure a desired level of protection from a set of given critical scenario several months ahead. An inner iterative procedure for each SDDP stage/scenario is applied, where feasibility cuts are included in the upper level subproblems to derive the RAS surface, which are multidimensional rule curves for reservoir operation. Such curves ensure that the policy provided by the SDDP algorithm yields storage levels in the reservoirs that are high enough to protect the system against such critical scenarios. A “max-type” time-linking penalization scheme for violation of RAS constraints is also proposed, which avoids the multiple application of the penalty value for the same violation in consecutive time steps, which may result in large marginal costs. Results are presented for the large-scale Brazilian system.

Optimal bidding of a virtual power plant on the Spanish day-ahead and intraday market for electricity

We develop a multi-stage stochastic programming approach to optimize the bidding strategy of a virtual power plant (VPP) operating on the Spanish spot market for electricity. The VPP markets electricity produced in the wind parks it manages on the day-ahead market and on six staggered auction-based intraday markets. Uncertainty enters the problem via stochastic electricity prices as well as uncertain wind energy production. We set up the problem of bidding for one day of operation as a Markov decision process (MDP) that is solved using a variant of the stochastic dual dynamic programming algorithm. We conduct an extensive out-of-sample comparison demonstrating that the optimal policy obtained by the stochastic program clearly outperforms deterministic planning, a pure day-ahead strategy, a benchmark that only uses the day-ahead market and the first intraday market, as well as a proprietary stochastic programming approach developed in the industry. Furthermore, we study the effect of risk aversion as modeled by the nested Conditional Value-at-Risk as well as the impact of changes in various problem parameters.

Multi-Stage Stochastic Programming to Joint Economic Dispatch for Energy and Reserve With Uncertain Renewable Energy

To address the uncertain renewable energy in the day-ahead optimal dispatch of energy and reserve, a multi-stage stochastic programming model is established in this paper to minimize the expected total costs. The uncertainties over the multiple stages are characterized by a scenario tree and the optimal dispatch scheme is cast as a decision tree which guarantees the flexibility to decide the reasonable outputs of generation and the adequate reserves accounting for different realizations of renewable energy. Most importantly, to deal with the “Curse of Dimensionality” of stochastic programming, stochastic dual dynamic programming (SDDP) is employed, which decomposes the original problem into several sub-problems according to the stages. Specifically, the SDDP algorithm performs forward pass and backward pass repeatedly until the convergence criterion is satisfied. At each iteration, the original problem is approximated by creating a linear piecewise function. Besides, an improved convergence criterion is adopted to narrow the optimization gaps. The results on the IEEE 118-bus system and real-life provincial power grid show the effectiveness of the proposed model and method.