Stochastic Dual Dynamic Programming Method Research Articles

Foreign exchange trading and management with the stochastic dual dynamic programming method

We present a novel tool for generating speculative and hedging foreign exchange (FX) trading policies. Our solution provides a schedule that determines trades in each rebalancing period based on future currency prices, net foreign account positions, and incoming (outgoing) flows from business operations. To obtain such policies, we construct a multistage stochastic programming (MSP) model and solve it using the stochastic dual dynamic programming (SDDP) numerical method, which specializes in solving high-dimensional MSP models. We construct our methodology within an open-source SDDP package, avoiding implementing the method from scratch. To measure the performance of our policies, we model FX prices as a mean-reverting stochastic process with random events that incorporate stochastic trends. We calibrate this price model on seven currency pairs, demonstrating that our trading policies not only outperform the benchmarks for each currency, but may also be close to ex-post optimal solutions. We also show how the tool can be used to generate more or less conservative strategies by adjusting the risk tolerance, and how it can be used in a variety of contexts and time scales, ranging from intraday speculative trading to monthly hedging for business operations. Finally, we examine the impact of increasing trade policy uncertainty (TPU) levels on our findings. Our findings show that the volatility of currencies from emerging economies rises in comparison to currencies from developed markets. We discover that an increase in the TPU level has no effect on the average profit obtained by our method. However, the risk exposure of the policies increases (decreases) for the group of currencies from emerging (developed) markets.

Multi-Stage Distributionally Robust Stochastic Dual Dynamic Programming to Multi-Period Economic Dispatch With Virtual Energy Storage

A virtual energy storage (VES) model is proposed in this paper to accommodate renewable energy under a special market regulation. Such VESs can provide or consume electricity to the main power grid under the premise that the daily net electricity energy is balanced. Furthermore, a multi-stage distributionally robust optimization (MSDRO) model is set up in this paper to address the temporal uncertainties in the day-ahead economic dispatch model. Compared with the traditional two-stage distributionally robust optimization, the proposed multi-stage approach provides more flexibilities so that the decision variables can be adjusted at each time period, leading to a complex nested formulation. To efficiently solve the MSDRO model, a stochastic dual dynamic programming method is employed to decompose the original large-scale optimization model into several sub-problems in the stages, as two steps: forward pass and backward pass. In the forward pass, the expected cost-to-go function is approximated by piecewise-linear functions and then several samples are used to generate a lower bound; the backward pass will generate Benders’ cuts at each stage from the solution of the forward pass. The forward and backward passes are performed iteratively until the convergence is reached. Numerical results on an IEEE 118-bus system and a practical power system in China verify the proposed method.

Asynchronous parallel stochastic dual dynamic programming applied to hydrothermal generation planning

Optimizing power generation planning is very important to achieve the lowest possible costs while meeting the required power demand along time. High computational resources are required to solve this problem and, in some cases, the use of parallel schemes is imperative. A widely used method to solve long-term energy planning problems is an extension of Dual Dynamic Programming (DDP) called Stochastic Dual Dynamic Programming (SDDP) which makes use of sampling techniques to be able to deal with high-dimensional state-spaces. In this work, we propose an asynchronous SDDP parallel scheme (labeled ASDDP) capable of overcoming the intrinsic synchronism of traditional versions of the SDDP method, by dividing subproblems by time steps instead of forward samples. This parallelization strategy, which includes a variant called totally asynchronous ASDDP (TASDDP), allows us to better exploit the parallel resources and decreases the overall CPU time to solve the problem. Consistency and performance tests were applied to evaluate the application of the proposed ASDDP and TASDDP algorithms in the real Brazilian system.

A stochastic dual dynamic programming method for two-stage distributionally robust optimization problems

This paper studies a class of two-stage distributionally robust optimization (TDRO) problems which comes from many practical application fields. In order to set up some implementable solution method, we first transfer the TDRO problem to its equivalent robust counterpart (RC) by the duality theorem of optimization. The RC reformulation of TDRO is a semi-infinite stochastic programming. Then we construct a conditional value-at-risk-based sample average approximation model for the RC problem. Furthermore, we analyse the error bound of the approximation model and obtain the convergent results with respect to optimal value and optimal solution set. Finally, a so-called stochastic dual dynamic programming approach is proposed to solve the approximate model. Numerical results validate the solution approach of this paper.

Mixed Spatial and Temporal Decompositions for Large-Scale Multistage Stochastic Optimization Problems

We consider multistage stochastic optimization problems involving multiple units. Each unit is a (small) control system. Static constraints couple units at each stage. We present a mix of spatial and temporal decompositions to tackle such large scale problems. More precisely, we obtain theoretical bounds and policies by means of two methods, depending on whether the coupling constraints are handled by prices or by resources. We study both centralized and decentralized information structures. We report the results of numerical experiments on the management of urban microgrids. It appears that decomposition methods are much faster and give better results than the standard stochastic dual dynamic programming method, both in terms of bounds and of policy performance.

Partially observable multistage stochastic programming

We propose a class of partially observable multistage stochastic programs and describe an algorithm for solving this class of problems. We provide a Bayesian update of a belief-state vector, extend the stochastic programming formulation to incorporate the belief state, and characterize saddle-function properties of the corresponding cost-to-go function. Our algorithm is a derivative of the stochastic dual dynamic programming method.

Nonconvex Medium-Term Hydropower Scheduling by Stochastic Dual Dynamic Integer Programming

Hydropower producers rely on stochastic optimization when scheduling their resources over long periods of time. Due to its computational complexity, the optimization problem is normally cast as a stochastic linear program. In a future power market with more volatile power prices, it becomes increasingly important to capture parts of the hydropower operational characteristics that are not easily linearized, e.g., unit commitment and nonconvex generation curves. Stochastic dual dynamic programming (SDDP) is a state-of-the-art algorithm for long- and medium-term hydropower scheduling with a linear problem formulation. A recently proposed extension of the SDDP method known as stochastic dual dynamic integer programming (SDDiP) has proven convergence also in the nonconvex case. We apply the SDDiP algorithm to the medium-term hydropower scheduling (MTHS) problem and elaborate on how to incorporate stagewise-dependent stochastic variables on the right-hand sides and the objective of the optimization problem. Finally, we demonstrate the capability of the SDDiP algorithm on a case study for a Norwegian hydropower producer. The case study demonstrates that it is possible but time-consuming to solve the MTHS problem to optimality. However, the case study shows that a new type of cut, known as strengthened Benders cut, significantly contributes to close the optimality gap compared to classical Benders cuts.

Worst-Case-Expectation Approach to Optimization Under Uncertainty

In this paper we discuss multistage programming with the data process subject to uncertainty. We consider a situation where the data process can be naturally separated into two components: one can be modeled as a random process, with a specified probability distribution, and the other one can be treated from a robust (worst-case) point of view. We formulate this in a time consistent way and derive the corresponding dynamic programming equations. To solve the obtained multistage problem, we develop a variant of the stochastic dual dynamic programming method. We give a general description of the algorithm and present computational studies related to planning of the Brazilian interconnected power system.

Risk neutral and risk averse Stochastic Dual Dynamic Programming method

In this paper we discuss risk neutral and risk averse approaches to multistage (linear) stochastic programming problems based on the Stochastic Dual Dynamic Programming (SDDP) method. We give a general description of the algorithm and present computational studies related to planning of the Brazilian interconnected power system.

Analysis of stochastic dual dynamic programming method

In this paper we discuss statistical properties and convergence of the Stochastic Dual Dynamic Programming (SDDP) method applied to multistage linear stochastic programming problems. We assume that the underline data process is stagewise independent and consider the framework where at first a random sample from the original (true) distribution is generated and consequently the SDDP algorithm is applied to the constructed Sample Average Approximation (SAA) problem. Then we proceed to analysis of the SDDP solutions of the SAA problem and their relations to solutions of the “true” problem. Finally we discuss an extension of the SDDP method to a risk averse formulation of multistage stochastic programs. We argue that the computational complexity of the corresponding SDDP algorithm is almost the same as in the risk neutral case.

Multi-Reservoir Operation Planning Using Hybrid Genetic Algorithm and Linear Programming (GA-LP): An Alternative Stochastic Approach

Many models have been suggested to deal with the multi-reservoir operation planning stochastic optimization problem involving decisions on water releases from various reservoirs in different time periods of the year. A new approach using genetic algorithm (GA) and linear programming (LP) is proposed here to determine operational decisions for reservoirs of a hydro system throughout a planning period, with the possibility of considering a variety of equally likely hydrologic sequences representing inflows. This approach permits the evaluation of a reduced number of parameters by GA and operational variables by LP. The proposed algorithm is a stochastic approximation to the hydro system operation problem, with advantages such as simple implementation and the possibility of extracting useful parameters for future operational decisions. Implementation of the method is demonstrated through a small hypothetical hydrothermal system used in literature as an example for stochastic dual dynamic programming (SDDP) method of Pereira and Pinto (Pereira, M. V. F. and Pinto, L. M. V. G.: 1985, Water Res. Res. 21(6), 779–792). The proposed GA-LP approach performed equally well as compared to the SDDP method.