Multistage Stochastic Programming Problems Research Articles

Approximation of multistage stochastic programming problems by smoothed quantization

We present an approximation technique for solving multistage stochastic programming problems with an underlying Markov stochastic process. This process is approximated by a discrete skeleton process, which is consequently smoothed down by means of the original unconditional distribution. Approximated in this way, the problem is solvable by means of Markov Stochastic Dual Dynamic Programming. We state an upper bound for the nested distance between the exact process and its approximation and discuss its convergence in the one-dimensional case. We further propose an adjustment of the approximation, which guarantees that the approximate problem is bounded. Finally, we apply our technique to a real-life production-emission trading problem and demonstrate the performance of its approximation given the “true” distribution of the random parameters.

Multistage stochastic dominance: an application to pension fund management

AbstractA pension fund manager typically decides the allocation of the pension fund assets taking into account a long-term sustainability goal. Many asset and liability management models, in the form of multistage stochastic programming problem, have been proposed to help the pension fund manager to define the optimal allocation given a multi-objective function. The recent literature proposes univariate stochastic dominance constraints to guarantee that the optimal strategy is able to stochastically dominate a benchmark portfolio. In this work we extend previous results (i) considering alternative types of multivariate stochastic dominance that appear more suitable in a multistage framework, (ii) proposing a way to measure the economic cost of introducing stochastic dominance constraints, (iii) proposing a sort of augmented stochastic dominance through a safety margin. Numerical results show the difference between the alternative ways to interpret and apply the multivariate stochastic dominance. These results are evaluated thanks to the proposed economic cost of the stochastic dominance constraints and either in presence or not of a safety margin.

Large-scale financial planning via a partially observable stochastic dual dynamic programming framework

The multi-stage stochastic programming (MSP) approach is widely used to solve financial planning problems owing to its flexibility. However, the size of an MSP problem grows exponentially with the number of stages, and such problem can easily become computationally intractable. Financial planning problems often consider planning horizons of several decades, and thus, the curse of dimensionality can become a critical issue. Stochastic dual dynamic programming (SDDP), a sampling-based decomposition algorithm, has emerged to resolve this issue. While SDDP has been successfully implemented in the energy domain, few applications of SDDP are found in the finance domain. In this study, we identify the major obstacle in using SDDP to solve financial planning problems to be the stagewise independence assumption and propose a partially observable SDDP (PO-SDDP) framework to overcome such limitations. We argue that the PO-SDDP framework, which models uncertainties using discrete-valued partially observable Markov states and introduces feasibility cuts, can properly address large-scale financial planning problems.

Pricing Under Uncertainty in Multi-Interval Real-Time Markets

A Novel Practical Stochastic Pricing Model for Multi-Interval Real-Time Markets Practical implementations of economic dispatch with associated pricing systems are crucial for operating electricity markets. Because of the high volatility caused by the increasing integration of renewable energy, consideration of the underlying stochastic problem is becoming more important than ever. It is a challenge to incorporate the uncertain nature of real-time operations into an already complex multi-interval dynamic problem with intertemporal constraints. Because solving a standard multi-stage stochastic programming problem is too burdensome in terms of calculation time for real-time markets, it has been standard practice in electricity markets to use a deterministic approximation with varying degrees of look-ahead. Cho and Papavasiliou, in their article “Pricing Under Uncertainty in Multi-Interval Real-Time Markets”, introduced a practical alternative method for pricing under uncertainty in multi-interval real-time markets. Using slightly different stochastic formulations, these authors propose an approach that preserves the attractive features from both the deterministic formulation (simpler calculation) and the standard stochastic formulation (better performance).

Value function gradient learning for large-scale multistage stochastic programming problems

A stagewise decomposition algorithm called “value function gradient learning” (VFGL) is proposed for large-scale multistage stochastic convex programs. VFGL finds the parameter values that best fit the gradient of the value function within a given parametric family. Widely used decomposition algorithms for multistage stochastic programming, such as stochastic dual dynamic programming (SDDP), approximate the value function by adding linear subgradient cuts at each iteration. Although this approach has been successful for linear problems, nonlinear problems may suffer from the increasing size of each subproblem as the iteration proceeds. On the other hand, VFGL has a fixed number of parameters; thus, the size of the subproblems remains constant throughout the iteration. Furthermore, VFGL can learn the parameters by means of stochastic gradient descent, which means that it can be easil0y parallelized and does not require a scenario tree approximation of the underlying uncertainties. VFGL was compared with a deterministic equivalent formulation of the multistage stochastic programming problem and SDDP approaches for three illustrative examples: production planning, hydrothermal generation, and the lifetime financial planning problem. Numerical examples show that VFGL generates high-quality solutions and is computationally efficient.

A Modified Progressive Hedging Approach for Multistage Intraday Trade of EV aggregators

The growing prominence of electric vehicle (EV) aggregators in the modern power system is drawing more attention towards modeling their behavior in the short-term electricity markets. The demand-side flexibility offered by the EVs can be leveraged to reduce their charging costs. In this paper, the participation of an EV aggregator in the intraday and balancing market is modeled as a multistage stochastic programming problem. The computational complexity introduced by the peculiarities of the intraday market is solved by a progressive hedging algorithm (PHA), a scenario-based decomposition technique. A randomized scenario sampling approach is implemented to accelerate the PHA which is further improved with a parallel randomized PHA. Finally, an asynchronous version of the parallel randomized PHA is leveraged to speed up the multistage model of EV aggregator trading. We compare the computation time of the modified versions of the PHA algorithm with the conventional PHA for the proposed EV aggregator model. Furthermore, we also show the value of EV aggregator trading in the intraday and balancing markets by comparing its cost to baseline models.

Modeling and optimization of bioethanol production planning under hybrid uncertainty: A heuristic multi-stage stochastic programming approach

When making planning decisions, fluctuation in both product demand and yield for the bioethanol plant denotes an important factor to affect the results. In this paper, a mixed multi-stage stochastic optimization model is proposed to solve the production planning problem of a bioethanol plant with multiple raw materials simultaneously considering uncertain demand and yield. The multi-source uncertainty is described as a multi-layer hybrid scenario tree model according to the historical statistics. Due to the diversity of pretreatment approach in the bioethanol production process, the yield of saccharification liquid is different. At each node of the scenario tree, the uncertainty of demand is expressed by probability. The uncertain scenarios of the yield are integrated at each node of demand to form a mixed scenario tree. To improve the solution efficiency, a heuristic algorithm based on fast forward selection is designed to simplify the decomposition of the large-scale hybrid scenario tree in the mixed multi-stage stochastic programming problem. The effectiveness of the proposed model is verified by a practical case. The results show that the proposed approach performs better than the two-stage stochastic programming and the deterministic approaches, and the algorithm can significantly reduce the computational efforts.

A flexible system design approach for multi-facility capacity expansion problems with risk aversion

This article studies a model for risk aversion when designing a flexible capacity expansion plan for a multi-facility system. In this setting, the decision maker can dynamically expand the capacity of each facility given observations of uncertain demand. We model this situation as a multi-stage stochastic programming problem, and we express risk aversion through the Conditional Value-at-Risk (CVaR) and a mean-CVaR objective. We optimize the multi-stage problem over a tractable family of if–then decision rules using a decomposition algorithm. This algorithm decomposes the stochastic program over scenarios and updates the solutions via the subgradients of the function of cumulative future costs. To illustrate the practical effectiveness of this method, we present a numerical study of a decentralized waste-to-energy system in Singapore. The simulation results show that the risk-averse model can improve the tail risk of investment losses by adjusting the weight factors of the mean-CVaR objective. The simulations also demonstrate that the proposed algorithm can converge to high-performance policies within a reasonable time, and that it is also more scalable than existing flexible design approaches.

Day-Ahead Optimal Dispatch Strategy for Active Distribution Network Based on Improved Deep Reinforcement Learning

To effectively deal with the challenge of optimal dispatch caused by uncertainties such as renewable energy in active distribution network, a day-ahead optimal dispatch strategy for active distribution network based on improved deep reinforcement learning is proposed. First, the day-ahead optimal dispatch problem for the active distribution network is modeled as a multistage stochastic programming model. Multistage models attempt to capture the dynamics of unfolding uncertainties over time, so that the interaction between decision making and uncertainty unfolding is represented more accurately and realistically, and the decision can be adjusted dynamically and adaptively. Second, with the help of deep neural network to approximate the power flow model of the active distribution network in a data-driven manner, an improved deep reinforcement learning algorithm is proposed to solve the corresponding multistage stochastic programming problem. The proposed improved deep reinforcement learning algorithm effectively solves the problem that trial-and-error information acquisition methods are difficult to apply to high-cost power systems. Finally, the effectiveness of the proposed day-ahead optimization dispatch strategy for active distribution network based on improved deep reinforcement learning is verified by a modified IEEE33 case.

Fast Stochastic Dual Dynamic Programming for Economic Dispatch in Distribution Systems

The multistage economic dispatch solution is of paramount importance for achieving the optimal unit commitment and the optimal decentralized decision-making in smart grid operations. Through the approximations of the optimal expected cost-to-go functions, the stochastic dual dynamic programming (SDDP) has been shown effective to obtain the optimum for the multistage stochastic programming (SP) problem, while without finite termination guarantee. Thus, we propose a fast stochastic dual dynamic programming (FSDDP) method to accelerate the convergence rate of the SDDP, including updating the candidate points based on the case with the maximum difference between the upper and lower bounds of the optimal value functions, adding a backward-forward-backward inner scheme in the backward pass, and initializing the bounds with finite constant values. We can verify that the FSDDP method enjoys a nice property with a finite convergence guarantee for the multistage SP problems. Numerical tests on 4-, 33-, 34-, 69-, 118- and 123-bus distribution systems have demonstrated that FSDDP can approach the optimal economic performance at significantly improved computational efficiency.

Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

Opportunistic computation offloading is an effective method to improve the computation performance of mobile-edge computing (MEC) networks under dynamic edge environment. In this paper, we consider a multi-user MEC network with time-varying wireless channels and stochastic user task data arrivals in sequential time frames. In particular, we aim to design an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. The online algorithm is practical in the sense that the decisions for each time frame are made without the assumption of knowing the future realizations of random channel conditions and data arrivals. We formulate the problem as a multi-stage stochastic mixed integer non-linear programming (MINLP) problem that jointly determines the binary offloading (each user computes the task either locally or at the edge server) and system resource allocation decisions in sequential time frames. To address the coupling in the decisions of different time frames, we propose a novel framework, named LyDROO, that combines the advantages of Lyapunov optimization and deep reinforcement learning (DRL). Specifically, LyDROO first applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems. By doing so, it guarantees to satisfy all the long-term constraints by solving the per-frame subproblems that are much smaller in size. Then, LyDROO integrates model-based optimization and model-free DRL to solve the per-frame MINLP problems with very low computational complexity. Simulation results show that under various network setups, the proposed LyDROO achieves optimal computation performance while stabilizing all queues in the system. Besides, it induces very low computation time that is particularly suitable for real-time implementation in fast fading environments.

Risk-averse multi-stage stochastic optimization for surveillance and operations planning of a forest insect infestation

We derive a nested risk measure for a maximization problem and implement it in a scenario-based formulation of a multi-stage stochastic mixed-integer programming problem. We apply the risk-averse formulation to the surveillance and control of a non-native forest insect, the emerald ash borer (EAB), a wood-boring beetle native to Asia and recently discovered in North America. Spreading across the eastern United States and Canada, EAB has killed millions of ash trees and cost homeowners and local governments billions of dollars. We present a mean-Conditional Value-at-Risk (CVaR), multi-stage, stochastic mixed-integer programming model to optimize a manager’s decisions about surveillance and control of EAB. The objective is to maximize the benefits of healthy ash trees in forests and urban environments under a fixed budget. Combining the risk-neutral objective with a risk measure allows for a trade-off between the weighted expected benefits from ash trees and the expected risks associated with experiencing extremely damaging scenarios. We define scenario dominance cuts (sdc) for the maximization problem and under the decision-dependent uncertainty. We then solve the model using the sdc cutting plane algorithm for varying risk parameters. Computational results demonstrate that scenario dominance cuts significantly improve the solution performance relative to that of CPLEX. Our CVaR risk-averse approach also raises the objective value of the least-benefit scenarios compared to the risk-neutral model. Results show a shift in the optimal strategy from applying less expensive insecticide treatment to more costly tree removal as the manager becomes more risk-averse. We also find that risk-averse managers survey more often to reduce the risk of experiencing adverse outcomes.

A distributed computing framework for multi-stage stochastic planning of renewable power systems with energy storage as flexibility option

An integrated generation, transmission, and energy storage planning model accounting for short-term constraints and long-term uncertainty is proposed. The model allows to accurately quantify the value of flexibility options in renewable power systems by representing short-term operation through the unit commitment constraints. Long-term uncertainty is represented through a scenario tree. The resulting model is a large-scale multi-stage stochastic mixed-integer programming problem. To overcome the computational burden, a distributed computing framework based on the novel Column Generation and Sharing algorithm is proposed. The performance improvement of the proposed approach is demonstrated through study cases applied to the NREL 118-bus power system. The results confirm the added value of modeling short-term constraints and long-term uncertainty simultaneously. The computational case studies show that the proposed solution approach clearly outperforms the state of the art in terms of computational performance and accuracy. The proposed planning framework is used to assess the value of energy storage systems in the transition to a low-carbon power system.

Multistage Transmission-Constrained Unit Commitment With Renewable Energy and Energy Storage: Implicit and Explicit Decision Methods

Generation scheduling decision-making of power systems with renewable energy and energy storage (ES) is a multistage stochastic programming problem in nature, in which unit commitment (UC) decisions have to be made one day ahead before uncertainties are revealed, and hourly economic dispatch (ED) decisions are successively determined when real uncertainty realizations are observed gradually (i.e., nonanticipativity). To this end, inappropriate ED decisions at current hour may cause infeasibility of future ED decisions (i.e., robustness). Thus, how to properly schedule thermal unit outputs and ES charging/ discharging power against uncertainties becomes an important and urgent issue. In this paper, two mixed-integer linear programming (MILP) methods are proposed to solve the scheduling problem of thermal units and ESs with uncertainties while ensuring solution robustness and nonanticipativity: explicit and implicit decision methods. Specifically, explicit decision method directly assumes affine policies between decision variables and uncertainty realizations; implicit decision method explores safe ranges of thermal unit outputs and ES state-of-charge (SOC) levels to guarantees feasibility of future ED solutions. Both methods can guarantee the nonanticipativity and robustness of multistage solutions. Numerical tests illustrate effectiveness of the proposed methods.

Pension fund management with investment certificates and stochastic dominance

This paper considers an extension of the common asset universe of a pension fund to investment certificates. Investment certificates are a class of structured products particularly interesting for their special payoff structures and they are acquiring relevancy in the worldwide markets. In fact, some subclasses of certificates offer loss protection and show high liquidity and, thus, they can be very appreciated by pension fund managers. We consider the problem of a pension fund manager who has to implement an Asset and Liability Management model trying to achieve a long-term sustainability. Therefore, we formulate a multi-stage stochastic programming problem adopting a discrete scenario tree and a multi-objective function. We propose a technique to price highly structured products such as investment certificates on a discrete scenario tree. Finally, we solve the investment problem considering some investment certificate types both in term of payoff structure and protection level, and we test whether they are preferred or not to standard hedging contract such as put options. Moreover, we test the inclusion of first-order and second-order stochastic dominance constraints on multiple stages with respect to a benchmark portfolio. Numerical results show that the portfolio composition reacts to the inclusion of the stochastic dominance constraints, and that the optimal portfolio is efficiently able to reach several targets such as liquidity, returns, sponsor’s extraordinary contribution and funding gap.

Decomposition and discrete approximation methods for solving two-stage distributionally robust optimization problems

Decomposition methods have been well studied for solving two-stage and multi-stage stochastic programming problems, see Rockafellar and Wets (Math. Oper. Res. 16:119–147, 1991), Ruszczynski and Shapiro (Stochastic Programming, Handbook in OR & MS, North-Holland Publishing Company, Amsterdam, 2003) and Ruszczynski (Math. Program. 79:333–353, 1997). In this paper, we propose an algorithmic framework based on the fundamental ideas of the methods for solving two-stage minimax distributionally robust optimization (DRO) problems where the underlying random variables take a finite number of distinct values. This is achieved by introducing nonanticipativity constraints for the first stage decision variables, rearranging the minimax problem through Lagrange decomposition and applying the well-known primal-dual hybrid gradient (PDHG) method to the new minimax problem. The algorithmic framework does not depend on specific structure of the ambiguity set. To extend the algorithm to the case that the underlying random variables are continuously distributed, we propose a discretization scheme and quantify the error arising from the discretization in terms of the optimal value and the optimal solutions when the ambiguity set is constructed through generalized prior moment conditions, the Kantorovich ball and $$\phi$$ -divergence centred at an empirical probability distribution. Some preliminary numerical tests show the proposed decomposition algorithm featured with parallel computing performs well.

Stochastic Lipschitz dynamic programming

We propose a new algorithm for solving multistage stochastic mixed integer linear programming (MILP) problems with complete continuous recourse. In a similar way to cutting plane methods, we construct nonlinear Lipschitz cuts to build lower approximations for the non-convex cost-to-go functions. An example of such a class of cuts are those derived using Augmented Lagrangian Duality for MILPs. The family of Lipschitz cuts we use is MILP representable, so that the introduction of these cuts does not change the class of the original stochastic optimization problem. We illustrate the application of this algorithm on two case studies, comparing our approach with the convex relaxation of the problems, for which we can apply SDDP, and for a discretized approximation, applying SDDiP.

SDDP.jl: A Julia Package for Stochastic Dual Dynamic Programming

We present SDDP.jl, an open-source library for solving multistage stochastic programming problems using the stochastic dual dynamic programming algorithm. SDDP.jl is built on JuMP, an algebraic modeling language in Julia. JuMP provides SDDP.jl with a solver-agnostic, user-friendly interface. In addition, we leverage unique features of Julia, such as multiple dispatch, to provide an extensible framework for practitioners to build on our work. SDDP.jl is well tested, and accessible documentation is available at https://github.com/odow/SDDP.jl .

Sustainable biomass portfolio sourcing plan using multi-stage stochastic programming

Selecting the most sustainable biomass sources for biofuel production can overcome sever obstacles in the reliable supply of biomass feedstock, such as its seasonal availability. Developing a biomass portfolio can generally improve the sustainability of future biofuel systems. Bioethanol is an attractive biofuel having the potential for enhancing energy security and environmental safety over fossil fuels. The decisions to be made comprise the optimum amount of bioethanol feedstocks, and the investments to make in alternative activities. Uncertainty is assumed for biomass conversion rate to bioethanol, which is described by a set of scenarios. Therefore, for the entire planning horizon, a tree of scenarios is organized. The problem is formulated as a multi-stage stochastic linear programming problem. The goal is to maximize the total financial benefit. The risk associated with biomass availability affects the viability of biomass utilization and, therefore, should be taken into account when evaluating and analyzing biomass portfolio. In addition, the conditional value at risk (CVaR) is adopted to consider the risk hedge. Using a real case study, computational experiments are presented, showing that the stochastic approach is worth considering in these types of problems. Case study results demonstrate the validity of the proposed model.

The policy graph decomposition of multistage stochastic programming problems

AbstractWe propose the policy graph as a structured way of formulating a general class of multistage stochastic programming problems in a way that leads to a natural decomposition. We also propose an extension to the stochastic dual dynamic programming algorithm to solve a subset of problems formulated as a policy graph. This subset includes discrete‐time, convex, infinite‐horizon, multistage stochastic programming problems with continuous state and control variables. To demonstrate the utility of our algorithm, we solve an existing multistage stochastic programming problem from the literature based on pastoral dairy farming. We show that the finite‐horizon model in the literature suffers from end‐of‐horizon effects, which we are able to overcome with an infinite‐horizon model.