Multistage Stochastic Optimization Research Articles

On a multistage discrete stochastic optimization problem with stochastic constraints and nested sampling

We consider a multistage stochastic discrete program in which constraints on any stage might involve expectations that cannot be computed easily and are approximated by simulation. We study a sample average approximation (SAA) approach that uses nested sampling, in which at each stage, a number of scenarios are examined and a number of simulation replications are performed for each scenario to estimate the next-stage constraints. This approach provides an approximate solution to the multistage problem. To establish the consistency of the SAA approach, we first consider a two-stage problem and show that in the second-stage problem, given a scenario, the optimal values and solutions of the SAA converge to those of the true problem with probability one when the sample sizes go to infinity. These convergence results do not hold uniformly over all possible scenarios for the second stage problem. We are nevertheless able to prove that the optimal values and solutions of the SAA converge to the true ones with probability one when the sample sizes at both stages increase to infinity. We also prove exponential convergence of the probability of a large deviation for the optimal value of the SAA, the true value of an optimal solution of the SAA, and the probability that any optimal solution to the SAA is an optimal solution of the true problem. All of these results can be extended to a multistage setting and we explain how to do it. Our framework and SAA results cover a large variety of resource allocation problems for which at each stage after the first one, new information becomes available and the allocation can be readjusted, under constraints that involve expectations estimated by Monte Carlo. As an illustration, we apply this SAA method to a staffing problem in a call center, in which the goal is to optimize the numbers of agents of each type under some constraints on the quality of service (QoS). The staffing allocation has to be decided under an uncertain arrival rate with a prior distribution in the first stage, and can be adjusted at some additional cost when better information on the arrival rate becomes available in later stages.

Challenges in the Management of Hydroelectric Generation in Power System Operations

The management of hydroelectric generation in the context of power system operations has been a difficult and important problem since the inception of power systems more than a century ago; however, various current developments are leading to important new associated challenges and opportunities: massive integration of variable renewable energy and other disruptive technologies, climate change effects on the availability of hydro inflows, and also new efficient techniques for optimization under uncertainty. Multistage stochastic optimization and stochastic dual dynamic programming are currently the dominant techniques for hydroelectric generation scheduling problems; however, there are many recent extensions and improvements on such techniques, and alternative approaches are being developed with significant potential for future concrete applications from power system operators and policy makers. In this context, this paper presents a literature review on hydroelectric generation scheduling models, and a discussion on the critical challenges, open research questions, and future lines of research associated to this problem.

Adapted Wasserstein distances and stability in mathematical finance

Assume that an agent models a financial asset through a measure ℚ with the goal to price/hedge some derivative or optimise some expected utility. Even if the model ℚ is chosen in the most skilful and sophisticated way, the agent is left with the possibility that ℚ does not provide an exact description of reality. This leads us to the following question: will the hedge still be somewhat meaningful for models in the proximity of ℚ?If we measure proximity with the usual Wasserstein distance (say), the answer is No. Models which are similar with respect to the Wasserstein distance may provide dramatically different information on which to base a hedging strategy.Remarkably, this can be overcome by considering a suitable adapted version of the Wasserstein distance which takes the temporal structure of pricing models into account. This adapted Wasserstein distance is most closely related to the nested distance as pioneered by Pflug and Pichler (SIAM J. Optim. 20:1406–1420, 2009, SIAM J. Optim. 22:1–23, 2012, Multistage Stochastic Optimization, 2014). It allows us to establish Lipschitz properties of hedging strategies for semimartingale models in discrete and continuous time. Notably, these abstract results are sharp already for Brownian motion and European call options.

Optimal Bidding and Operation Strategies for EV Aggegators by Regrouping Aggregated EV Batteries

We propose an optimal operation strategy for an electric vehicle (EV) aggregator (AGG), which performs energy arbitrage in the energy market and provides ancillary services from aggregated EVs, while providing charging services to EVs to maximize the profit in a future energy market. We group EV batteries as several virtual batteries (VB) with respect to their departure time and stages to implement multiple schedules simultaneously in the multi-stage stochastic optimization (MSSO). They are continuously grouped and regrouped as EVs arrive and depart. We predict VB scenario trees of stepwise EV driving routes and optimize the decisions in responding to uncertain EV movements. The VB states change as EVs enter and exit through the stages, so we should indirectly track time-varying VB characteristics at each stage. Then, we distribute the regulation bids for the VBs to the individual EVs. To reduce mismatches between the bidding amounts for VBs and actual transacted amounts for EVs, we suggest a novel binary progressive hedging algorithm to quickly determine the VB operations. Based on data from historical vehicle statistics, we verify that our strategy with the MSSO model provides a higher profit to the AGG than two-stage stochastic models.

Evaluation of scenario reduction algorithms with nested distance

Multistage stochastic optimization is used to solve many real-life problems where decisions are taken at multiple times. Such problems need the representation of stochastic processes, which are usually approximated by scenario trees. In this article, we implement seven scenario reduction algorithms: three based on random extraction, named Random, and four based on specific distance measures, named Distance-based. Three of the latter are well known in literature while the fourth is a new approach, namely nodal clustering. We compare all the algorithms in terms of computational cost and information cost. The computational cost is measured by the time needed for the reduction, while the information cost is measured by the nested distance between the original and the reduced tree. Moreover, we also formulate and solve a multistage stochastic portfolio selection problem to measure the distance between the optimal solutions and between the optimal objective values of the original and the reduced tree.

Robust multiperiod inventory model considering trade-in program and refurbishment service: Implications to emerging markets

We propose a closed-loop supply chain system that incorporates a trade-in program and refurbishment service simultaneously. Through the trade-in program, retailers collect used old-generation products from customers and provide them with new-generation products at a discount price. It helps to acquire the additional products required for the refurbishment service. The proposed integrated system could be useful to retailers selling the smartphone which has a high potential in emerging markets. We approximate the multistage stochastic optimization model to the robust counterpart which features a second-order cone program. Computational results provide managerial insights that could be beneficial to the retailer.

Compacting multistage stochastic programming models through a new implicit extensive framework

Deterministic equivalent models reformulate optimization problems from a computational perspective. Nonetheless, these models become computationally intractable quickly when the number of stages increase. In this context, a framework to reduce the size of scenario tree and multistage stochastic optimization problems is proposed. Scenario trees are generated using the Knuth transformation for a more compact representation. Moreover, the optimization model is described by using an implicit extensive form approach. The framework is tested in an asset-liability management multistage stochastic model with joint chance constraints, making it possible to acquire the optimal solution for large instances without any relaxation or decomposition mechanism.

Balancing supply and demand in the presence of renewable generation via demand response for electric water heaters

With the increasing presence of renewable energy sources in the electrical power grid, demand response via thermostatic appliances such as electric water heaters is a promising way to compensate for the significant variability in renewable power generation. We propose a multistage stochastic optimization model that computes the optimal day-ahead target profile of the mean thermal energy contained in a large population of heaters, given various possible wind power production and uncontrollable load scenarios. This optimal profile is calculated to make the variable net demand as even as possible.

Dynamic stochastic approximation for multi-stage stochastic optimization

In this paper, we consider multi-stage stochastic optimization problems with convex objectives and conic constraints at each stage. We present a new stochastic first-order method, namely the dynamic stochastic approximation (DSA) algorithm, for solving these types of stochastic optimization problems. We show that DSA can achieve an optimal $${{\mathcal {O}}}(1/\epsilon ^4)$$ rate of convergence in terms of the total number of required scenarios when applied to a three-stage stochastic optimization problem. We further show that this rate of convergence can be improved to $${{\mathcal {O}}}(1/\epsilon ^2)$$ when the objective function is strongly convex. We also discuss variants of DSA for solving more general multi-stage stochastic optimization problems with the number of stages $$T > 3$$ . The developed DSA algorithms only need to go through the scenario tree once in order to compute an $$\epsilon $$ -solution of the multi-stage stochastic optimization problem. As a result, the memory required by DSA only grows linearly with respect to the number of stages. To the best of our knowledge, this is the first time that stochastic approximation type methods are generalized for multi-stage stochastic optimization with $$T \ge 3$$ .

Risk-averse real-time dispatch of integrated electricity and heat system using a modified approximate dynamic programming approach

Coordinated operation of integrated electricity and heat system can improve operation flexibility and reduce cost. However, multiple uncertainties challenge its optimal operation. This paper aims at developing a risk-averse and computationally efficient policy for real-time stochastic dispatch of integrated electricity and heat system, which improves the economy as well as avoiding the risk of high costs in critical scenarios. First, real-time dispatch of integrated electricity and heat system is formulated as a multistage risk-averse stochastic sequential optimization problem with dynamic risk measure, where combined heat and power unit, energy storage, flexible electricity and heat load are jointly utilized to minimize the risk-adjusted total costs. Next, a risk-averse dynamic programming formulation of the original problem is presented, upon which a data-driven risk-averse approximate dynamic programming is employed to address computational challenge, and develop almost optimal and computationally efficient policy. By exploiting information from training samples in off-line learning, the proposed algorithm can efficiently responses to the stochastic exogenous information. Comparative simulations with different risk-aversion preferences and different methods verify the effectiveness of the proposed algorithm.

Some matheuristic algorithms for multistage stochastic optimization models with endogenous uncertainty and risk management

Two matheuristic decomposition algorithms are introduced. The first one is a Progressive Hedging type so-named Regularized scenario Cluster Progressive Algorithm. The second one is a Frank-Wolfe PH type so-named Regularized scenario Cluster Simplicial Decomposition Progressive Algorithm. An extension of endogenous Type III uncertainty is considered for representing the decision-dependent scenario probability and outlook. Its performance is tested in the time-consistent Expected Conditional Stochastic Dominance risk averse environment. As a result of the modeling, the typical risk neutral multistage mixed 0–1 linear stochastic problem under uncertainty is replaced with an enlarged model that is equivalent to the required mixed 0–1 bilinear model. Based on the special features of the problem, it is unrealistic to seek the optimal solution for large-scale instances. Feasible solutions and lower bounds on the solution value of the original model are provided. In total, 48 strategies are considered, each one consists of a combination of a regularization norm, a calibration type for the PH pseudo-gradient computation, and a set of value intervals of the influential variables on a representative endogenous uncertainty-based piecewise function in the scenarios. Computational results are reported for a large-scale extension of a well-known real-life pilot case for preparedness resource allocation planning aiming to natural disaster relief. The matheuristics outperform the plain use of a state-of-the-art solver.

Stochastic Maintenance Schedules of Active Distribution Networks Based on Monte-Carlo Tree Search

The integration of volatile distributed energy resources (DERs) brings new challenges for the active distribution network maintenance scheduling (DN-MS). Conventionally, the DN-MS is formulated as a deterministic optimization model without considering the uncertainties of DERs. In this paper, the DN-MS is formulated as a multistage stochastic optimization problem, which is cast as a stochastic mixed-integer nonlinear programming model. It aims to reduce the total maintenance cost constrained by the reliability indices. To capture the operational characteristics of active distribution networks, the uncertainties of DERs and post-outage operation strategies of switching devices are incorporated into the model. In general, this type of model is intractable and mainly solved by heuristic search methods with low efficiency. Recently, Monte-Carlo tree search (MCTS) is emerging as a scalable and promising reinforcement learning approach. We propose a stochastic MCTS solution to this problem. In the tree search procedure, a sample average approximation technique is developed to estimate multistage maintenance costs considering uncertainties. To speed up the MCTS, the complicated constraints of the original problem are transformed to penalty or heuristics functions. This approach can asymptotically approximate the optimum with promising computation efficiency. Numerical test results demonstrate the superiority of the proposed method over benchmark methods.

Optimal distribution systems operation in the presence of wind power by coordinating network reconfiguration and demand response

This paper discusses the optimal distribution system operation in the presence of wind power by coordinating network reconfiguration and demand response method. Under the specific demand response program, the paper proposes an optimal control method of coordinating thermostatically controlled load (TCL) and battery energy storage (BES). The TCL of the demand response can be controlled in the premise of not hurting customer comfort. The risk of exceeding thermal customer comfort limit is measured by conditional value-at-risk (CVaR) index. Under the requirement of timely wind power balancing, the hourly reconfiguration plan is adopted. Because of the stochastic characteristics of wind power, this is a risk-constrained optimization problem with the objective of minimizing operation cost. Due to the time interdependence of wind power, a Markov Decision Process (MDP) formulation in combination with dynamic programming is adopted to solve the multi-stage stochastic optimization problem. The results show that coordinating reconfiguration and demand response method could significantly contribute to compensating for the forecast error of wind power, therefore, bringing considerable economic savings on operation cost in real-time electricity market.

Optimal Stochastic Deployment of Heterogeneous Energy Storage in a Residential Multienergy Microgrid With Demand-Side Management

The optimal deployment of heterogeneous energy storage (HES), mainly consisting of electrical and thermal energy storage, is essential for increasing the holistic energy utilization efficiency of multienergy systems. Consequently, this article proposes a risk-averse method for HES deployment in a residential multienergy microgrid (RMEMG), considering the diverse uncertainties and multienergy demand-side management (DSM). Apart from the HES size and location planning, its optimal investment phase is also determined by maximizing the system equivalent daily profit (EDP) and minimizing the risk. To handle the system uncertainties from renewable energy sources, power demands, outdoor temperature, and residential hot water needs, the multistage adaptive stochastic optimization approach is utilized. Then, through the constraint linearization and stochastic scenario sampling, the original nonlinear deployment model is converted to a mixed-integer linear programming one and tested on an IEEE 33-bus distribution network based RMEMG. The effectiveness of the proposed method is verified by comparing it with the existing practices. The comparison results indicate that the proposed risk-averse deployment method can effectively increase the system EDP and more immune to the uncertainties. Besides, this method can be practically applied for the emerging RMEMGs, such as smart buildings, intelligent homes, etc., which get long-term DSM contracts.

Hierarchical Model Predictive Control for Hydraulic Hybrid Powertrain of a Construction Vehicle

Hybrid hydraulic technology has the advantages of high-power density and low price and shows good adaptability in construction machinery. A complex hybrid powertrain architecture requires optimization and management of power demand distribution and an accurate response to desired power distribution of the power source subsystems in order to achieve target performances in terms of fuel consumption, drivability, component lifetime, and exhaust emissions. For hybrid hydraulic vehicles (HHVs) that are used in construction machinery, the challenge is to design an appropriate control scheme to actually achieve fuel economy improvement taking into consideration the relatively low energy density of the hydraulic accumulator and frequent load changes, the randomness of the driving conditions, and the uncertainty of the engine dynamics. To improve fuel economy and adaptability of various driving conditions to online energy management and to enhance the response performance of an engine to a desired torque, a hierarchical model predictive control (MPC) scheme is presented in this paper using the example of a spray-painting construction vehicle. The upper layer is a stochastic MPC (SMPC) based energy management control strategy (EMS) and the lower layer is an MPC-based tracking controller with disturbance estimator of the diesel engine. In the SMPC-EMS of the upper-layer management, a Markov model is built using driving condition data of the actual construction vehicle to predict future torque demands over a finite receding horizon to deal with the randomness of the driving conditions. A multistage stochastic optimization problem is formulated, and a scenario-based enumeration approach is used to solve the stochastic optimization problem for online implementation. In the lower-layer tracking controller, a disturbance estimator is designed to handle the uncertainty of the engine, and the MPC is introduced to ensure the tracking performance of the output torque of the engine for the distributed torque from the upper-layer SMPC-EMS, and therefore really achieve high efficiency of the diesel engine. The proposed strategy is evaluated using both simulation MATLAB/Simulink and the experimental test platform through a comparison with several existing strategies in two real driving conditions. The results demonstrate that the proposed strategy (SMPC+MPC) improves miles per gallon an average by 7.3% and 5.9% as compared with the control strategy (RB+PID) consisting of a rule-based (RB) management strategy and proportional-integral-derivative (PID) controller of the engine in simulation and experiment, respectively.

Closed-loop stochastic oilfield optimization for hedging against geologic, development, and operation uncertainty

Development and management of oilfields involve several sources of uncertainty that complicate an already challenging decision-making process. Two main sources of uncertainty are related to geologic description of reservoirs and future development and operation scenarios. While geologic uncertainty has been widely studied and robust optimization methods have been developed to account for it, the uncertainty in future development and operations has not been considered. Traditionally, future development strategies have been included as optimization decision variables. However, in practice, due to unpredictable factors, future plans tend to deviate from the solutions obtained in optimization problems. In a recent publication, we showed that a more prudent and realistic approach toward oilfield optimization is to consider future development plans as uncertain parameters to ensure that the obtained solutions remain robust when future plans change. Here, we develop a closed-loop stochastic field development optimization formulation to account for the uncertainty in geologic description and future development and operation scenarios. The proposed approach optimizes the decision variables for current stage of planning (e.g., well locations and operational settings) while accounting for geologic and future development and operation uncertainties. Geologic uncertainty is represented with several reservoir model realizations, while development uncertainty is incorporated through drilling scenario trees and stochastic well placement. To account for operation uncertainty, future well controls are also described probabilistically. Prior to each development decision-making stage, the reservoir is operated using a closed-loop control approach, including model calibration against dynamic data, subject to geologic and future development and operation uncertainties. After each data assimilation step, a new stochastic optimization is performed to adjust controllable decision variables for the current well configuration (e.g., well rates or BHPs) using the updated models and potentially revised future development scenarios. Using a multistage stochastic optimization workflow, this process is repeated after each decision stage. Several numerical experiments are presented to discuss various aspects of the proposed closed-loop stochastic optimization formulation and to evaluate its robustness.

Inexact Cuts in Stochastic Dual Dynamic Programming

We introduce an extension of stochastic dual dynamic programming (SDDP) to solve stochastic convex dynamic programming equations. This extension applies when some or all primal and dual subproblems to be solved along the forward and backward passes of the method are solved with bounded errors (inexactly). This inexact variant of SDDP is described for both linear problems (the corresponding variant being denoted by ISDDP-LP) and nonlinear problems (the corresponding variant being denoted by ISDDP-NLP). We prove convergence theorems for ISDDP-LP and ISDDP-NLP for both bounded and asymptotically vanishing errors. Finally, we present the results of numerical experiments comparing SDDP and ISDDP-LP on a portfolio problem with direct transaction costs modeled as a multistage stochastic linear optimization problem. In these experiments, ISDDP-LP allows us to strike a different balance between policy quality and computing time, trading off the former for the latter.

Coordinated and Priority-based Surgical Care: An Integrated Distributionally Robust Stochastic Optimization Approach

We study a Coordinated clinic and surgery Appointment Scheduling (CAS) problem for advance scheduling of surgical patients. Our models seek to provide timely access to care by coordinating clinic and surgery appointments to ensure that patients can see a surgeon in the clinic and (if needed) schedule their surgery within a maximum wait time target based on patient classes. There are different types of uncertainty including the number of appointment requests, whether a patient requires surgery, and surgery durations. We develop an Integrated Multi-stage Stochastic and Distributionally Robust Optimization (IMSDRO) approach to determine the optimal clinic and surgery dates for patients such that the access target constraints are satisfied, and the clinical and surgical overtimes are minimized. The IMSDRO approach synergizes multi-stage stochastic optimization with distributionally robust optimization to simultaneously incorporate multiple types of uncertainties by including stochastic scenarios for appointment request arrivals and ambiguity sets for surgery durations. Several new transformations are introduced to turn the nonlinear model derived from the IMSDRO approach to a tractable one, and a constraint generation algorithm is developed to solve it efficiently. We propose a data-driven Rolling Horizon Procedure (RHP) to facilitate implementation. We use case data to assess the performance of our policies. The results suggest our policy can significantly improve surgical access delay times compared to the current practice. Our methodology is not limited to a particular setting and can be applied to other service industries where access delay matters.

다단계 확률론적 계획법과 점진적 회피 알고리즘을 이용한 에너지저장장치 최적 동작에 관한 연구

다단계 확률론적 계획법과 점진적 회피 알고리즘을 이용한 에너지저장장치 최적 동작에 관한 연구

Robust multiperiod inventory model with a new type of buy one get one promotion: “My Own Refrigerator”

Recently, one of the largest retail companies in Korea introduced a mobile application that enables customers to store buy-one-get-one (BOGO) products in their virtual storage for later use. That is, customers who store the extra freebies in their virtual storage can drop by the store to pick them up in the future. Consequently, the application was successful in attracting customers. However, this type of promotion has significant implications for inventory levels. Since customers who buy the product do not need to take both products on the day of purchase, the promotion involves a high degree of uncertainty regarding the revisiting date. To deal with this uncertainty, we propose a robust multiperiod inventory model by addressing the approximation of a multistage stochastic optimization model. Without full information on the distribution, the inventory policy can be derived with support and the first and second moments of uncertainty factors. The presented model is different from previous studies in that the sum of the uncertainty factors in a particular interval is constrained to less than or equal to 1. This part is reformulated as a robust counterpart that retains tractability under modest data sizes. The results of the comparative simulation experiments show that the presented model provides a robust and stable solution against the worst-case scenario. We also obtain managerial insights from the experiments by varying the expiry date according to three types of customers’ revisiting tendencies.