Multistage Stochastic Programming Research Articles

Batch Learning SDDP for Long-Term Hydrothermal Planning

We consider the stochastic dual dynamic programming (SDDP) algorithm - a widely employed algorithm applied to multistage stochastic programming - and propose a variant using experience replay - a batch learning technique from reinforcement learning. To connect SDDP with reinforcement learning, we cast SDDP as a Q-learning algorithm and describe its application in both risk-neutral and risk-averse settings. We demonstrate the superiority of the algorithm over conventional SDDP by benchmarking it against PSR's SDDP software using a large-scale instance of the long-term planning problem of inter-connected hydropower plants in Colombia. We find that SDDP with batch learning is able to produce tighter optimality gaps in a shorter amount of time than conventional SDDP. We also find that batch learning improves the parallel efficiency of SDDP backward passes.

Multi-stage scenario-based stochastic programming for managing lot sizing and workforce scheduling at Vestel

Multi-stage scenario-based stochastic programming for managing lot sizing and workforce scheduling at Vestel

Combining deep reinforcement learning and multi-stage stochastic programming to address the supply chain inventory management problem

We introduce a novel heuristic designed to address the supply chain inventory management problem in the context of a two-echelon divergent supply chain. The proposed heuristic advances the current state-of-the-art by combining deep reinforcement learning with multi-stage stochastic programming. In particular, deep reinforcement learning is employed to determine the number of batches to produce, while multi-stage stochastic programming is applied to make shipping decisions. To support further research, we release a publicly available software environment that simulates a wide range of two-echelon divergent supply chain settings, allowing the manipulation of various parameter values, including those associated with seasonal demands. We then present a comprehensive set of numerical experiments considering constraints on production and warehouse capacities under fixed and variable logistic costs. The results demonstrate that the proposed heuristic significantly and consistently outperforms pure deep reinforcement learning algorithms in minimizing total costs. Moreover, it overcomes several inherent limitations of multi-stage stochastic programming models, thus underscoring its potential advantages in addressing complex supply chain scenarios.

Multistage stochastic dominance: an application to pension fund management

AbstractA pension fund manager typically decides the allocation of the pension fund assets taking into account a long-term sustainability goal. Many asset and liability management models, in the form of multistage stochastic programming problem, have been proposed to help the pension fund manager to define the optimal allocation given a multi-objective function. The recent literature proposes univariate stochastic dominance constraints to guarantee that the optimal strategy is able to stochastically dominate a benchmark portfolio. In this work we extend previous results (i) considering alternative types of multivariate stochastic dominance that appear more suitable in a multistage framework, (ii) proposing a way to measure the economic cost of introducing stochastic dominance constraints, (iii) proposing a sort of augmented stochastic dominance through a safety margin. Numerical results show the difference between the alternative ways to interpret and apply the multivariate stochastic dominance. These results are evaluated thanks to the proposed economic cost of the stochastic dominance constraints and either in presence or not of a safety margin.

Green and reliable medical device supply chain network design under deep dynamic uncertainty: A novel approach in the context of COVID-19 outbreak

Conditions governing industrial activities during and after global shocks with societal and economic transformations such as the COVID-19 pandemic have led to the loss of effectiveness of conventional approaches to dealing with uncertainties. The occurrence of sharp fluctuations in the essential parameters has left decision-makers in an unpredictable situation. Therefore, proactive efforts should be made to develop current approaches for adapting to new conditions. This paper establishes a strategic, tactical, and operational decision-making framework under the COVID-19 outbreak by developing a new uncertainty type called deep dynamic uncertainty. In the first step, a Mixed-Integer Linear Programming (MILP) model is proposed for the green and reliable closed-loop supply chain network design. The proposed model allows the decision-maker (DM) to manage and control co2emissions and e-waste generation. In the second step, a new three-step algorithm called Augmented Adjustable Column-Wise Robust Optimization (AACWRO) is first proposed. Then, by combining the proposed column-wise uncertainty with multi-stage stochastic programming (MSSP) approach, deep dynamic uncertainty is theorized for modeling the demand uncertainty under pandemic conditions. The model's performance under deep dynamic uncertainty has been carefully investigated based on the real ventilator and infusion pump supply chain network in Iran. The model under deep dynamic uncertainty, while maintaining tractability and adjustability, provides flexibility in entering data into the problem and significantly increases the coverage of modeling uncertainties. The results clearly demonstrate the efficiency of the proposed approach. The model under deep dynamic uncertainty at all levels of conservatism has on average 42.96% lower cost and 32% higher stability than the MSSP model.

A multistage distributionally robust optimization approach for generation dispatch with demand response under endogenous and exogenous uncertainties

AbstractDecision‐dependent (endogenous) uncertainties (DDUs), as a new type of uncertainties revealed recently, couple dispatch decisions with uncertainty parameters and thus render power system dispatch more challenging. However, most previous works handled various DDUs via stochastic programming (SP) or robust optimization (RO) in a two‐stage framework, which undoubtedly introduces the drawbacks of SP and RO, and cannot meet the nonanticipativity requirements in power scheduling. In this paper, a multistage distributionally robust optimization (DRO) method for generation dispatch with demand response (DR) is proposed considering the DDUs of deferrable loads and the decision‐independent (exogenous) uncertainties (DIUs) of wind power and regular loads. By analyzing the structure of decision‐dependency parameters, a novel data‐driven decision‐dependent ambiguity set is proposed, which provides a generic framework for formulating DDUs and DIUs simultaneously. Then a multistage DRO model with nested max‐min structure is developed to integrate the merits of DRO and nonanticipativity into generation dispatch. The proposed model is solved by tailored reformulation method and improved stochastic dual dynamic integer programming (SDDiP). Case studies illustrate the effectiveness of the proposed approach by comparing with the multistage SP, RO, and decision‐independent DRO methods.

How to operate ship fleets under uncertainty

Ships operated by a liner company are scattered around the world to transport goods. A liner company needs to adjust its shipping network every few months by repositioning its ships to respond to uncertain container shipping demand. Few studies investigate a liner company's multiperiod heterogeneous fleet deployment problem under uncertainty, considering fleet repositioning, ship chartering, demand fulfillment, cargo allocation, and adaptive fleet sizes. To this end, this study formulates a mixed‐integer linear programming model that captures all of these elements. This study also designs a Benders‐based branch‐and‐cut algorithm for this non‐deterministic polynomial‐time (NP)‐hard problem. Two types of acceleration strategies, including approximate upper bound tightening inequalities and Pareto‐optimal cuts, are applied to improve the performance of the algorithm. Extensive numerical experiments show that the proposed algorithm significantly outperforms CPLEX and its Benders decomposition framework in solving the model. We conduct an intensive analysis and find that multistage stochastic programming can lead to better solutions than two‐stage stochastic programming. We also find that 10% of the benefit provided by the multistage model over the two‐stage model is due to better fleet deployment decisions and that 90% of the benefit is due to better demand fulfillment and allocation decisions. By exploring three practical questions regarding driver analysis of liner company profitability, benefits analysis of adaptive fleet sizes, and the influence of the COVID‐19 pandemic on liner shipping, we show how liner companies can benefit from managerial insights obtained in this study.

Sustainability-based enterprise supply chain optimization and response under circular economy approach: agile, adaptive and coordinated

PurposeFrom a supply chain perspective, logistics firms collaborate with other supply chain members to extend their business scope. Investment in circular economy projects in the supply chain can not only broaden the scope of business but also increase the value of the entire supply chain. Third-party logistics companies are gradually participating in the construction and operation of many circular economy projects. How to coordinate multiple circular economy supply chain projects is at the core of its operation.Design/methodology/approachThis paper first analyzes some typical supply chain projects in China and summarizes the main features of these projects. Secondly, considering the benefits of the project and the stakes of each project, a multi-stage stochastic programming model is established. Finally, Cplex, nested decomposition, LocalSolver and other methods are adopted to simulate and analyze the model.FindingsThe final experimental results find that the importance of coordinating multiple circular economy supply chain projects to increase the value of the entire supply chain. The multi-stage stochastic programming model presented in this research can provide a useful tool for logistics enterprises and third-party logistics companies to optimize their investment decisions and maximize their profits in the context of a circular economy.Research limitations/implicationsThere are still some limitations to this study; for example, it is limited to the analysis of circular economy supply chain projects in China. The study focused on third-party logistics companies, and other enterprises in the circular economy supply chain were not considered. The research also assumed that the benefits of each circular economy project and the stakes of each project were known, which may not always be the case in real-world scenarios.Originality/valueThis manuscript found that investing in other circular economy projects in the supply chain can broaden the scope of business and increase the value of the entire supply chain. Third-party logistics companies are gradually participating in the construction and operation of many circular economy projects, such as recycling and repurposing initiatives. It highlights the importance of coordinating multiple circular economy supply chain projects to increase the value of the entire supply chain. The multi-stage stochastic programming model presented in this research can provide a useful tool for logistics enterprises and third-party logistics companies to optimize their investment decisions and maximize their profits in the context of a circular economy.

Optimal portfolio choice of couples with tax-deferred accounts and survival-contingent products

Financial products for retirement planning generally have complex taxation structures and death conditions. In particular, tax-deferred accounts (TDAs) can provide tax-sheltered wealth accumulation by deferring taxes, even with the same financial products. Additionally, various survival-contingent products (SCPs), such as annuity products and life insurance contracts, have different payouts for policyholders. In this study, considering both the TDA and SCPs, we formulate and solve a couple’s lifetime portfolio choice problem using a multistage stochastic programming model. Owing to its high-dimensional state space and lifelong planning periods, stochastic dual dynamic programming (SDDP) was used to solve this problem. We find some interesting results; when both the TDA and SCPs are available, the portfolio is less concentrated in annuity holdings than when the TDA is unavailable. Moreover, the couple ends their contribution to the TA earlier than when SCPs are unavailable.

Design of a Supply Chain-Based Production and Distribution System Based on Multi-Stage Stochastic Programming

Abstract Supply chains are one of the key tools in optimizing production and distribution simultaneously. However, information uncertainty is always a challenge in production and distribution management. The main purpose of this paper is to design a two-echelon supply chain in a multi-cycle state and in conditions of demand uncertainty. The task includes determining the number and location of distribution centers, planning capacity for active distribution centers, and determining the amount of shipments between different levels so that the total costs of the chain are minimized. Uncertainty is applied through discrete scenarios in the model and the problem is formulated by multi-stage stochastic programming method in the form of a mixed integer linear model. The results acquired using two indicators called VMS and VSS demonstrated that modeling the supply chain design problem with the multi-stage stochastic approach can result in significant costs reduction. Plus, utilizing mathematical expectation can generate misleading results, therefore resulting in the development of supply chain designs incapable of satisfying demand due to its overlooked limitations.

A multi-stage stochastic programming approach for an inventory–routing problem considering life cycle

Food waste and proper methods to deal with it are one of the main challenges of supply chain network management. The majority of studies on how to use mathematical models in the supply chain have focused on goods that are at their peak of freshness as soon as they are produced and deteriorate over time. While some products experience an increase in value at the start of their life cycle, this value eventually reaches its maximum level, and after this point, these products experience a decline in value before being eliminated from the consumption cycle. The objective of this study is to develop a comprehensive inventory–routing model suitable for supply chain networks where products exhibit an increase and decrease in value over time. By considering the randomness and dynamic uncertainty of market demands and the fact that each period has effects on the next period, The proposed model employs a multi-stage stochastic programming (MSSP) approach. By doing so, the model ensures a balanced flow between different components of the network while considering nondeterministic demand under various scenarios that are shown in a tree of scenarios. The utilization of MSSP leads to more reliable solutions compared to deterministic models, making it possible for chain stores to make well-informed decisions in their inventory management and distribution strategies. Ultimately, this approach results in cost savings for chain stores handling such products. This research makes a significant contribution to the existing literature by demonstrating the effectiveness of the proposed model on actual data and highlighting the benefits of using stochastic programming in supply chain optimization.

A hybrid inverse optimization-stochastic programming framework for network protection

Disaster management is a complex problem demanding sophisticated modeling approaches. We propose utilizing a hybrid method involving inverse optimization to parameterize the cost functions for a road network’s traffic equilibrium problem and employing a modified version of a two-stage stochastic model to make protection decisions using the information gained from inverse optimization. Inverse optimization allows users to take observations of solutions of optimization and/or equilibrium problems and estimate the parameter values of the functions defining them. In the case of multi-stage stochastic programs for disaster relief, using inverse optimization to parameterize the cost functions can prevent users from making incorrect protection decisions. We demonstrate the framework using two types of cost functions for the traffic equilibrium problem and two different networks. We showcase the value of inverse optimization by demonstrating that, in most of the experiments, different decisions are made when the stochastic network protection problem is parameterized by inverse optimization versus when it is parameterized using a uniform cost assumption. We also demonstrate that similar decisions are made when the stochastic network protection problem is parameterized by inverse optimization versus when it is parameterized by the original/“true” cost parameters.

Multistage stochastic fractionated intensity modulated radiation therapy planning

Intensity modulated radiation therapy (IMRT) is a widely used cancer treatment technique designed to target malignant cells. To enhance its effectiveness on tumors and reduce side effects, radiotherapy plans are usually divided into consecutive treatments, or fractions, that are delivered over multiple weeks. However, typical planning approaches have focused on finding the full sequence of radiation intensities prior to the treatment, or were restricted to a single treatment session. In this work, we investigate the fractionated IMRT planning problem that accounts for geometric motion-related uncertainty during treatment. We propose a novel multistage stochastic programming (MSP) modeling framework that incorporates the sequential decision-making nature of the problem and prevailing stochasticity in cancer treatment. We model the uncertainty in the form of a scenario tree following the multiple instance of geometry approximation, and solve the MSP model using stochastic dual dynamic programming considering a variety of risk measures. We conduct computational experiments on five test cases that are generated based on clinical data. Through extensive simulations, we show that our MSP model generates higher quality treatment plans compared to deterministic and two-stage program counterparts based on multiple performance measures. In particular, our model leads to a higher rate of tumor coverage and a lower rate of radiation exposure for healthy tissues. Accordingly, the proposed MSP framework can greatly contribute to the clinical practice in fractionated IMRT.

Dynamic container drayage booking and routing decision support approach for E-commerce platforms

E-commerce drayage platforms are relatively new to the container drayage industry. Their revenue management poses an enormous logistical challenge owing to the unpredictable demand coupled with complex operations. In this paper, we propose a novel dynamic container drayage booking and routing problem with uncertain demand that incorporates important features of this emerging business model from the perspective of an e-commerce drayage platform. We formulate a multi-stage stochastic programming model for problem followed by developing an effective two-phase heuristic method based on a multi-scenario approach to derive both service booking and fleet routing solutions. In the two-phase heuristic, a container drayage service booking sub-problem is solved at the first phase to determine whether each request should be accepted for service or rejected, as well as the time period for serving the accepted requests, while a fleet routing sub-problem is tackled at the second phase to generate work schedules for the tractors and trailers to transport the containers. We conduct comprehensive computational experiments to demonstrate the quality and value of the proposed two-phase heuristic, and reveal insights into the practical operations management of e-commerce drayage platforms.

Hydropower preventive maintenance scheduling in a deregulated market: A multi-stage stochastic programming approach

Hydropower preventive maintenance scheduling in a deregulated market: A multi-stage stochastic programming approach

Toward circular economy for pomegranate fruit supply chain under dynamic uncertainty: A case study

Food security is a fundamental prerequisite of human survival. This study focuses on providing a novel bi-objective model for the design of a green closed-loop pomegranate supply chain based on the value chain. To maximize the benefits of circular economy, we propose a thermochemical conversion process and develop a novel hybrid risk-neutral and risk-averse multi-stage stochastic programming (RNRAMSSP) approach to cope with supply uncertainty. We employ the expected value of perfect information (EVPI) and value of stochastic solution (VSS) to validate the proposed approach. For the VSS metric, we define a novel measure to compare the worst-case cost of the stochastic and deterministic model solutions. To demonstrate the applicability of the proposed model, we provide a real case study in Iran. Based on the VSS metric, the risk-neutral and risk-averse stochastic model solutions save up to 1% and 1.76% in the worst-case cost, respectively, compared to the deterministic model.

Large-scale financial planning via a partially observable stochastic dual dynamic programming framework

The multi-stage stochastic programming (MSP) approach is widely used to solve financial planning problems owing to its flexibility. However, the size of an MSP problem grows exponentially with the number of stages, and such problem can easily become computationally intractable. Financial planning problems often consider planning horizons of several decades, and thus, the curse of dimensionality can become a critical issue. Stochastic dual dynamic programming (SDDP), a sampling-based decomposition algorithm, has emerged to resolve this issue. While SDDP has been successfully implemented in the energy domain, few applications of SDDP are found in the finance domain. In this study, we identify the major obstacle in using SDDP to solve financial planning problems to be the stagewise independence assumption and propose a partially observable SDDP (PO-SDDP) framework to overcome such limitations. We argue that the PO-SDDP framework, which models uncertainties using discrete-valued partially observable Markov states and introduces feasibility cuts, can properly address large-scale financial planning problems.

Projective Hedging Algorithms for Multistage Stochastic Programming, Supporting Distributed and Asynchronous Implementation

In “Projective Hedging Algorithms for Multistage Stochastic Programming, Supporting Distributed and Asynchronous Implementation,” Eckstein, Watson, and Woodruff derive a new class of decomposition methods for convex multistage stochastic programs defined on finite but potentially large scenario trees. These methods resemble Rockafellar and Wets’ now-classical progressive hedging (PH) method but are based on a flexible projective operator-splitting scheme instead of the standard alternating direction method of multipliers (ADMM). The new algorithms only need to reoptimize subproblems for a subset of the scenarios at each iteration, instead of all of them, and are also amenable to a form of asynchronous implementation, without the algorithm randomization or small step-size requirements usually imposed in such contexts. In the online appendix, the authors demonstrate significant computational gains over PH, applying hundreds or thousands of processor cores to problem instances with up to a million scenarios.

Production planning in maintenance facilities under uncertain repair time

Production planning in maintenance facilities that rely on the availability of certified operators is a challenging task. This is primarily attributed to the need for the integration of workforce training with the operations schedule, in addition to the uncertain repair time of some components. We confine our attention to the repair and overhaul of advanced technological devices that are composed of several components. In this context, this study proposes a novel approach to embedding uncertain repair times of faulty components and develops a multi-stage stochastic programming model for integrated production and workforce planning in such facilities under independent random repair time. Our numerical experiments conducted on a range of problem instances expose the significant value of incorporating repair time uncertainty in this problem.

A decomposition algorithm for multi‐item production planning with independent random demand

AbstractProduction planning in a multiproduct setting where the demands for different items are independent random variables that are also featured with a dynamic behavior over the planning horizon is a challenging task. With a particular focus on maintenance facilities, this study proposes a multistage stochastic programming (MSP) model for operations planning under independent random demand of faulty components in the modular‐structured devices (e.g., gas turbines) received for repair and overhaul services. A Lagrangian relaxation‐based decomposition heuristic is also developed to efficiently solve the problem for real‐size instances. This heuristic relies on decomposing the MSP model into submodels corresponding to component STs and coordinating them via a subgradient algorithm to obtain a high‐quality feasible solution. Our numerical experiments conducted on a range of problem instances endorse the significant value of incorporating demand uncertainty and the effectiveness of the proposed solution methodology in overcoming computational complexity.