Multistage Stochastic Optimization Problems Research Articles

Near-Optimal Adaptive Policies for Serving Stochastically Departing Customers

In, “Near-Optimal Adaptive Policies for Serving Stochastically Departing Customers,” Segev considers a multistage stochastic optimization problem originally introduced by Cygan et al. [Cygan M, Englert M, Gupta A, Mucha M, Sankowski P (2013) Catch them if you can: How to serve impatient users. Proc. 4th Innovations Theoretical Comput. Sci. Conf., 485–494], studying how a single server should prioritize stochastically departing customers. In this setting, the objective is to determine an adaptive service policy that maximizes the expected total reward collected along a discrete planning horizon, in the presence of customers who are independently departing between one stage and the next with known stationary probabilities. The paper’s main contribution resides in proposing a quasi-polynomial-time approximation scheme for serving impatient customers. Specifically, letting n be the number of underlying customers, our algorithm identifies in [Formula: see text] time a service policy whose expected reward is within factor [Formula: see text] of the optimal adaptive reward. The method for deriving this approximation scheme synthesizes various stochastic analyses in order to investigate how the adaptive optimum is affected by alterations to several instance parameters, including the reward values, the departure probabilities, and the collection of customers itself.

Maximizing Computation Rate for Sustainable Wireless-Powered MEC Network: An Efficient Dynamic Task Offloading Algorithm with User Assistance

In the Internet of Things (IoT) era, Mobile Edge Computing (MEC) significantly enhances the efficiency of smart devices but is limited by battery life issues. Wireless Power Transfer (WPT) addresses this issue by providing a stable energy supply. However, effectively managing overall energy consumption remains a critical and under-addressed aspect for ensuring the network’s sustainable operation and growth. In this paper, we consider a WPT-MEC network with user cooperation to migrate the double near–far effect for the mobile node (MD) far from the base station. We formulate the problem of maximizing long-term computation rates under a power consumption constraint as a multi-stage stochastic optimization (MSSO) problem. This approach is tailored for a sustainable WPT-MEC network, considering the dynamic and varying MEC network environment, including randomness in task arrivals and fluctuating channels. We introduce a virtual queue to transform the time-average energy constraint into a queue stability problem. Using the Lyapunov optimization technique, we decouple the stochastic optimization problem into a deterministic problem for each time slot, which can be further transformed into a convex problem and solved efficiently. Our proposed algorithm works efficiently online without requiring further system information. Extensive simulation results demonstrate that our proposed algorithm outperforms baseline schemes, achieving approximately 4% enhancement while maintain the queues stability. Rigorous mathematical analysis and experimental results show that our algorithm achieves O(1/V),O(V) trade-off between computation rate and queue stability.

Time Consistency for Multistage Stochastic Optimization Problems under Constraints in Expectation

Time Consistency for Multistage Stochastic Optimization Problems under Constraints in Expectation

Three-Stage Stochastic Unit Commitment for Microgrids Toward Frequency Security via Renewable Energy Deloading

Renewable energy is boosting the deployment of microgrids (MGs) with stochastic and low inertia nature. To improve the operational efficiency of MGs while guaranteeing frequency security, a three-stage stochastic unit commitment problem is proposed, where renewable energy can be deloaded. In the first stage, the diesel generators (DGs) are scheduled, responding to uncertainties of loads and photovoltaic generator output. In the second stage, the outputs of DGs are optimized to reduce the operational cost under uncertain disturbances. In the third stage, a novel PV deloading strategy is proposed to test the frequency security of MGs. The three-stage optimization problem is formulated as a multi-stage stochastic optimization problem with recourse. This problem is then solved using a novel nested Benders decomposition algorithm with both dual cuts and partial primal cuts. Simulations are performed on an AC MG under different PV penetration levels and disturbances. The results verify the effectiveness of the proposed model in balancing operational efficiency, renewable energy utilization, and frequency security.

A Nonconvex Regularization Scheme for the Stochastic Dual Dynamic Programming Algorithm

We propose a new nonconvex regularization scheme to improve the performance of the stochastic dual dynamic programming (SDDP) algorithm for solving large-scale multistage stochastic programs. Specifically, we use a class of nonconvex regularization functions, namely folded concave penalty functions, to improve solution quality and the convergence rate of the SDDP procedure. We develop a strategy based on mixed-integer programming to guarantee global optimality of the nonconvex regularization problem. Moreover, we establish provable convergence guarantees for our customized SDDP algorithm. The benefits of our regularization scheme are demonstrated by solving large-scale instances of two multistage stochastic optimization problems. History: Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2021.0255 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0255 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Combining predictive and prescriptive techniques for optimizing electric vehicle fleet charging

The last decade has witnessed a burgeoning interest in transportation electrification from the academia, government, and industry. A current barrier faced by fleet operators is the charge scheduling, a problem that becomes more pronounced with the fleet size, heterogeneity (in both the vehicle fleet and the charging infrastructure), and uncertainty, which has given rise to the Charging-as-a-Service (CaaS) industry. A CaaS provider intermediates between the fleet owner and the macrogrid, and is key to ease the transition to the future of transportation with electric vehicles. This paper addresses the CaaS providers’ electric vehicle fleet (EVF) charge scheduling problem with time-varying electricity prices. We develop a rolling-horizon online optimization approach reinforced with a predictive model and a heuristic warm-start to solve this emerging multi-stage stochastic optimization problem. A numerical experiment demonstrates that our method outperforms an industry benchmark by 13.24%–18.44% with respect to charging costs under the tested conditions. In addition to cost savings, the energy use profile of resulting schedules consumes less energy in peak hours, which can reduce carbon emissions and improve grid stability. Thus, the proposed approach identifies charging schedules that simultaneously benefit CaaS providers, fleet owners, electric power producers, and the macrogrid in general.

On solving large-scale multistage stochastic optimization problems with a new specialized interior-point approach

A novel approach based on a specialized interior-point method (IPM) is presented for solving large-scale stochastic multistage continuous optimization problems, which represent the uncertainty in strategic multistage and operational two-stage scenario trees. This new solution approach considers a split-variable formulation of the strategic and operational structures. The specialized IPM solves the normal equations by combining Cholesky factorizations with preconditioned conjugate gradients, doing so for, respectively, the constraints of the stochastic formulation and those that equate the split-variables. We show that, for multistage stochastic problems, the preconditioner (i) is a block-diagonal matrix composed of as many shifted tridiagonal matrices as the number of nested strategic-operational two-stage trees, thus allowing the efficient solution of systems of equations; (ii) its complexity in a multistage stochastic problem is equivalent to that of a very large-scale two-stage problem. A broad computational experience is reported for large multistage stochastic supply network design (SND) and revenue management (RM) problems. Some of the most difficult instances of SND had 5 stages, 839 million linear variables, 13 million quadratic variables, 21 million constraints, and 3750 scenario tree nodes; while those of RM had 8 stages, 278 million linear variables, 100 million constraints, and 100,000 scenario tree nodes. For those problems, the proposed approach obtained the solution in 1.1 days using 174 gigabytes of memory for SND, and in 1.7 days using 83 gigabytes for RM; while CPLEX v20.1 required more than 53 days and 531 gigabytes for SND, and more than 19 days and 410 gigabytes for RM.

On a Class of Multistage Stochastic Hierarchical Problems

In this paper, following the multistage stochastic approach proposed by Rockafellar and Wets, we analyze a class of multistage stochastic hierarchical problems: the Multistage Stochastic Optimization Problem with Quasi-Variational Inequality Constraints. Such a problem is defined in a suitable functional setting relative to a finite set of possible scenarios and certain information fields. The key of this multistage stochastic hierarchical problem turns out to be the nonanticipativity: some constraints have to be included in the formulation to take into account the partial information progressively revealed. In this way, we are able to study real-world problems in which the hierarchical decision processes are characterized by sequential decisions in response to an increasing level of information. As an application of this class of multistage stochastic hierarchical problems, we focus on the study of a suitable Single-Leader-Multi-Follower game.

Energy-Efficient Online Data Sensing and Processing in Wireless Powered Edge Computing Systems

Wireless powered multi-access edge computing (MEC) has emerged as a promising paradigm to enable high-performance computation of energy-constrained wireless devices (WDs) in internet of things (IoT) systems. However, to overcome the severe path loss of both energy transfer and data communications, wireless powered MEC suffers from high operating power consumption. To achieve sustainable and economic system operation, this paper focuses on developing energy-efficient online data processing strategy for wireless powered MEC systems under stochastic fading channels. In particular, we consider a hybrid access point (HAP) transmitting RF energy to and processing the sensing data offloaded from multiple WDs. Under an average power constraint of the HAP, we target at maximizing the long-term average data sensing rate of the WDs while maintaining task data queue stability. To this end, we formulate a multi-stage stochastic optimization problem to control the energy transfer and task data processing in sequential time slots. Without the knowledge of future channel fading, it is very challenging to determine the sequential control actions that are tightly coupled by the battery and data buffer dynamics. To solve the problem, we propose a Lyapunov optimization-based online algorithm named LEESE, which decomposes the multi-stage stochastic problem into per-slot deterministic optimization problems. We show that each per-slot problem can be equivalently transformed into a convex optimization problem. To facilitate online implementation in large-scale MEC systems, instead of solving the per-slot problem with off-the-shelf convex algorithms, we propose a block coordinate descent (BCD)-based method that produces a close-to-optimal solution in less than 0.04% of the computation delay. Simulation results demonstrate that the proposed LEESE algorithm can provide 18% higher data sensing rate than the representative benchmark methods considered, while incurring sub-millisecond computation delay suitable for real-time control under fading channel.

A Data-Driven Approach to Multistage Stochastic Linear Optimization

We propose a new data-driven approach for addressing multistage stochastic linear optimization problems with unknown distributions. The approach consists of solving a robust optimization problem that is constructed from sample paths of the underlying stochastic process. We provide asymptotic bounds on the gap between the optimal costs of the robust optimization problem and the underlying stochastic problem as more sample paths are obtained, and we characterize cases in which this gap is equal to zero. To the best of our knowledge, this is the first sample path approach for multistage stochastic linear optimization that offers asymptotic optimality guarantees when uncertainty is arbitrarily correlated across time. Finally, we develop approximation algorithms for the proposed approach by extending techniques from the robust optimization literature and demonstrate their practical value through numerical experiments on stylized data-driven inventory management problems. This paper was accepted by David Simchi-Levi, optimization. Funding: S. Shtern was supported by the Israel Science Foundation [Grant 1460/19]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/mnsc.2022.4352 .

Distributed Multistage Optimization of Large-Scale Microgrids Under Stochasticity

Microgrids are recognized as a relevant tool to absorb decentralized renewable energies in the energy mix. However, the sequential handling of multiple stochastic productions and demands, and of storage, make their management a delicate issue. We add another layer of complexity by considering microgrids where different buildings stand at the nodes of a network and are connected by the arcs; some buildings host local production and storage capabilities, and can exchange with others their energy surplus. We formulate the problem as a multistage stochastic optimization problem, corresponding to the minimization of the expected temporal sum of operational costs, while satisfying the energy demand at each node, for all time. The resulting mathematical problem has a large-scale nature, exhibiting both spatial and temporal couplings. However, the problem displays a network structure that makes it amenable to a mix of spatial decomposition-coordination with temporal decomposition methods. We conduct numerical simulations on microgrids of different sizes and topologies, with up to 48 nodes and 64 state variables. Decomposition methods are faster and provide more efficient policies than a state-of-the-art Stochastic Dual Dynamic Programming algorithm. Moreover, they scale almost linearly with the state dimension, making them a promising tool to address more complex microgrid optimal management problems.

Long-Term Expansion Planning of the Transmission Network in India under Multi-Dimensional Uncertainty

Considerable investment in India’s electricity system may be required in the coming decades in order to help accommodate the expected increase of renewables capacity as part of the country’s commitment to decarbonize its energy sector. In addition, electricity demand is geared to significantly increase due to the ongoing electrification of the transport sector, the growing population, and the improving economy. However, the multi-dimensional uncertainty surrounding these aspects gives rise to the prospect of stranded investments and underutilized network assets, rendering investment decision making challenging for network planners. In this work, a stochastic optimization model is applied to the transmission network in India to identify the optimal expansion strategy in the period from 2020 until 2060, considering conventional network reinforcements as well as energy storage investments. An advanced Nested Benders decomposition algorithm was used to overcome the complexity of the multistage stochastic optimization problem. The model additionally considers the uncertainty around the future investment cost of energy storage. The case study shows that deployment of energy storage is expected on a wide scale across India as it provides a range of benefits, including strategic investment flexibility and increased output from renewables, thereby reducing total expected system costs; this economic benefit of planning with energy storage under uncertainty is quantified as Option Value and is found to be in excess of GBP 12.9 bn. The key message of this work is that under potential high integration of wind and solar in India, there is significant economic benefit associated with the wide-scale deployment of storage in the system.

Dynamic Risked Equilibrium

We study a competitive partial equilibrium in markets where risk-averse agents solve multistage stochastic optimization problems formulated in scenario trees. The agents trade a commodity that is produced from an uncertain supply of resources. Both resources and the commodity can be stored for later consumption. Several examples of a multistage risked equilibrium are outlined, including aspects of battery and hydroelectric storage in electricity markets, distributed ownership of competing technologies relying on shared resources, and aspects of water control and pricing. The agents are assumed to have nested coherent risk measures based on one-step risk measures with polyhedral risk sets that have a nonempty intersection over agents. Agents can trade risk in a complete market of Arrow-Debreu securities. In this setting, we define a risk-trading competitive market equilibrium and establish two welfare theorems. Competitive equilibrium will yield a social optimum (with a suitably defined social risk measure) when agents have strictly monotone one-step risk measures. Conversely, a social optimum with an appropriately chosen risk measure will yield a risk-trading competitive market equilibrium when all agents have strictly monotone risk measures. The paper also demonstrates versions of these theorems when risk measures are not strictly monotone.

Multistage adaptive stochastic mixed integer optimization under endogenous and exogenous uncertainty

AbstractTo solve multistage adaptive stochastic optimization problems under both endogenous and exogenous uncertainty, a novel solution framework based on robust optimization technique is proposed. The endogenous uncertainty is modeled as scenarios based on an uncertainty set partitioning method. For each scenario, the adaptive binary decision is assumed constant and the continuous variable is approximated by a function linearly dependent on endogenous uncertain parameters. The exogenous uncertainty is modeled using lifting methods. The adaptive decisions are approximated using affine functions of the lifted uncertain parameters. In order to demonstrate the applicability of the proposed framework, a number of numerical examples of different complexity are studied and a case study for infrastructure and production planning of shale gas field development are presented. The results show that the proposed framework can effectively solve multistage adaptive stochastic optimization problems under both types of uncertainty.

Stochastic Dynamic Cutting Plane for Multistage Stochastic Convex Programs

We introduce Stochastic Dynamic Cutting Plane (StoDCuP), an extension of the Stochastic Dual Dynamic Programming (SDDP) algorithm to solve multistage stochastic convex optimization problems. At each iteration, the algorithm builds lower bounding affine functions not only for the cost-to-go functions, as SDDP does, but also for some or all nonlinear cost and constraint functions. We show the almost sure convergence of StoDCuP. We also introduce an inexact variant of StoDCuP where all subproblems are solved approximately (with bounded errors) and show the almost sure convergence of this variant for vanishing errors. Finally, numerical experiments are presented on nondifferentiable multistage stochastic programs where Inexact StoDCuP computes a good approximate policy quicker than StoDCuP while SDDP and the previous inexact variant of SDDP combined with Mosek library to solve subproblems were not able to solve the differentiable reformulation of the problem.

Multistage adaptive stochastic mixed integer optimization through piecewise decision rule approximation

This work studies the multistage adaptive stochastic mixed integer optimization problem, where the aim is to find adaptive continuous and integer decision policies that optimize the expected objective. While searching for the exact optimal policy is challenging, piecewise decision rule-based solution framework is studied and two types of strategies are compared: uncertainty set partitioning and uncertainty lifting. Adaptive binary and continuous decisions are approximated through piecewise constant and affine decision rule, respectively. The original optimization problem is solved through robust counterpart optimization technique. The proposed methods are applied to an inventory control problem and a chemical process capacity planning problem. Results show that the two methods provide flexible options with the trade-off between solution quality and computational efficiency. The uncertainty set partitioning based method leads to better solution quality and is appropriate to small-scale problems. On the other hand, the lifting method provides significant computational efficiency especially for large problems.

Subregular recourse in nonlinear multistage stochastic optimization

We consider nonlinear multistage stochastic optimization problems in the spaces of integrable functions. We allow for nonlinear dynamics and general objective functionals, including dynamic risk measures. We study causal operators describing the dynamics of the system and derive the Clarke subdifferential for a penalty function involving such operators. Then we introduce the concept of subregular recourse in nonlinear multistage stochastic optimization and establish subregularity of the resulting systems in two formulations: with built-in nonanticipativity and with explicit nonanticipativity constraints. Finally, we derive optimality conditions for both formulations and study their relations.

COVID-19: data-driven dynamic asset allocation in times of pandemic

<abstract> The COVID-19 pandemic has demonstrated the importance and value of multi-period asset allocation strategies responding to rapid changes in market behavior. In this article, we formulate and solve a multi-stage stochastic optimization problem, choosing the indices' optimal weights dynamically in line with a customized data-driven Bellman's procedure. We use basic asset classes (equities, fixed income, cash and cash equivalents) and five corresponding indices for the development of optimal strategies. In our multi-period setup, the probability model describing the uncertainty about the value of asset returns changes over time and is scenario-specific. Given a high enough variation of model parameters, this allows to account for possible crises events. In this article, we construct optimal allocation strategies accounting for the influence of the COVID-19 pandemic on financial returns. We observe that the growth in the number of infections influences financial markets and makes assets' behavior more dependent. Solving the multi-stage asset allocation problem dynamically, we (i) propose a fully data-driven method to estimate time-varying conditional probability models and (ii) we implement the optimal quantization procedure for the scenario approximation. We consider optimality of quantization methods in the sense of minimal distances between continuous-state distributions and their discrete approximations. Minimizing the well-known Kantorovich-Wasserstein distance at each time stage, we bound the approximation error, enhancing accuracy of the decision-making. Using the first-stage allocation strategy developed via our method, we observe ca. 10% wealth growth on average out-of-sample with a maximum of ca. 20% and a minimum of ca. 5% over a three-month period. Further, we demonstrate that monthly reoptimization aids in reducing uncertainty at a cost of maximal wealth. Also, we show that optimistically offsetted distribution parameters lead to a reduction in out-of-sample wealth due to the COVID-19 crisis. </abstract>

Multistage Sample Average Approximation for Harvest Scheduling under Climate Uncertainty

Forest planners have traditionally used expected growth and yield coefficients to predict future merchantable timber volumes. However, because climate change affects forest growth, the typical forest planning methods using expected value of forest growth can lead to sub-optimal harvest decisions. In this paper, we propose to formulate the harvest planning with growth uncertainty due to climate change problem as a multistage stochastic optimization problem and use sample average approximation (SAA) as a tool for finding the best set of forest units that should be harvested in the first period even though we have a limited knowledge of what future climate will be. The objective of the harvest planning model is to maximize the expected value of the net present value (NPV) considering the uncertainty in forest growth and thus in revenues from timber harvest. The proposed model was tested on a small forest with 89 stands and the numerical results showed that the approach allows to have superior solutions in terms of net present value and robustness in face of different growth scenarios compared to the approach using the expected growth and yield. The SAA method requires to generate samples from the distribution of the random parameter. Our results suggested that a sampling scheme that focuses on generating high number of samples in distant future stages is favorable compared to having large sample sizes for the near future stages. Finally, we demonstrated that, depending on the level of forest growth change, ignoring this uncertainty can negatively affect forest resources sustainability.

Randomized Progressive Hedging methods for multi-stage stochastic programming

Progressive Hedging is a popular decomposition algorithm for solving multi-stage stochastic optimization problems. A computational bottleneck of this algorithm is that all scenario subproblems have to be solved at each iteration. In this paper, we introduce randomized versions of the Progressive Hedging algorithm able to produce new iterates as soon as a single scenario subproblem is solved. Building on the relation between Progressive Hedging and monotone operators, we leverage recent results on randomized fixed point methods to derive and analyze the proposed methods. Finally, we release the corresponding code as an easy-to-use Julia toolbox and report computational experiments showing the practical interest of randomized algorithms, notably in a parallel context. Throughout the paper, we pay a special attention to presentation, stressing main ideas, avoiding extra-technicalities, in order to make the randomized methods accessible to a broad audience in the Operations Research community.