Sample Average Approximation Problem Research Articles

Confidence Intervals for Piecewise Normal Distributions and Stochastic Variational Inequalities

In this paper, we first show how to obtain easy-to-compute confidence intervals for the center of a piecewise normal distribution given a sample from this distribution (assuming that the center belongs to a known affine set parallel to the common lineality space of all cones defining the piecewise normal distribution) by using certain skewed projectors on that space. We then extend this method to an asymptotic setting. Next, we apply this method to compute confidence intervals for the true solution of a stochastic variational inequality given a solution to a sample average approximation (SAA) problem for the general situation in which the asymptotic distribution of SAA solutions is piecewise normal. For stochastic complementarity problems, we obtain asymptotic normality of certain estimators of the true solution when the asymptotic distribution of the SAA solutions is piecewise normal. Funding: The research reported in this paper was supported the National Science Foundation [Grant DMS-1814894].

Sample average approximation method for a class of stochastic vector variational inequalities

In the present paper, the expected-value (EV) reformulation of a class of stochastic vector variational inequalities (SVVI) is investigated. By using the regularized gap function, the EV reformulation of SVVI is transformed into a constrained optimization problem. Then a sample average approximation (SAA) method is proposed for solving the constrained optimization problem. Under suitable assumptions, the limiting behaviors of the optimal values and optimal solutions of the approximation problem are investigated. Finally, the rates of convergence in the different senses of optimal solutions for sample average approximation problem are discussed under the error bound condition.

Solving multistage stochastic linear programming via regularized linear decision rules: An application to hydrothermal dispatch planning

The solution of multistage stochastic linear problems (MSLPs) represents a challenge for many applications. Long-term hydrothermal dispatch planning (LHDP) materializes this challenge in a real-world problem that affects electricity markets, economies, and natural resources worldwide. No closed-form solutions are available for MSLP and the definition of non-anticipative policies with high-quality out-of-sample performance is crucial. Linear decision rules (LDRs) provide an interesting simulation-based framework for finding high-quality policies for MSLP through two-stage stochastic models. In practical applications, however, the number of parameters to be estimated when using an LDR may be close to or higher than the number of scenarios of the sample average approximation problem, thereby generating an in-sample overfit and poor performances in out-of-sample simulations. In this paper, we propose a novel regularized LDR to solve MSLP based on the AdaLASSO (adaptive least absolute shrinkage and selection operator). The goal is to use the parsimony principle, as largely studied in high-dimensional linear regression models, to obtain better out-of-sample performance for a LDR applied to MSLP. Computational experiments show that overfitting is a non-negligible threat when using the classical non-regularized LDR to solve the LHDP, one of the most studied MSLP with relevant applications in industry. Our analysis highlights the following benefits of the proposed framework in comparison to the non-regularized benchmark: 1) significant reductions in the number of non-zero coefficients (model parsimony), 2) substantial cost reductions in out-of-sample evaluations, and 3) improved spot-price profiles.

On sample average approximation for two-stage stochastic programs without relatively complete recourse

We investigate sample average approximation (SAA) for two-stage stochastic programs without relatively complete recourse, i.e., for problems in which there are first-stage feasible solutions that are not guaranteed to have a feasible recourse action. As a feasibility measure of the SAA solution, we consider the “recourse likelihood”, which is the probability that the solution has a feasible recourse action. For $$\epsilon \in (0,1)$$ , we demonstrate that the probability that a SAA solution has recourse likelihood below $$1-\epsilon $$ converges to zero exponentially fast with the sample size. Next, we analyze the rate of convergence of optimal solutions of the SAA to optimal solutions of the true problem for problems with a finite feasible region, such as bounded integer programming problems. For problems with non-finite feasible region, we propose modified “padded” SAA problems and demonstrate in two cases that such problems can yield, with high confidence, solutions that are certain to have a feasible recourse decision. Finally, we conduct a numerical study on a two-stage resource planning problem that illustrates the results, and also suggests there may be room for improvement in some of the theoretical analysis.

Randomized Policy Optimization for Optimal Stopping

Optimal stopping is the problem of determining when to stop a stochastic system in order to maximize reward, which is of practical importance in domains such as finance, operations management and healthcare. Existing methods for high-dimensional optimal stopping that are popular in practice produce deterministic linear policies -- policies that deterministically stop based on the sign of a weighted sum of basis functions -- but are not guaranteed to find the optimal policy within this policy class given a fixed basis function architecture. In this paper, we propose a new methodology for optimal stopping based on randomized linear policies, which choose to stop with a probability that is determined by a weighted sum of basis functions. We motivate these policies by establishing that under mild conditions, given a fixed basis function architecture, optimizing over randomized linear policies is equivalent to optimizing over deterministic linear policies. We formulate the problem of learning randomized linear policies from data as a smooth non-convex sample average approximation (SAA) problem. We theoretically prove the almost sure convergence of our randomized policy SAA problem and establish bounds on the out-of-sample performance of randomized policies obtained from our SAA problem based on Rademacher complexity. We also show that the SAA problem is in general NP-Hard, and consequently develop a practical heuristic for solving our randomized policy problem. Through numerical experiments on a benchmark family of option pricing problem instances, we show that our approach can substantially outperform state-of-the-art methods.

Improving Sample Average Approximation Using Distributional Robustness

Sample average approximation is a popular approach to solving stochastic optimization problems. It has been widely observed that some form of robustification of these problems often improves the out-of-sample performance of the solution estimators. In estimation problems, this improvement boils down to a trade-off between the opposing effects of bias and shrinkage. This paper aims to characterize the features of more general optimization problems that exhibit this behaviour when a distributionally robust version of the sample average approximation problem is used. The paper restricts attention to quadratic problems for which sample average approximation solutions are unbiased and shows that expected out-of-sample performance can be calculated for small amounts of robustification and depends on the type of distributionally robust model used and properties of the underlying ground-truth probability distribution of random variables. The paper was written as part of a New Zealand funded research project that aimed to improve stochastic optimization methods in the electric power industry. The authors of the paper have worked together in this domain for the past 25 years.

Central limit theorem and sample complexity of stationary stochastic programs

In this paper we discuss sample complexity of solving stationary stochastic programs by the Sample Average Approximation (SAA) method. We investigate this in the framework of Optimal Control (in discrete time) setting. In particular we derive a Central Limit Theorem type asymptotics for the optimal values of the SAA problems. The main conclusion is that the sample size, required to attain a given relative error of the SAA solution, is not sensitive to the discount factor, even if the discount factor is very close to one. We consider the risk neutral and risk averse settings. The presented numerical experiments confirm the theoretical analysis.

Quantitative stability of the ERM formulation for a class of stochastic linear variational inequalities

<p style='text-indent:20px;'>This paper focuses on the quantitative stability analysis of the expected residual minimization (ERM) formulation for a class of stochastic linear variational inequalities. Firstly, the existence of solutions of the ERM formulation and its perturbed problem is discussed. Then, the quantitative stability of the ERM formulation is derived under suitable probability metrics. Finally, the sample average approximation (SAA) problem of the ERM formulation is studied, and the rates of convergence of optimal solution sets are obtained under different assumptions.</p>

Stochastic joint homecare service and capacity planning with nested decomposition approaches

In this paper, we study a joint service authorization and capacity planning problem for the home health care (HHC) system with demand uncertainty. We formulate the problem as a two-stage stochastic programming with recourse which maximizes the expected total revenue. The service authorization and capacity planning are the mixed binary-and-continuous first stage decisions, and the service hour (resource) allocation is modeled as the second-stage decision which adapts to the demand realizations. The resulting sample average approximation problem (of the stochastic program) could be a large scaled mixed integer linear program. To solve the model in a more scalable fashion, we propose a supergradient-based nested decomposition algorithm that exploits the nice decomposable structure of the problem to cope with the binary and continuous variables separately. The proposed nested decomposition scheme consists of two-layer of decomposition, where the outer decomposition solves for the authorization decision (binaries) with the supergradient cuts and each supergradient can be computed iteratively in the inner decomposition scheme. The proposed nested decomposition algorithm is feasibility-cut-free and is guaranteed to reach the exact optimality in finite steps. Furthermore, we extend our stochastic HHC planning model to a more general framework of Conditional Value-at-Risk (CV@R), and by model transformation with variable change we show that the CV@R model can actually be reformulated in a structure that the proposed nested decomposition scheme can be applied almost directly. Finally, a comprehensive computational study is performed which demonstrates the effectiveness of our model and the performance of the algorithm.

Autonomous and conventional bus fleet optimization for fixed-route operations considering demand uncertainty

The emerging technology of autonomous vehicles has been widely recognized as a promising urban mobility solution in the future. This paper considers the integration of autonomous vehicles into bus transit systems and proposes a modeling framework to determine the optimal bus fleet size and its assignment onto multiple bus lines in a bus service network considering uncertain demand. The mixed-integer stochastic programming approach is applied to formulate the problem. We apply the sample average approximation (SAA) method to solve the formulated stochastic programming problem. To tackle the nonconvexity of the SAA problem, we first present a reformulation method that transforms the problem into a mixed-integer conic quadratic program (MICQP), which can be solved to its global optimal solution by using some existing solution methods. However, this MICQP based approach can only handle the small-size problems. For the cases with large problem size, we apply the approach of quadratic transform with linear alternating algorithm, which allows for efficient solution to large-scale instances with up to thousands of scenarios in a reasonable computational time. Numerical results demonstrate the benefits of introducing autonomous buses as they are flexible to be assigned across different bus service lines, especially when demand uncertainty is more significant. The introduction of autonomous buses would enable further reduction of the required fleets and total cost. The model formulation and solution methods proposed in this study can be used to provide bus transit operators with operational guidance on including autonomous buses into bus services, especially on the autonomous and conventional bus fleets composition and allocation.

Quantitative stability of two-stage stochastic linear variational inequality problems with fixed recourse

ABSTRACT This paper focus on the quantitative stability of a class of two-stage stochastic linear variational inequality problems whose second stage problems are stochastic linear complementarity problems with fixed recourse matrix. Firstly, we discuss the existence of solutions to this two-stage stochastic problems and its perturbed problems. Then, by using the corresponding residual function, we derive the quantitative stability of this two-stage stochastic problem under Fortet-Mourier metric. Finally, we study the sample average approximation problem, and obtain the convergence of optimal solution sets under moderate assumptions.

Interpretable Optimal Stopping

Optimal stopping is the problem of deciding when to stop a stochastic system to obtain the greatest reward, arising in numerous application areas such as finance, healthcare, and marketing. State-of-the-art methods for high-dimensional optimal stopping involve approximating the value function or the continuation value and then using that approximation within a greedy policy. Although such policies can perform very well, they are generally not guaranteed to be interpretable; that is, a decision maker may not be able to easily see the link between the current system state and the policy’s action. In this paper, we propose a new approach to optimal stopping wherein the policy is represented as a binary tree, in the spirit of naturally interpretable tree models commonly used in machine learning. We show that the class of tree policies is rich enough to approximate the optimal policy. We formulate the problem of learning such policies from observed trajectories of the stochastic system as a sample average approximation (SAA) problem. We prove that the SAA problem converges under mild conditions as the sample size increases but that, computationally, even immediate simplifications of the SAA problem are theoretically intractable. We thus propose a tractable heuristic for approximately solving the SAA problem by greedily constructing the tree from the top down. We demonstrate the value of our approach by applying it to the canonical problem of option pricing, using both synthetic instances and instances using real Standard & Poor’s 500 Index data. Our method obtains policies that (1) outperform state-of-the-art noninterpretable methods, based on simulation regression and martingale duality, and (2) possess a remarkably simple and intuitive structure. This paper was accepted by Chung Piaw Teo, Management Science Special Section on Data-Driven Prescriptive Analytics.

Consistency bounds and support recovery of d-stationary solutions of sparse sample average approximations

This paper studies properties of the d(irectional)-stationary solutions of sparse sample average approximation problems involving difference-of-convex sparsity functions under a deterministic setting. Such properties are investigated with respect to a vector which satisfies a verifiable assumption to relate the empirical sample average approximation problem to the expectation minimization problem defined by an underlying data distribution. We derive bounds for the distance between the two vectors and the difference of the model outcomes generated by them. Furthermore, the inclusion relationships between their supports, sets of nonzero valued indices, are studied. We provide conditions under which the support of a d-stationary solution is contained within, and contains, the support of the vector of interest; the first kind of inclusion can be shown for any given arbitrary set of indices. Some of the results presented herein are generalization of the existing theory for a specialized problem of $$\ell _1$$ -norm regularized least squares minimization for linear regression.

Sampling-Based Approximation Schemes for Capacitated Stochastic Inventory Control Models

We study the classical multiperiod capacitated stochastic inventory control problems in a data-driven setting. Instead of assuming full knowledge of the demand distributions, we assume that the demand distributions can only be accessed through drawing random samples. Such data-driven models are ubiquitous in practice, where the cumulative distribution functions of the underlying random demand are either unavailable or too complex to work with. We consider the sample average approximation (SAA) method for the problem and establish an upper bound on the number of samples needed for the SAA method to achieve a near-optimal expected cost, under any level of required accuracy and prespecified confidence probability. The sample bound is polynomial in the number of time periods as well as the confidence and accuracy parameters. Moreover, the bound is independent of the underlying demand distributions. However, the SAA requires solving the SAA problem, which is #P-hard. Thus, motivated by the SAA analysis, we propose a polynomial time approximation scheme that also uses polynomially many samples. Finally, we establish a lower bound on the number of samples required to solve this data-driven newsvendor problem to near-optimality.

A Sampling Approach for Proactive Project Scheduling under Generalized Time-dependent Workability Uncertainty


 
 
 In real-world project scheduling applications, activity durations are often uncertain. Proactive scheduling can effectively cope with the duration uncertainties, by generating robust baseline solutions according to a priori stochastic knowledge. However, most of the existing proactive approaches assume that the duration uncertainty of an activity is not related to its scheduled start time, which may not hold in many real-world scenarios. In this paper, we relax this assumption by allowing the duration uncertainty to be time-dependent, which is caused by the uncertainty of whether the activity can be executed on each time slot. We propose a stochastic optimization model to find an optimal Partial-order Schedule (POS) that minimizes the expected makespan. This model can cover both the time-dependent uncertainty studied in this paper and the traditional time-independent duration uncertainty. To circumvent the underlying complexity in evaluating a given solution, we approximate the stochastic optimization model based on Sample Average Approximation (SAA). Finally, we design two efficient branch-and-bound algorithms to solve the NP-hard SAA problem. Empirical evaluation confirms that our approach can generate high-quality proactive solutions for a variety of uncertainty distributions.
 
 

Asymptotic convergence of stationary points of stochastic multiobjective programs with parametric variational inequality constraint via SAA approach

We consider the sample average approximation method for a stochastic multiobjective programming problem constrained by parametric variational inequalities. The first order necessary conditions for the original problem and its sample average approximation (SAA) problem are established under constraint qualifications. By graphical convergence of set-valued mappings, the stationary points of the SAA problem converge to the stationary points of the true problem when the sample size tends to infinity. Under proper assumptions, the convergence of optimal solutions of SAA problems is also obtained. The numerical experiments on stochastic multiobjective optimization problems with variational inequalities are given to illustrate the efficiency of SAA estimators.

Interpretable Optimal Stopping

Optimal stopping is the problem of deciding when to stop a stochastic system to obtain the greatest reward, arising in numerous application areas such as finance, healthcare and marketing. State-of-theart methods for high-dimensional optimal stopping involve approximating the value function or the continuation value, and then using that approximation within a greedy policy. Although such policies can perform very well, they are generally not guaranteed to be interpretable; that is, a decision maker may not be able to easily see the link between the current system state and the policy's action. In this paper, we propose a new approach to optimal stopping, wherein the policy is represented as a binary tree, in the spirit of naturally interpretable tree models commonly used in machine learning. We show that the class of tree policies is rich enough to approximate the optimal policy. We formulate the problem of learning such policies from observed trajectories of the stochastic system as a sample average approximation (SAA) problem. We prove that the SAA problem converges under mild conditions as the sample size increases, but that computationally even immediate simplifications of the SAA problem are theoretically intractable. We thus propose a tractable heuristic for approximately solving the SAA problem, by greedily constructing the tree from the top down. We demonstrate the value of our approach by applying it to the canonical problem of option pricing, using both synthetic instances and instances using real S&P-500 data. Our method obtains policies that (1) outperform state-of-the-art noninterpretable methods, based on simulation-regression and martingale duality, and (2) possess a remarkably simple and intuitive structure.

Photovoltaic power plant design considering multiple uncertainties and risk

The objective of this study was to develop stochastic optimization tools for determining the best strategy of photovoltaic installations in a campus environment with consideration of uncertainties in load, power generation and system performance. In addition to a risk neutral approach, we used Conditional Value-at-Risk to estimate the risk in our problem. The resulting Mixed Integer Programming models were formulated using a scenario-based approach. To minimize the mismatch between supply and demand, hourly solar resource and electricity demand levels were characterized via refined models. A sample-average approximation (SAA) method was proposed to provide high-quality solutions efficiently. The SAA problems were solved using exact and heuristic methods. A complete numerical study was conducted to examine the performance of the proposed solution methods, identify optimal selection strategies and consider the sensitivity of the solution to varying levels of risk.

Solving equilibrium standby redundancy optimization problem by hybrid PSO algorithm

Redundancy allocation is a direct way of enhancing the series-parallel system lifetime and reliability. Since it is difficult to obtain the exact probability distributions about the lifetimes of components, fuzzy random variables are used to characterize them. Under the given system weights and cost constraints, we maximize the equilibrium optimistic system lifetime of redundant elements. This paper proposes an equilibrium optimization model for the standby redundancy system. Since the exact analytical expressions of the equilibrium optimistic system lifetimes are unavailable in general case, the proposed model cannot be analytically solved. Under mild assumptions, the new equilibrium model can be divided into its equivalent stochastic programming subproblems. Moreover, a new approximation method is proposed to solve the general equilibrium model. For the equivalent stochastic programming subproblems, sample average approximation (SAA) is adapted to gain their SAA problems. A hybrid particle swarm optimization algorithm with local search is designed to solve the SAA problems. Several numerical experiments are conducted to investigate the effectiveness of proposed model and designed solution method. The comparative studies indicate the randomness, and fuzziness cannot be ignored in the equilibrium standby redundancy optimization problem.

Optimization of Airport Surface Operations Under Uncertainty

With the rapidly increasing air traffic volume and airports becoming bottlenecks of the air transportation system, integrated aircraft routing and flight scheduling have the potential to improve the efficiency of airport surface operations. Past research has mainly been focusing on solving subsets of airport operations based on deterministic data. This research develops a fast-time decision support algorithm that solves the integrated aircraft routing and flight scheduling at an airport in the presence of uncertainty. The algorithm is based on a machine job-shop scheduling formulation to model the integrated airport surface operations. The problem is formulated as a multistage stochastic program, and sample average approximation problems are solved to compute solutions. A proof-of-concept study case that considers 13 aircraft is conducted on a model of the northern field of Los Angeles International Airport. To test scenarios representing realistic conditions, perturbed input schedules of 30 min are generated using errors drawn from arrival and departure delay distributions. Outcomes are compared with results obtained with a first-come, first-served approach. Results illustrate that improved runway sequencing can be computed by the stochastic optimization, offering a better use of airport resources with reduced taxi and gate waiting times.