Distributionally Robust Optimization Problem Research Articles

Distributionally robust PV planning and curtailment considering cyber attacks on electric vehicle charging under PV/load uncertainties

A manipulated charging behavior of electric vehicles (EVs) due to an adversary along with the uncertain photovoltaic (PV) generation outputs and loads may lead to unstable power distribution system operations with voltage and current violations. To resolve this issue, this paper proposes an optimization framework where the following two types of uncertainties are addressed: (i) the natural uncertainties of PV generation outputs/loads and (ii) artificial uncertainties of load altering attacks (LAAs) on EV charging stations (EVCSs) via the manipulation of EV charging control signals. The proposed framework is formulated as a Wasserstein metric-integrated distributionally robust optimization (DRO)-based Volt/VAR optimization (VVO) problem. The proposed DRO-based VVO framework combined with PV planning and curtailment aims to minimize substation energy and voltage imbalance along with the complete removal of the constraint violations while handling uncertain PV generation outputs/loads and LAAs. To use off-the-shelf optimization solvers, tractable reformulation of the chance constraints of the voltage, current, and curtailed PV real power of the original DRO problem is provided. Numerical examples tested over IEEE 13-bus and 37-bus systems with PV systems and EVCSs show the efficiency of the proposed DRO framework in terms of substation energy, voltage imbalance, and PV planning/curtailment cost under stochastic LAAs.

Data-Driven Distributionally Robust Optimization over Time

Stochastic optimization (SO) is a classical approach for optimization under uncertainty that typically requires knowledge about the probability distribution of uncertain parameters. Because the latter is often unknown, distributionally robust optimization (DRO) provides a strong alternative that determines the best guaranteed solution over a set of distributions (ambiguity set). In this work, we present an approach for DRO over time that uses online learning and scenario observations arriving as a data stream to learn more about the uncertainty. Our robust solutions adapt over time and reduce the cost of protection with shrinking ambiguity. For various kinds of ambiguity sets, the robust solutions converge to the SO solution. Our algorithm achieves the optimization and learning goals without solving the DRO problem exactly at any step. We also provide a regret bound for the quality of the online strategy that converges at a rate of [Formula: see text], where T is the number of iterations. Furthermore, we illustrate the effectiveness of our procedure by numerical experiments on mixed-integer optimization instances from popular benchmark libraries and give practical examples stemming from telecommunications and routing. Our algorithm is able to solve the DRO over time problem significantly faster than standard reformulations. Funding: This work was supported by Deutsche Forschungsgemeinschaft (DFG): Projects B06 and B10 in CRC TRR 154 and Project-ID 416229255 - SFB 1411 and Federal Ministry for Economic Affairs and Energy, Germany [Grant 03EI1036A]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoo.2023.0091 .

Distributionally Robust Optimization with Moment Ambiguity Sets

This paper studies distributionally robust optimization (DRO) when the ambiguity set is given by moments for the distributions. The objective and constraints are given by polynomials in decision variables. We reformulate the DRO with equivalent moment conic constraints. Under some general assumptions, we prove the DRO is equivalent to a linear optimization problem with moment and psd polynomial cones. A Moment-SOS relaxation method is proposed to solve it. Its asymptotic and finite convergence are shown under certain assumptions. Numerical examples are presented to show how to solve DRO problems.

Effective Scenarios in Multistage Distributionally Robust Optimization with a Focus on Total Variation Distance

We study multistage distributionally robust optimization (DRO) to hedge against ambiguity in quantifying the underlying uncertainty of a problem. Recognizing that not all the realizations and scenario paths might have an “effect” on the optimal value, we investigate the question of how to define and identify critical scenarios for nested multistage DRO problems. Our analysis extends the work of Rahimian, Bayraksan, and Homem-de-Mello [Math. Program., 173 (2019), pp. 393--430], which was in the context of a static/two-stage setting, to the multistage setting. To this end, we define the notions of effectiveness of scenario paths and the conditional effectiveness of realizations along a scenario path for a general class of multistage DRO problems. We then propose easy-to-check conditions to identify the effectiveness of scenario paths in the multistage setting when the distributional ambiguity is modeled via the total variation distance. Numerical results show that these notions provide useful insight on the underlying uncertainty of the problem.

Distributionally robust co-optimized offering for transactive multi-energy microgrids

This paper proposes a two-stage distributionally robust co-optimization model for the microgrid offering problem. This model addresses the market participation of the microgrid including offering, market-clearing and reserve activation stages and captures the information availability from the market-side and the local distributed energy resource (DER) assets during the different stages. The DER assets are modeled through state-space models that capture the joint energy and contingency reserves provision capability for distributed generators, flexible building loads and battery storage systems. Moreover, uncertainties pertaining to market-clearing process, renewable generation, solar irradiation and ambient temperature are modeled in an ambiguity set that comprises statistical characterizations. Distributionally robust optimization (DRO) is employed to generate robust, but less conservative solutions. Linear decision rules and the second-order conic reformulations are employed to ensure the computational tractability of the DRO problem. The suitability of the proposed model is demonstrated for a synthetic multi-phase microgrid with spot market data from Singapore.

Computationally Efficient Approximations for Distributionally Robust Optimization Under Moment and Wasserstein Ambiguity

Distributionally robust optimization (DRO) is a modeling framework in decision making under uncertainty in which the probability distribution of a random parameter is unknown although its partial information (e.g., statistical properties) is available. In this framework, the unknown probability distribution is assumed to lie in an ambiguity set consisting of all distributions that are compatible with the available partial information. Although DRO bridges the gap between stochastic programming and robust optimization, one of its limitations is that its models for large-scale problems can be significantly difficult to solve, especially when the uncertainty is of high dimension. In this paper, we propose computationally efficient inner and outer approximations for DRO problems under a piecewise linear objective function and with a moment-based ambiguity set and a combined ambiguity set including Wasserstein distance and moment information. In these approximations, we split a random vector into smaller pieces, leading to smaller matrix constraints. In addition, we use principal component analysis to shrink uncertainty space dimensionality. We quantify the quality of the developed approximations by deriving theoretical bounds on their optimality gap. We display the practical applicability of the proposed approximations in a production–transportation problem and a multiproduct newsvendor problem. The results demonstrate that these approximations dramatically reduce the computational time while maintaining high solution quality. The approximations also help construct an interval that is tight for most cases and includes the (unknown) optimal value for a large-scale DRO problem, which usually cannot be solved to optimality (or even feasibility in most cases). Summary of Contribution: This paper studies an important type of optimization problem, that is, distributionally robust optimization problems, by developing computationally efficient inner and outer approximations via operations research tools. Specifically, we consider several variants of such problems that are practically important and that admit tractable yet large-scale reformulation. We accordingly utilize random vector partition and principal component analysis to derive efficient approximations with smaller sizes, which, more importantly, provide a theoretical performance guarantee with respect to low optimality gaps. We verify the significant efficiency (i.e., reducing computational time while maintaining high solution quality) of our proposed approximations in solving both production–transportation and multiproduct newsvendor problems via extensive computing experiments.

Optimal Transport-Based Distributionally Robust Optimization: Structural Properties and Iterative Schemes

We consider optimal transport-based distributionally robust optimization (DRO) problems with locally strongly convex transport cost functions and affine decision rules. Under conventional convexity assumptions on the underlying loss function, we obtain structural results about the value function, the optimal policy, and the worst-case optimal transport adversarial model. These results expose a rich structure embedded in the DRO problem (e.g., strong convexity even if the non-DRO problem is not strongly convex, a suitable scaling of the Lagrangian for the DRO constraint, etc., which are crucial for the design of efficient algorithms). As a consequence of these results, one can develop efficient optimization procedures that have the same sample and iteration complexity as a natural non-DRO benchmark algorithm, such as stochastic gradient descent.

Distributionally robust trade‐off design of parity relation based fault detection systems

AbstractThe fault detection (FD) system design aims at optimizing the trade‐off between false alarm rate (FAR) and fault detection rate (FDR) under stochastic disturbances or uncertainties. A challenging difficulty in practice is the inexact information of stochastic disturbance distribution, that is, the actual distribution deviates from the one used in the design. To address this challenge, a distributionally robust optimization (DRO) approach that accounts for the inexact distribution information is proposed for the parity relation based FD of stochastic discrete‐time linear systems. It introduces moment‐based and entropy‐based ambiguity sets to describe the inexact disturbance distribution. Over such ambiguity sets, the FD system design for a scalar residual maximizes the worst‐case FDR with respect to a reference fault mode, while ensuring a predefined worst‐case FAR. To address the limitation of a scalar residual, the FD test of a vector residual is constructed with respect to a parameterized set of multiple fault modes. The resulting FD tests can be expressed in the same structure as the celebrated generalized likelihood ratio test (GLRT), while only the detection threshold is adjusted to compensate for distribution ambiguity. Moreover, the worst‐case FDR in the presence of any given fault is evaluated by solving another DRO problem. Using a continuous stirred tank reactor example with inexact distribution information, it is demonstrated that the proposed designs achieve desirable performance trade‐off and provide effective worst‐case FDR evaluations, while the GLRT fails to ensure the predefined FAR.

Distributionally-robust machine learning using locally differentially-private data

We consider machine learning, particularly regression, using locally-differentially private datasets. The Wasserstein distance is used to define an ambiguity set centered at the empirical distribution of the dataset corrupted by local differential privacy noise. The radius of the ambiguity set is selected based on privacy budget, spread of data, and size of the problem. Machine learning with private dataset is rewritten as a distributionally-robust optimization. For general distributions, the distributionally-robust optimization problem can be relaxed as a regularized machine learning problem with the Lipschitz constant of the machine learning model as a regularizer. For Gaussian data, the distributionally-robust optimization problem can be solved exactly to find an optimal regularizer. Training with this regularizer can be posed as a semi-definite program.

The Value of Randomized Solutions in Mixed-Integer Distributionally Robust Optimization Problems

Randomized decision making refers to the process of making decisions randomly according to the outcome of an independent randomization device, such as a dice roll or a coin flip. The concept is unconventional, and somehow counterintuitive, in the domain of mathematical programming, in which deterministic decisions are usually sought even when the problem parameters are uncertain. However, it has recently been shown that using a randomized, rather than a deterministic, strategy in nonconvex distributionally robust optimization (DRO) problems can lead to improvements in their objective values. It is still unknown, though, what is the magnitude of improvement that can be attained through randomization or how to numerically find the optimal randomized strategy. In this paper, we study the value of randomization in mixed-integer DRO problems and show that it is bounded by the improvement achievable through its continuous relaxation. Furthermore, we identify conditions under which the bound is tight. We then develop algorithmic procedures, based on column generation, for solving both single- and two-stage linear DRO problems with randomization that can be used with both moment-based and Wasserstein ambiguity sets. Finally, we apply the proposed algorithm to solve three classical discrete DRO problems: the assignment problem, the uncapacitated facility location problem, and the capacitated facility location problem and report numerical results that show the quality of our bounds, the computational efficiency of the proposed solution method, and the magnitude of performance improvement achieved by randomized decisions. Summary of Contribution: In this paper, we present both theoretical results and algorithmic tools to identify optimal randomized strategies for discrete distributionally robust optimization (DRO) problems and evaluate the performance improvements that can be achieved when using them rather than classical deterministic strategies. On the theory side, we provide improvement bounds based on continuous relaxation and identify the conditions under which these bound are tight. On the algorithmic side, we propose a finitely convergent, two-layer, column-generation algorithm that iterates between identifying feasible solutions and finding extreme realizations of the uncertain parameter. The proposed algorithm was implemented to solve distributionally robust stochastic versions of three classical optimization problems and extensive numerical results are reported. The paper extends a previous, purely theoretical work of the first author on the idea of randomized strategies in nonconvex DRO problems by providing useful bounds and algorithms to solve this kind of problems.

Two-stage distributionally robust optimization for maritime inventory routing

This work addresses uncertain sailing times and uncertain waiting times at ports in a maritime inventory routing problem (MIRP). As the probability distribution of these uncertain parameters is difficult to estimate and hence not known exactly, we propose a two-stage distributionally robust optimization (DRO) approach in which the uncertainty is described by a Wasserstein ambiguity set. Our model is based on a continuous-time arc-flow mixed integer linear programming (MILP) formulation of the MIRP, and an equivalent robust counterpart of the two-stage DRO problem is derived under the 1-norm Wasserstein metric. We also develop a tailored Benders decomposition algorithm that combines the strengths of Pareto-optimal and high-density cuts to solve large-scale model instances. Computational case studies, including a real-world industrial case considering the maritime transportation of refined diesel along the east coast of China, demonstrate the benefits of the DRO model and the effectiveness of the proposed Benders decomposition algorithm. In general, compared to a traditional stochastic programming approach, the DRO model yields routing solutions that are significantly less sensitive to variations in sailing and port waiting times, and exhibit improved out-of-sample performance.

Efficient Algorithms for Distributionally Robust Stochastic Optimization with Discrete Scenario Support

Recently, there has been a growing interest in distributionally robust optimization (DRO) as a principled approach to data-driven decision making. In this paper, we consider a distributionally robust two-stage stochastic optimization problem with discrete scenario support. While much research effort has been devoted to tractable reformulations for DRO problems, especially those with continuous scenario support, few efficient numerical algorithms were developed, and most of them can neither handle the non-smooth second-stage cost function nor the large number of scenarios $K$ effectively. We fill the gap by reformulating the DRO problem as a trilinear min-max-max saddle point problem and developing novel algorithms that can achieve an $\mathcal{O}(1/\epsilon)$ iteration complexity which only mildly depends on $K$. The major computations involved in each iteration of these algorithms can be conducted in parallel if necessary. Besides, for solving an important class of DRO problems with the Kantorovich ball ambiguity set, we propose a slight modification of our algorithms to avoid the expensive computation of the probability vector projection at the price of an $\mathcal{O}(\sqrt{K})$ times more iterations. Finally, preliminary numerical experiments are conducted to demonstrate the empirical advantages of the proposed algorithms.

Data-Driven Feature-Based Newsvendor: A Distributionally Robust Approach

In this paper, we study the feature-based newsvendor problem in the context of available historical demand observations and related demand covariates. The demand distribution conditioned on these features is based on general kernel estimation methods that can incorporate machine learning algorithms, such as the random forest. To account for estimation errors, we propose a distributionally robust optimization (DRO) approach that contains all distributions close to the estimated conditional demand distribution under a discrepancy measure based on the cumulative distribution functions. Interestingly, we show that the DRO problem with such an ambiguity set admits a closed-form solution for the newsvendor loss. In addition, we show that the optimal solution produced by this approach converges to the optimal inventory decision asymptotically and provides a finite-sample performance guarantee. The results of numerical experiments with synthetic and real-world data sets show that compared with other approaches, our model performs well in terms of the out-of-sample cost and computational time.

Consensus Distributionally Robust Optimization With Phi-Divergence

We study an efficient algorithm to solve the distributionally robust optimization (DRO) problem, which has recently attracted attention as a new paradigm for decision making in uncertain situations. In traditional stochastic programming, a decision is sought that minimizes the expected cost over the probability distribution of the unknown parameters. In contrast, in DRO, robust decision making can be derived from data without assuming a probability distribution; thus, it is expected to provide a powerful method for data-driven decision making. However, it is computationally difficult to solve the DRO problem and even by state-of-art solvers the problem size that can be solved to optimality is still limited. Therefore, we propose an efficient algorithm for solving DRO based on consensus optimization (CO). CO is a distributed algorithm in which a large-scale problem is decomposed into smaller subproblems. Because different local solutions are obtained by solving subproblems, a consensus constraint is imposed to ensure that these solutions are equal, thereby guaranteeing global convergence. We applied the proposed method to linear programming, quadratic programming, and second-order cone programming in numerical experiments and verified its effectiveness.

Data-driven stochastic programming for energy storage system planning in high PV-penetrated distribution network

Energy storage systems (ESSs) facilitate the reliable and economic operation of distribution systems with high PV penetration. Establishing uncertainty models is the key to the optimal planning and operation of ESSs in distribution systems. Widely used parametric models cannot describe the variability of uncertainties thoroughly. In this paper, a data-driven method is designed for uncertainty modeling, and a distributionally robust optimization (DRO) model is developed to determine the optimal ESS planning strategy in distribution systems. First, a deterministic optimization model is established considering both ESS planning and distribution system operation. Then, the Wasserstein-metric-based ambiguity set is designed for the probability distributions of random variables. To hedge against the distributional ambiguity, the optimization problem is expanded into a two-stage DRO problem, of which the second-stage problem minimizes the expected operating cost under the worst-case probability distribution. Finally, the two-stage DRO model is reformulated as a mixed-integer second-order cone programming (MISOCP) problem, which is solved by optimization solvers. The proposed method is tested on modified 33-bus and 123-bus distribution systems with actual solar irradiance and load data. The influence of different strategies on ESS planning and operation is discussed. The ESS planning results obtained by DRO are compared with those of conventional stochastic optimization and deterministic optimization to verify the superiority of the proposed method.

Distributionally Robust Counterfactual Risk Minimization

This manuscript introduces the idea of using Distributionally Robust Optimization (DRO) for the Counterfactual Risk Minimization (CRM) problem. Tapping into a rich existing literature, we show that DRO is a principled tool for counterfactual decision making. We also show that well-established solutions to the CRM problem like sample variance penalization schemes are special instances of a more general DRO problem. In this unifying framework, a variety of distributionally robust counterfactual risk estimators can be constructed using various probability distances and divergences as uncertainty measures. We propose the use of Kullback-Leibler divergence as an alternative way to model uncertainty in CRM and derive a new robust counterfactual objective. In our experiments, we show that this approach outperforms the state-of-the-art on four benchmark datasets, validating the relevance of using other uncertainty measures in practical applications.

Distributionally Robust Learning Based on Dirichlet Process Prior in Edge Networks

In order to meet the real-time performance requirements, intelligent decisions in Internet of things applications must take place right here right now at the network edge. Pushing the artificial intelligence frontier to achieve edge intelligence is nontrivial due to the constrained computing resources and limited training data at the network edge. To tackle these challenges, we develop a distributionally robust optimization (DRO)-based edge learning algorithm, where the uncertainty model is constructed to foster the synergy of cloud knowledge and local training. Specifically, the cloud transferred knowledge is in the form of a Dirichlet process prior distribution for the edge model parameters, and the edge device further constructs an uncertainty set centered around the empirical distribution of its local samples. The edge learning DRO problem, subject to these two distributional uncertainty constraints, is recast as a single-layer optimization problem using a duality approach. We then use an Expectation-Maximization algorithm-inspired method to derive a convex relaxation, based on which we devise algorithms to learn the edge model. Furthermore, we illustrate that the meta-learning fast adaptation procedure is equivalent to our proposed Dirichlet process prior-based approach. Finally, extensive experiments are implemented to showcase the performance gain over standard approaches using edge data only.

An Approximation Scheme for Distributionally Robust Nonlinear Optimization

We consider distributionally robust optimization problems (DROPs) with nonlinear and nonconcave dependence on uncertain parameters. The DROP can be written as a nonsmooth, nonlinear program with a ...

Understanding Distributional Ambiguity via Non-robust Chance Constraint

This paper provides a non-robust interpretation of the distributionally robust optimization (DRO) problem by relating the distributional uncertainties to the chance probabilities. Our analysis allows a decision-maker to interpret the size of the ambiguity set, which is often lack of business meaning, through the chance parameters constraining the objective function. We first show that, for general ϕ-divergences, a DRO problem is asymptotically equivalent to a class of mean-deviation problems. These mean-deviation problems are not subject to uncertain distributions, and the ambiguity radius in the original DRO problem now plays the role of controlling the risk preference of the decision-maker. We then demonstrate that a DRO problem can be cast as a chance-constrained optimization (CCO) problem when a boundedness constraint is added to the decision variables. Without the boundedness constraint, the CCO problem is shown to perform uniformly better than the DRO problem, irrespective of the radius of the ambiguity set, the choice of the divergence measure, or the tail heaviness of the center distribution. Thanks to our high-order expansion result, a notable feature of our analysis is that it applies to divergence measures that accommodate well heavy tail distributions such as the student t-distribution and the lognormal distribution, besides the widely-used Kullback-Leibler (KL) divergence, which requires the distribution of the objective function to be exponentially bounded. Using the portfolio selection problem as an example, our comprehensive testings on multivariate heavy-tail datasets, both synthetic and real-world, shows that this business-interpretation approach is indeed useful and insightful.

Decomposition algorithm for distributionally robust optimization using Wasserstein metric with an application to a class of regression models

Abstract We study distributionally robust optimization (DRO) problems where the ambiguity set is defined using the Wasserstein metric and can account for a bounded support. We show that this class of DRO problems can be reformulated as decomposable semi-infinite programs. We use a cutting-surface method to solve the reformulated problem for the general nonlinear model, assuming that we have a separation oracle. As examples, we consider the problems arising from the machine learning models where variables couple with data in a bilinear form in the loss function. We present a branch-and-bound algorithm to solve the separation problem for this case using an iterative piece-wise linear approximation scheme. We use a distributionally robust generalization of the logistic regression model to test our algorithm. We also show that it is possible to approximate the logistic-loss function with significantly less linear pieces than those needed for a general loss function to achieve a given accuracy when generating a cutting surface. Numerical experiments on the distributionally robust logistic regression models show that the number of oracle calls are typically 20–50 to achieve 5-digit precision. The solution found by the model has better predicting power than classical logistic regression when the sample size is small.