Decomposition algorithm for distributionally robust optimization using Wasserstein metric with an application to a class of regression models

Abstract

Abstract We study distributionally robust optimization (DRO) problems where the ambiguity set is defined using the Wasserstein metric and can account for a bounded support. We show that this class of DRO problems can be reformulated as decomposable semi-infinite programs. We use a cutting-surface method to solve the reformulated problem for the general nonlinear model, assuming that we have a separation oracle. As examples, we consider the problems arising from the machine learning models where variables couple with data in a bilinear form in the loss function. We present a branch-and-bound algorithm to solve the separation problem for this case using an iterative piece-wise linear approximation scheme. We use a distributionally robust generalization of the logistic regression model to test our algorithm. We also show that it is possible to approximate the logistic-loss function with significantly less linear pieces than those needed for a general loss function to achieve a given accuracy when generating a cutting surface. Numerical experiments on the distributionally robust logistic regression models show that the number of oracle calls are typically 20–50 to achieve 5-digit precision. The solution found by the model has better predicting power than classical logistic regression when the sample size is small.