Sandwich Algorithm Research Articles

Mitigating the impact of outliers in traffic crash analysis: A robust Bayesian regression approach with application to tunnel crash data

Traffic crash datasets are often marred by the presence of anomalous data points, commonly referred to as outliers. These outliers can have a profound impact on the results obtained through the application of traditional methods such as logit and probit models, commonly used in the domain of traffic safety analysis, resulting in biased and unreliable estimates. To mitigate this issue, this study introduces a robust Bayesian regression approach, the robit model, which utilizes a heavy-tailed Student's t distribution to replace the link function of these thin-tailed distributions, effectively reducing the influence of outliers on the analysis. Furthermore, a sandwich algorithm based on data augmentation is proposed to enhance the estimation efficiency of posteriors. The proposed model is rigorously tested using a dataset of tunnel crashes, and the results demonstrate its efficiency, robustness, and superior performance compared to traditional methods. The study also reveals that several factors such as night and speeding have a significant impact on the injury severity of tunnel crashes. This research provides a comprehensive understanding of the outliers treatment methods in traffic safety studies and offers valuable recommendations for the development of appropriate countermeasures to effectively prevent severe injuries in tunnel crashes.

Multi-objective optimization of equation of state molecular parameters: SAFT-VR Mie models for water

The determination of a suitable set of molecular interaction parameters for use with an equation of state (EoS) can be viewed as a multi-objective optimization (MOO) problem, where each objective quantifies the quality of the description for a particular type of thermodynamic property. We outline a methodology for the determination of a set of Pareto-optimal interaction parameters. The Pareto front is generated efficiently using a sandwich algorithm where one solves a sequence of weighted-sum scalarized single objective optimization problems. The algorithm presented can be used for any number of objective functions, allowing for the consideration of multiple thermodynamic property types as competing objectives in the MOO. The methodology is applied to the determination of suitable parameter sets for models of water within the SAFT-VR Mie framework. Three competing property targets are considered as objective functions: saturated-liquid density, vapour pressure and isobaric heat capacity. Two different types of molecular models are considered: spherical models of water, and non-spherical model of water. We analyse the two- and three-dimensional Pareto surfaces and parameter sets obtained for different property combinations in the MOO. The proposed methodology can be used to provide a rigorous comparison between different model types. Numerous Pareto-optimal parameter sets for SAFT-VR Mie water models are documented, and we recommend two new models (one spherical model and one non-spherical model) with an appropriate compromise between the competing objectives.

Multi-attribute based influence maximization in social networks: Algorithms and analysis

The most valuable feature of social networks is that they can generate contents for users and spread them quickly on the network, which is a very important platform for viral marketing. Most of the related work on viral marketing focuses on the spread of single information, while a product may associate with multiple attributes in real life. Information about multiple attributes of a product propagates in the social networks simultaneously and independently. The attribute information that a user receives will determine whether he would purchase the product or not. We extend the traditional single information influence maximization problem to the multi-attribute based influence maximization problem. We also present the Multi-dimensional IC model (MIC model) for the proposed problem, then formulate the problem as the Multi-attribute based Influence Maximization Problem (MIMP). The objective function for MIMP is proved to be non-submodular, then we solve the problem with two different algorithms: the Sandwich Algorithm and the Supermodular Algorithm, whose solutions can get a max{f(SU)f‾(SU),f_(SL⁎)f(So⁎)}(1−1/e) approximation ratio and an 1/(d+2) approximation ratio to the optimal solution, respectively. Experiments based on the real world social network datasets verify the effectiveness and correctness of our proposed solutions.

Linearization of McCormick relaxations and hybridization with the auxiliary variable method

The computation of lower bounds via the solution of convex lower bounding problems depicts current state-of-the-art in deterministic global optimization. Typically, the nonlinear convex relaxations are further underestimated through linearizations of the convex underestimators at one or several points resulting in a lower bounding linear optimization problem. The selection of linearization points substantially affects the tightness of the lower bounding linear problem. Established methods for the computation of such linearization points, e.g., the sandwich algorithm, are already available for the auxiliary variable method used in state-of-the-art deterministic global optimization solvers. In contrast, no such methods have been proposed for the (multivariate) McCormick relaxations. The difficulty of determining a good set of linearization points for the McCormick technique lies in the fact that no auxiliary variables are introduced and thus, the linearization points have to be determined in the space of original optimization variables. We propose algorithms for the computation of linearization points for convex relaxations constructed via the (multivariate) McCormick theorems. We discuss alternative approaches based on an adaptation of Kelley’s algorithm; computation of all vertices of an n-simplex; a combination of the two; and random selection. All algorithms provide substantial speed ups when compared to the single point strategy used in our previous works. Moreover, we provide first results on the hybridization of the auxiliary variable method with the McCormick technique benefiting from the presented linearization strategies resulting in additional computational advantages.

A comparative study of multi-objective optimization methodologies for molecular and process design

The need to consider multiple objectives in molecular design, whether based on techno-economic, environmental or health and safety metrics is increasingly recognized. There is, however, limited understanding of the suitability of different multi-objective optimization (MOO) algorithm for the solution of such design problems. In this work, we present a systematic comparison of the performance of five mixed-integer non-linear programming (MINLP) MOO algorithms on the selection of computer-aided molecular design (CAMD) and computer-aided molecular and process design (CAMPD) problems. The five methods are designed to address the discrete and nonlinear nature of the problem, with the aim of generating an accurate approximation of the Pareto front. They include: a weighted sum approach without global search phases (SWS), a weighted sum approach with simulated annealing (WSSA), a weighted sum approach with multi level single linkage (WSML), the sandwich algorithm with MLSL (SDML) and the non dominated sorting genetic algorithm-II (NSGA-II). The algorithms are compared systematically in two steps. The effectiveness of the global search methods is evaluated with SWS, WSSA and WSML. WSML is found to be most effective and a comparative analysis of WSML, SDML and NSGA-II is then undertaken. As a test set of these optimization techniques, two CAMD and one CAMPD problems of varying dimensionality are formulated as case studies. The results show that the SDML provides the most efficient generation of a diverse set of Pareto points, leading to the construction of an approximate Pareto front close to exact Pareto front.

Learning Mixtures of Random Utility Models

We tackle the problem of identifiability and efficient learning of mixtures of Random Utility Models (RUMs). We show that when the PDFs of utility distributions are symmetric, the mixture of k RUMs (denoted by k-RUM) is not identifiable when the number of alternatives m is no more than 2k-1. On the other hand, when m ≥ max{4k-2,6}, any k-RUM is generically identifiable. We then propose three algorithms for learning mixtures of RUMs: an EM-based algorithm, which we call E-GMM, a direct generalized-method-of-moments (GMM) algorithm, and a sandwich (GMM-E-GMM) algorithm that combines the other two. Experiments on synthetic data show that the sandwich algorithm achieves the highest statistical efficiency and GMM is the most computationally efficient. Experiments on real-world data at Preflib show that Gaussian k-RUMs provide better fitness than a single Gaussian RUM, the Plackett-Luce model, and mixtures of Plackett-Luce models w.r.t. commonly-used model fitness criteria. To the best of our knowledge, this is the first work on learning mixtures of general RUMs.

Hybrid Stochastic-Deterministic Multiperiod DC Optimal Power Flow

Stochastic optimal power flow (OPF) formulations that minimize the expected operating costs over forecast scenarios generally result in lower costs than the standard (deterministic) OPF problem for power systems with significant forecast error, for example, from renewable energy sources. However, this type of stochastic OPF problem is more computationally demanding than the deterministic OPF, and even more so when storage units or ramp-constrained generators are included, as they require solving a multi-period OPF problem. We propose a hybrid method approaching the cost performance of the stochastic OPF problem and the computational burden of the deterministic OPF problem. Our method decomposes the problem into stochastic and deterministic subproblems, and relies on Benders’ Cuts to interface them. We present three versions of the method, which achieve different cost/computational burden trade-offs. The versions can be parametrized so that they scale well with the problem dimension. For one of the versions, we develop a multi-dimensional formulation of the Sandwich Algorithm , which is used to iteratively approximate a convex function. Through a case study using a 118-bus system, we find that our hybrid method achieves between 25% and 81% of the cost improvement of the stochastic OPF, but requires only 12 to 41% of the increased computation time required by the stochastic OPF. Both the hybrid method and the multi-dimensional Sandwich Algorithm can be used for problems outside of the field of power systems.

Convergence properties of Gibbs samplers for Bayesian probit regression with proper priors

The Bayesian probit regression model (Albert and Chib (1993)) is popular and widely used for binary regression. While the improper flat prior for the regression coefficients is an appropriate choice in the absence of any prior information, a proper normal prior is desirable when prior information is available or in modern high dimensional settings where the number of coefficients ($p$) is greater than the sample size ($n$). For both choices of priors, the resulting posterior density is intractable and a Data Dugmentation (DA) Markov chain is used to generate approximate samples from the posterior distribution. Establishing geometric ergodicity for this DA Markov chain is important as it provides theoretical guarantees for constructing standard errors for Markov chain based estimates of posterior quantities. In this paper, we first show that in case of proper normal priors, the DA Markov chain is geometrically ergodic *for all* choices of the design matrix $X$, $n$ and $p$ (unlike the improper prior case, where $n \geq p$ and another condition on $X$ are required for posterior propriety itself). We also derive sufficient conditions under which the DA Markov chain is trace-class, i.e., the eigenvalues of the corresponding operator are summable. In particular, this allows us to conclude that the Haar PX-DA sandwich algorithm (obtained by inserting an inexpensive extra step in between the two steps of the DA algorithm) is strictly better than the DA algorithm in an appropriate sense.

A novel sandwich algorithm for empirical Bayes analysis of rank data

Rank data arises frequently in marketing, finance, organizational behavior, and psychology. Most analysis of rank data reported in the literature assumes the presence of one or more variables (sometimes latent) based on whose values the items are ranked. In this paper we analyze rank data using a purely probabilistic model where the observed ranks are assumed to be perturbed versions of the true rank and each perturbation has a specific probability of occurring. We consider the general case when covariate information is present and has an impact on the rankings. An empirical Bayes approach is taken for estimating the model parameters. The Gibbs sampler is shown to converge very slowly to the target posterior distribution and we show that some of the widely used empirical convergence diagnostic tools may fail to detect this lack of convergence. We propose a novel, fast mixing sandwich algorithm for exploring the posterior distribution. An EM algorithm based on Markov chain Monte Carlo (MCMC) sampling is developed for estimating prior hyperparameters. A real life rank data set is analyzed using the methods developed in the paper. The results obtained indicate the usefulness of these methods in analyzing rank data with covariate information.

A comparison theorem for data augmentation algorithms with applications

The data augmentation (DA) algorithm is considered a useful Markov chain Monte Carlo algorithm that sometimes suffers from slow convergence. It is often possible to convert a DA algorithm into a sandwich algorithm that is computationally equivalent to the DA algorithm, but converges much faster. Theoretically, the reversible Markov chain that drives the sandwich algorithm is at least as good as the corresponding DA chain in terms of performance in the central limit theorem and in the operator norm sense. In this paper, we use the sandwich machinery to compare two DA algorithms. In particular, we provide conditions under which one DA chain can be represented as a sandwich version of the other. Our results are used to extend Hobert and Marchev’s (2008) results on the Haar PX-DA algorithm and to improve the collapsing theorem of Liu et al. (1994) and Liu (1994). We also illustrate our results using Brownlee’s (1965) stack loss data.

Approximation in p-Norm of Univariate Concave Functions

We derive worst-case bounds, with respect to the L p norm, on the error achieved by algorithms aimed at approximating a concave function of a single variable, through the evaluation of the function and its subgradient at a fixed number of points to be determined. We prove that, for p larger than 1, adaptive algorithms outperform passive ones. Next, for the uniform norm, we propose an improvement of the Sandwich algorithm, based on a dynamic programming formulation of the problem.

Analysis of MCMC algorithms for Bayesian linear regression with Laplace errors

Let π denote the intractable posterior density that results when the standard default prior is placed on the parameters in a linear regression model with iid Laplace errors. We analyze the Markov chains underlying two different Markov chain Monte Carlo algorithms for exploring π. In particular, it is shown that the Markov operators associated with the data augmentation (DA) algorithm and a sandwich variant are both trace-class. Consequently, both Markov chains are geometrically ergodic. It is also established that for each i∈{1,2,3,…}, the ith largest eigenvalue of the sandwich operator is less than or equal to the corresponding eigenvalue of the DA operator. It follows that the sandwich algorithm converges at least as fast as the DA algorithm.

Flowpipe-Guard Intersection for Reachability Computations with Support Functions

Recently, efficient reachability algorithms for hybrid systems with piecewise affine dynamics have been developed. They achieve good scalability and precision by using support functions to represent continuous sets. In this paper, we propose an improvement of these algorithms that reduces the overapproximation error of the image computation of discrete transitions (jumps). The critical operation of this image computation is the intersection of the flowpipe with the guard sets of the transitions, since intersection is in general a difficult operation when using support functions. We propose an approach for computing the intersection of the flowpipe with polyhedral guards up to arbitrary accuracy. We reduce computing the support function of the intersection of a single convex set with a guard to a convex minimization problem. To solve it, we present a custom-tailored sandwich algorithm. The intersection of a flowpipe (a sequence of convex sets) with a guard reduces to a set of such minimization problems. Where possible, we use branch-and-bound techniques and solve these minimization problems simultaneously to avoid redundant computations. Experimental results illustrate the gain in accuracy and the performance of the algorithms.

A spectral analytic comparison of trace-class data augmentation algorithms and their sandwich variants

The data augmentation (DA) algorithm is a widely used Markov chain Monte Carlo algorithm that is easy to implement but often suffers from slow convergence. The sandwich algorithm is an alternative that can converge much faster while requiring roughly the same computational effort per iteration. Theoretically, the sandwich algorithm always converges at least as fast as the corresponding DA algorithm in the sense that $\Vert {K^*}\Vert \le \Vert {K}\Vert$, where $K$ and $K^*$ are the Markov operators associated with the DA and sandwich algorithms, respectively, and $\Vert\cdot\Vert$ denotes operator norm. In this paper, a substantial refinement of this operator norm inequality is developed. In particular, under regularity conditions implying that $K$ is a trace-class operator, it is shown that $K^*$ is also a positive, trace-class operator, and that the spectrum of $K^*$ dominates that of $K$ in the sense that the ordered elements of the former are all less than or equal to the corresponding elements of the latter. Furthermore, if the sandwich algorithm is constructed using a group action, as described by Liu and Wu [J. Amer. Statist. Assoc. 94 (1999) 1264--1274] and Hobert and Marchev [Ann. Statist. 36 (2008) 532--554], then there is strict inequality between at least one pair of eigenvalues. These results are applied to a new DA algorithm for Bayesian quantile regression introduced by Kozumi and Kobayashi [J. Stat. Comput. Simul. 81 (2011) 1565--1578].

Spectral analytic comparisons for data augmentation

The sandwich algorithm (SA) is an alternative to the data augmentation (DA) algorithm that uses an extra simulation step at each iteration. In this paper, we show that the sandwich algorithm always converges at least as fast as the DA algorithm, in the Markov operator norm sense. We also establish conditions under which the spectrum of SA dominates that of DA. An example illustrates the results.

Derivative-free generation and interpolation of convex Pareto optimal IMRT plans

In inverse treatment planning for intensity-modulated radiation therapy (IMRT), beamlet intensity levels in fluence maps of high-energy photon beams are optimized. Treatment plan evaluation criteria are used as objective functions to steer the optimization process. Fluence map optimization can be considered a multi-objective optimization problem, for which a set of Pareto optimal solutions exists: the Pareto efficient frontier (PEF). In this paper, a constrained optimization method is pursued to iteratively estimate the PEF up to some predefined error. We use the property that the PEF is convex for a convex optimization problem to construct piecewise-linear upper and lower bounds to approximate the PEF from a small initial set of Pareto optimal plans. A derivative-free Sandwich algorithm is presented in which these bounds are used with three strategies to determine the location of the next Pareto optimal solution such that the uncertainty in the estimated PEF is maximally reduced. We show that an intelligent initial solution for a new Pareto optimal plan can be obtained by interpolation of fluence maps from neighbouring Pareto optimal plans. The method has been applied to a simplified clinical test case using two convex objective functions to map the trade-off between tumour dose heterogeneity and critical organ sparing. All three strategies produce representative estimates of the PEF. The new algorithm is particularly suitable for dynamic generation of Pareto optimal plans in interactive treatment planning.

Error bounds for the approximative solution of restricted planar location problems

Facility location problems in the plane play an important role in mathematical programming. When looking for new locations in modeling real-world problems, we are often confronted with forbidden regions, that are not feasible for the placement of new locations. Furthermore these forbidden regions may have complicated shapes, even if we require them to be convex. It may be more useful or even necessary to use approximations of such forbidden regions when trying to solve location problems. In this paper, we develop error bounds for the approximative solution of restricted planar location problems using the so called sandwich algorithm. The number of approximation steps required to achieve a specified error bound is analyzed. As examples of these approximation schemes, we discuss round norms and polyhedral norms. Computational tests are also included.

The convergence rate of the sandwich algorithm for approximating convex functions

The Sandwich algorithm approximates a convex function of one variable over an interval by evaluating the function and its derivative at a sequence of points. The connection of the obtained points is a piecewise linear upper approximation, and the tangents yield a piecewise linear lower approximation. Similarly, a planar convex figure can be approximated by convex polygons.