Least trimmed squares regression, least median squares regression, and mathematical programming

Abstract

In this paper, we study LTS and LMS regression, two high breakdown regression estimators, from an optimization point of view. We show that LTS regression is a nonlinear optimization problem that can be treated as a concave minimization problem over a polytope. We derive several important properties of the corresponding objective function that can be used to obtain algorithms for the exact solution of LTS regression problems, i.e., to find a global optimum to the problem. Because of today's limited problem-solving capabilities in exact concave minimization, we give an easy-to-implement pivoting algorithm to determine regression parameters corresponding to local optima of the LTS regression problem. For the LMS regression problem, we briefly survey the existing solution methods which are all based on enumeration. We formulate the LMS regression problem as a mixed zero-one linear programming problem which we analyze in depth to obtain theoretical insights required for future algorithmic and computational work.