Combinatorial properties of support vectors of separating hyperplanes

Abstract

In this paper we study the relationship between separating hyperplanes and the Radon partitions of their support vectors. This study is relevant for maximal margin separations, which appear in Support Vector Machines (SVM), as well as for separations that optimize a Chebyshev norm. We propose a new version of the Stiefel exchange algorithm where we exploit the property that each Stiefel exchange is in fact a Radon exchange. Originally, the Stiefel exchange algorithm was developed to find Chebyshev approximations, but we show that it is also suited for finding hyperplane separations. We also show that many important properties in approximation theory are closely related to fundamental results in convex set theory, in particular to Helly's, Radon's and Caratheodory's Theorem. Within this context, we prove a new result that generalizes both Radon's and Caratheodory's Theorem.