Combinatorial variability of vapnik-chervonenkis classes with applications to sample compression schemes

Abstract

We define embeddings between concept classes that are meant to reflect certain aspects of their combinatorial structure. Furthermore, we introduce a notion of universal concept classes - classes into which any member of a given family of classes can be embedded. These universal classes play a role similar to that played in computational complexity by languages that are hard for a given complexity class. We show that classes of half-spaces in R n are universal with respect to families of algebraically defined classes. We present some combinatorial parameters along which the family of classes of a given VC-dimension can be grouped into sub-families. We use these parameters to investigate the existence of embeddings and the scope of universality of classes. We view the formulation of these parameters and the related questions that they raise as a significant component in this work. A second theme in our work is the notion of sample compression schemes. Intuitively, a class C has a sample compression scheme if for any finite sample, labeled according to a member of C, there exists a short sub-sample so that the labels of the full sample can be reconstructed from this sub-sample. By demonstrating the existence of certain compression schemes for the classes of half-spaces the existence of similar compression schemes for every class embeddable in half-spaces readily follows. We apply this approach to prove the existence of compression schemes for all ‘geometric concept classes’.