Vapnik-Chervonenkis, Pseudo- and Fat-Shattering Dimensions

Abstract

In this chapter, we introduce three distinct notions of “dimension” that play an important role in the subsequent development. The phrase “dimension” is rather unfortunate, as the three “dimensions” have nothing at all to do with the dimension of a vector space, except in very special situations. Rather, these “dimensions” are combinatorial parameters that measure the “richness” of concept classes or function classes. The Vapnik-Chervonenkis dimension, often referred to as the VC-dimension, is historically the first dimension to be introduced into the subject, and is defined for concept classes, or equivalently, binary-valued functions. The Pseudo-dimension, also referred to by some authors as the Pollard dimension, is a generalization of the VC-dimension to real-valued functions. The fat-shattering dimension, unlike the Pseudo-dimension, is a “scale-sensitive” measure of richness. All three of these dimensions are used in deriving conditions for the uniform convergence of empirical means and for PAC learnability.