Testability and Statistical Learning Theory

Abstract

Statistical learning theory has subsequently expanded into a more general theory of learning from statistical data. Statistical learning theory is of philosophical interest in a number of ways, but especially because one of its central concepts, Vapnik-Chernovenkis (VC) dimension, bears more than a passing resemblance to Karl Popper's notion of testable or falsifiable theories. This chapter explores this connection with an emphasis on the underlying motivations of the two concepts. The concept of VC dimension is involved in two central results in statistical learning theory: the first identifies finite VC dimension as a necessary condition for long run convergence, while the second shows how a preference for lower VC dimension can improve “the rate of convergence.” Popper's falsificationism is a staunchly scientific realist perspective driven by the goal of enhancing the efficiency of scientific progress towards truth. Statistical learning theory aims to minimize the expected errors of predictions. The similarity of the concepts of VC and Popper dimension, therefore, raises some intriguing questions about the connection between predictive accuracy and efficient convergence to the truth.