Simplicity and model selection

Abstract

In this paper I compare parametric and nonparametric regression models with the help of a simulated data set. Doing so, I have two main objectives. The first one is to differentiate five concepts of simplicity and assess their respective importance. The second one is to show that the scope of the existing philosophical literature on simplicity and model selection is too narrow because it does not take the nonparametric approach into account (Sober in Philosophy of Science 69(S3), S112–S123, 2002; Forster and Sober in The British Journal for the Philosophy of Science 45(1), 1–35, 1994; Forster, 2001, in Philosophy of Science 74(5), 588–600, 2007; Hitchcock and Sober in The British Journal for the Philosophy of Science 55(1), 1–34, 2004; Mikkelson in Philosophy of Science 73(4), 440–447, 2006; Baker 2013). More precisely, I point out that a measure of simplicity in terms of the number of adjustable parameters is inadequate to characterise nonparametric models and to compare them with parametric models. This allows me to weed out false claims about what makes a model simpler than another. Furthermore, I show that the importance of simplicity in model selection cannot be captured by the notion of parametric simplicity. ‘Simplicity’ is an umbrella term. While parametric simplicity can be ignored, there are other notions of simplicity that need to be taken into consideration when we choose a model. Such notions are not discussed in the previously mentioned literature. The latter therefore portrays an incomplete picture of why simplicity matters when we choose a model. Overall I support a pluralist view according to which we cannot give a general and interesting (epistemic or pragmatic) justification for the importance of simplicity in science.