Model Selection in Regression: Statistical and Scientific Perspectives

Abstract

Abstract Model selection in regression and the choice of what might be considered a best model is often fraught with difficulty. This article briefly reviews select topics in model and variable selection, both from a statistical and scientific point of view. Topics discussed include the importance of selecting good candidate predictors; simultaneous inference in regression; best subsets selection; penalized fit statistics and ridge regression; lasso; backward, forward, and stepwise algorithms; and statistical versus pragmatic model selection. It is concluded that in addition to statistical criteria, substantive utility and usability of a model should also be incorporated into the final model selection.