A new algorithm for variable selection

Abstract

A new method for variable selection and estimation called Iteratively Scaled Ridge Regression, ISRR, is proposed. The method is an iterative algorithm based on ridge regression. Simulation studies show that ISRR shares the properties of both subset selection and ridge regression. It selects an optimal subset of the regressor variables and is robust to small changes in the data set. The ISRR algorithm was primarily developed for linear models, but is quite simple and general and can easily be extended to more general linear and nonlinear models.