Model averaging estimator in ridge regression and its large sample properties

Abstract

In linear regression, when the covariates are highly collinear, ridge regression has become the standard treatment. The choice of ridge parameter plays a central role in ridge regression. In this paper, instead of ending up with a single ridge parameter, we consider a model averaging method to combine multiple ridge estimators with $$M_n$$ different ridge parameters, where $$M_n$$ can go to infinity with sample size n. We show that when the fitting model is correctly specified, the resulting model averaging estimator is $$n^{1/2}$$ -consistent. When the fitting model is misspecified, the asymptotic optimality of the model averaging estimator is also established rigorously. The results of simulation studies and our case study concerning the urbanization level of Chinese ethnic areas demonstrate the usefulness of the model averaging method.