Improved Empirical Bayes Ridge Regression Estimators Under Multicollinearity

Abstract

In this paper, we consider the problem of estimating the regression parameters in a multiple linear regression model when the multicollinearity is present. Under the assumption of normality, we present three empirical Bayes estimators. One of them shrinks the least squares (LS) estimator towards the principal component. The second one is a hierarchical empirical Bayes estimator shrinking the LS estimator twice. The third one is obtained by choosing different priors for the two sets of regression parameters that arise in the case of multicollinearity; this estimator is termed decomposed empirical Bayes estimator. These proposed estimators are not only proved to be uniformly better than the LS estimator, that is, minimax in terms of risk under the Strawderman loss function, but also shown to be useful in the multicollinearity cases through simulation and empirical studies.