Normalization Ridge Regression in Practice

Abstract

Both ordinary least squares (OLS) and two-stage least squares (2SLS) regression methods are sensitive to multicollinearity. The standard statistical solution to the multicollinearity problem is to use one of a family of biased, variance-reduced estimation methods collectively known as ridge regression (RR). In the presence of multicollinearity, RR is usually more efficient than OLS; thus, in theory, two-stage ridge regression (2SRR) should be able to outperform 2SLS. RR, however, is not a problem-free method for reducing variance inflation. It is a stochastic procedure when it should be nonstochastic, and it does not meet the boundary condition. It is argued that the problems of RR stem from the use of the minimum mean square error criterion. An alternative called the variance normalization criterion is proposed that, in theory, copes with the dilemmas of RR while preserving the advantages of RR over OLS. Normalization ridge regression (NR) is used to estimate a nonorthogonal recursive model, and two-stage normalization ridge regression (2SNR) is used to estimate a nonorthogonal, nonrecursive model. Comparisons were made between OLS, RR, and NR estimates and between 2SLS, 2SRR, and 2SNR. On the basis of eight performance indices it was concluded that the NR and 2SNR procedures were efficient variance reduction methods, and hence, useful statistical solutions in the last resort to the multicollinearity problem.