The impact of collinearity involving the intercept term on the numerical accuracy of regression

Abstract

It is well known that multiple linear regression models with ill-conditioning can produce coefficient estimates with degraded numerical accuracy. This study examines the numerical accuracy of regression algorithms in the presence a particular type of ill-conditioning, that arising from collinear relationships that involve the intercept term and the independent variables. A benchmark data set is used to produce ill-conditioned data by introducing near linear relationships among the independent variables and the intercept term. The experiments reported here demonstrate that centering does not prevent a loss in numerical accuracy for this particular type of ill-conditioning. In addition, the ability of commonly used diagnostic checks to detect these problems is studied. As an example of the problems that arise from ignoring the relationships studied here we demonstrate that the regression procedures in two widely used statistical packages, SAS and SPSS-X, fail to detect this type of ill-conditioning and report highly inaccurate coefficient estimates.