Improved robust ridge regression estimates

Abstract

In multiple linear regression when the predictors are strongly correlated, the least-squares estimates (LSE) usually provide inaccurate predictions. Ridge regression, based on the minimization of a quadratic loss function, is sensitive to outliers. Two smoothly redescending ψ-functions based on the Winsor's principle, which lead to asymptotically efficient estimates were considered. The method of iteratively reweighted least squares (IRLS) based on the proposed ψ-functions can be used to produce the resulting robust ridge estimates for identifying outliers and ignoring zero-weight outliers. Examples, selected from the relevant literature, are used for illustrative purposes. It is possible to obtain convergence to the final estimates of the coefficients with fewer iterations than without using ridge regression. The combined robust and ridge estimates result in stable coefficients and balances that help in determining the true coefficients and outliers.