ROBUST RIDGE REGRESSION BASED ON AN M‐ESTIMATOR

Abstract

SummaryConsider the linear regression model y=β01 +Xβ+ in the usual notation. It is argued that the class of ordinary ridge estimators obtained by shrinking the least squares estimator by the matrix (X1X + kI)‐1X'X is sensitive to outliers in the ^variable. To overcome this problem, we propose a new class of ridge‐type M‐estimators, obtained by shrinking an M‐estimator (instead of the least squares estimator) by the same matrix. Since the optimal value of the ridge parameter k is unknown, we suggest a procedure for choosing it adaptively. In a reasonably large scale simulation study with a particular M‐estimator, we found that if the conditions are such that the M‐estimator is more efficient than the least squares estimator then the corresponding ridge‐type M‐estimator proposed here is better, in terms of a Mean Squared Error criteria, than the ordinary ridge estimator with k chosen suitably. An example illustrates that the estimators proposed here are less sensitive to outliers in the y‐variable than ordinary ridge estimators.