On the Performance of Jackknife Based Estimators for Ridge Regression

Abstract

Regression techniques are generally used to predict a response variable using one or more predictor variables. In many fields of study, the regressors can be highly intercorrelated, which leads to the problem of multicollinearity. Consequently, the ordinary least squares estimates become inconsistent and lead to wrong inferences. To handle the problem, machine learning techniques particularly, the ridge regression approach, are commonly used. In this paper, we revisit the problem of estimating the ridge parameter “ ${k}$ ” by proposing some new estimators using the Jackknife method and compare them with some existing estimators. The performance of the proposed estimators compared to the existing ones is evaluated using extensive Monte Carlo simulations as well as two real data sets. The results suggested that the proposed estimators outperform the existing estimators.