Bootstrap selection of ridge regularization parameter: a comparative study via a simulation study

Abstract

In multiple linear regressions, it is known that least-squares estimates of the parameters are likely to be too large in absolute value and possibly of wrong sign, if explanatory variables are correlated. To reduce the undesirable effects of collinearity, the ridge estimator has been proposed as an alternative method to the least squares estimator. The biggest debate with the ridge estimator is the selection of the regularization parameter. Several methods for the selection of the regularization parameter have been discussed. Although most of these methods are based on minimizing the mean square error of the ridge estimator, bootstrap resampling method is also proposed to provide an optimal regularization parameter which gives an estimate of mean square error of prediction. The purpose of this paper is to provide a comprehensive study of regularization parameter selection methods which consider bootstrap approach as well as estimation and prediction approaches. The paper concludes with application of these procedures to simulation studies and real data.