New Approaches for Choosing the Ridge Parameters

Abstract

Consider the standard multiple linear regression model $y=x\beta+\varepsilon.$ If the correlation matrix $x^tx$ is ill-conditioned, the ordinary least squared estimate (ols) $\hat{\beta}$ of $\beta$ is not the best choice. In this paper, multiple regularization parameters for different coefficients in ridge regression are proposed. The Mean Squared Error (MSE) of a ridge estimate based on the multiple regularization parameters is less than or equal to the MSE of the ridge estimate based on Hoerl and Kennard, 1970. The proposed approach, depending on the condition numbers, leave's zero for the largest eigenvalue of $x^tx$ and gives the largest value for the smallest eigenvalue of $x^tx.$ Furthermore, if $x^tx$ is nearly a unit matrix, $x^tx$ is not an ill-conditioned one. The proposed approach gives approximately the same results as the ols estimates. The proposed approach can also be modified to give other new ridge parameters. The modified approach depends on the eigenvalues of $x^tx$ and differ from the ridge parameter proposed by Khalaf and Shukur by a factor. The body fat data set has severe multicollinearity and is used to compare different approaches.