Effect of Multicollinearity on Variable Selection in Multiple Regression

Abstract

When Multicollinearity exists in a data set, the data is considered deficient. Multicollinearity is frequently encountered in observational studies. It creates difficulties when building regression models. It is a phenomenon whereby two or more explanatory variable in a multiple regression model are highly correlated. Variable selection is an important aspect of model building as such the choice of the best subset among many variables to be included in a model is the most difficult part of model building in regression analysis. Data was obtained from Nigerian Stock Exchange Fact Book, Nigerian Stock Exchange Annual Report and Account, CBN Statistical Bulletin and FOS Statistical bulletin from 1987 to 2018. Variance Inflation Factor (VIF) and correlation matrices were used to detect the presence of multicollinearity. Ridge regression and Least Square Regression were applied using R-package, Minitab and SPSS Packages. Ridge Models with constant range of 0.01 ≤ K ≤ 1.5 and Least Square Regression models were considered for each value of P = 2, 3, …,7. The optimal Ridge and Least Square model from the Ridge and Least Square Regression models were obtained by taking the average rank of the Coefficient of Determination and Mean Square Error. The result showed that the choices of variable selection were affected by the presence of multicollinearity as different variables were selected under Ridge and Least Square Regression for same level of P.