PRINCIPAL COMPONENTS TO OVERCOME MULTICOLLINEARITY PROBLEM

Abstract

The impact of ignoring collinearity among predictors is well documented in a statistical literature. An attempt has been made in this study to document application of Principal components as remedial solution to this problem. Using a sample of six hundred participants, linear regression model was fitted and collinearity between predictors was detected using Variance Inflation Factor (VIF). After confirming the existence of high relationship between independent variables, the principal components was utilized to find the possible linear combination of variables that can produce large variance without much loss of information. Thus, the set of correlated variables were reduced into new minimum number of variables which are independent on each other but contained linear combination of the related variables. In order to check the presence of relationship between predictors, dependent variables were regressed on these five principal components. The results show that VIF values for each predictor ranged from 1 to 3 which indicates that multicollinearity problem was eliminated. Finally another linear regression model was fitted using Principal components as predictors. The assessment of relationship between predictors indicated that no any symptoms of multicollinearity were observed. The study revealed that principal component analysis is one of the appropriate methods of solving the collinearity among variables. Therefore this technique produces better estimation and prediction than ordinary least squares when predictors are related. The study concludes that principal component analysis is appropriate method of solving this matter.