Comparison of Classical Least Squares (CLS), Ridge and Principal Component Methods of Regression Analyses using Gynecological Data

Abstract

This study compares results obtained from the application of Classical Least Squares with that obtained from the two major biased estimation methods: Ridge and Principal Component Regressions in multicollinear situations using gynecological data from University College Hospital, Ibadan, Oyo State, Nigeria. Numerical values of baby's weights (less than 2.5kg) at birth were considered as response variable while mother's age, weight and height, as well as preterm delivery, multiple pregnancies, parity, graphidity, maternal infections such as malaria, tuberculosis, sexually transmitted diseases, anaemia/shortage of blood, intra-uterine infections, congenital abnormalities, etc and fetal infections serve as explanatory variables. Regression method is used as the statistical tool. A number of assumptions of the regression analysis were inspected. Normality assumption was confirmed by plotting Normal Q-Q Plot and Histogram of the Standardized Residuals. The data sets were also inspected for homoscedasticity of error variances using Residual Plot and Fligner-Killeen Test; it was established that homoscedasticity assumption was not violated. Autocorrelation problem on the data set was checked by Durbin-Watson statistic. The test revealed that autocorrelation problem can be tolerated. Existence of multicollinearity problem was further checked in the data set using Farrar-Glauber Chi-squared test. It was established that some predictors are highly correlated; this was also true when correlation matrix table was obtained. Shrinkage estimator of Ridge Regression was obtained by Iterative Method of Hoerl and Kennard. Ridge Regression coefficients were later computed. Kaiser's and Cattell's Screen Criteria were employed for determining number of principal components to be retained in the analysis. The two criteria suggested that the first three components should be retained but only one component is significant. Thereafter, Ridge Regression was finally recommended as the best method to handle multicollinearity problem under Frequentist Approach.