On Bayesian Conjugate Normal Linear Regression and Ordinary Least Square Regression Methods: A Monte Carlo Study

Abstract

In this study,  comparison between the classical ordinary least square (OLS) regression technique and the Bayesian conjugate normal linear regression method when the data satisfy all the necessary assumptions of OLS technique is presented. The Bayesian normal linear regression model was fitted using Normal-Gamma conjugate prior. Results from Monte Carlo study showed that the OLS estimator is as good as the Bayesian estimator in terms of the closeness of their estimated parameters to the true values. However, using the criteria of the mean square errors of parameters’ estimates and other performance indices, the results showed that Bayesian estimator is more efficient,  more consistent and relatively more stable than the classical least squares method even when the sample data satisfy all the necessary assumptions of the OLS method. The apparent better performance of Bayesian estimator over the OLS is justified by the prior information about the data that Bayesian technique employed in its estimation. Therefore, it could be concluded that if reliable information about the data under investigation is available, Bayesian regression technique(s) that would make use of such information should be preferred for efficient model’s estimation and better inference. The R statistical package (www.cran.org) was employed for all the analysis in this study.