Limitations of Ensemble Bayesian Model Averaging for forecasting social science problems

Abstract

We compare the accuracies of simple unweighted averages and Ensemble Bayesian Model Averaging (EBMA) for combining forecasts in the social sciences. A review of prior studies from the domain of economic forecasting finds that the simple average was more accurate than EBMA in four studies out of five. On average, the error of EBMA was 5% higher than that of the simple average. A reanalysis and extension of a published study provides further evidence for US presidential election forecasting. The error of EBMA was 33% higher than the corresponding error of the simple average. Simple averages are easy both to describe and to understand, and thus are easy to use. In addition, simple averages provide accurate forecasts in many settings. Researchers who are developing new approaches to combining forecasts need to compare the accuracy of their method to this widely established benchmark. Forecasting practitioners should favor simple averages over more complex methods unless there is strong evidence in support of differential weights.