Improving the wisdom of crowds with analysis of variance of predictions of related outcomes

Abstract

Decision-makers often collect and aggregate experts’ point predictions about continuous outcomes, such as stock returns or product sales. In this article, we model experts as Bayesian agents and show that means, including the (weighted) arithmetic mean, trimmed means, median, geometric mean, and essentially all other measures of central tendency, do not use all information in the predictions. Intuitively, they assume idiosyncratic differences to arise from error instead of private information and hence do not update the prior with all available information. Updating means in terms of unused information improves their expected accuracy but depends on the experts’ prior and information structure that cannot be estimated based on a single prediction per expert. In many applications, however, experts consider multiple stocks, products, or other related items at the same time. For such contexts, we introduce ANOVA updating – an unsupervised technique that updates means based on experts’ predictions of multiple outcomes from a common population. The technique is illustrated on several real-world datasets.