A Bayesian Hierarchical Model of Crowd Wisdom Based on Predicting Opinions of Others

Abstract

In many domains, it is necessary to combine opinions or forecasts from multiple individuals. However, the average or modal judgment is often incorrect, shared information across respondents can result in correlated errors, and weighting judgments by confidence does not guarantee accuracy. We develop a Bayesian hierarchical model of crowd wisdom that incorporates predictions about others to address these aggregation challenges. The proposed model can be applied to single questions, and it can also estimate respondent expertise given multiple questions. Unlike existing Bayesian hierarchical models for aggregation, the model does not link the correct answer to consensus or privilege majority opinion. The model extends the “surprisingly popular algorithm” to enable statistical inference and in doing so, overcomes several of its limitations. We assess performance on empirical data and compare the results with other aggregation methods, including leading Bayesian hierarchical models. This paper was accepted by Manel Baucells, behavioral economics and decision analysis. Funding: This work was supported in part by the National Science Foundation [Grant MMS 2019982] and All Souls College Oxford [Visiting Fellowships in 2020 and 2022 to D. Prelec]. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2023.4955 .