Note—On the Aggregation of Individual Probability Estimates

Abstract

This Note considers the problem of aggregating individual probability estimates of an event to obtain a group estimate. Norman Dalkey has argued that no rigorous theory of probability aggregation is possible for the following reasons: (1) There is no consistent way of aggregating individual probability estimates (Dalkey's Impossibility Theorem). But Dalkey's Impossibility Theorem makes what we call the context-free assumption, namely that the group estimate of an event's probability is a function only of the estimates of the individuals and, in particular, is not dependent on the event in question. We argue that the context-free assumption is unreasonable and present several known aggregation methods which don't make it. (2) Although there is a Bayesian approach to the problem, it requires that the estimates of all but at most one of the experts be independent of the event in question. To make this argument, Dalkey assumed both: a1: The individual estimates are independent random variables, and a2: Given the occurrence or nonoccurrence of the event, the individual estimates are independent random variables. This paper presents an alternate derivation of the result, abstracting from Dalkey's context. The point is that Dalkey's result is simply a consequence of assuming both a1 and a2. It is unreasonable to assume both a1 and a2. And hence Dalkey's result does not discredit the Bayesian approach. Thus this paper resolves some theoretical objections to the possibility of a rigorous theory of aggregation. Furthermore, it clarifies the nature of the context-free assumption and the independence assumptions: a1 and a2.