Quality of group judgment.

Abstract

The quality of group judgment is examined in situations in which groups have to express an opinion in quantitative form. To provide a yardstick for evaluating the quality of group performance (which is itself defined as the absolute value of the discrepancy between the judgment and the true value), four baseline models are considered. These models provide a standard for evaluating how well groups perform. The four models are: (a) randomly picking a single individual; (b) weighting the judgments of the individual group members equally (the group mean); (c) weighting the group member (i.e., the one closest to the true value) totally where the best is known, a priori, with certainty; (d) weighting the best member totally where there is a given probability of misidentifyin g the best and getting the second, third, etc., best member. These four models are examined under varying conditions of group size and bias. Bias is denned as the degree to which the expectation of the population of individual judgments does not equal the true value (i.e., there is systematic bias in individual judgments). A method is then developed to evaluate the accuracy of group judgment in terms of the four models. The method uses a Bayesian approach by estimating the probability that the accuracy of actual group judgment could have come from distributions generated by the four models. Implications for the study of group processes and improving group judgment are discussed. Consider a group of size N that has to arrive at some quantitative judgment, for example, a sales forecast, a prediction of next year's gross national product, the number of bushels of wheat expected in the next quarter, and the like. Given the prevalence of such predictive activity in the real world, it is clearly important to consider how well groups can and do perform such tasks, as well as to consider strategies that may be used to improve performance. In this paper we address the issue of defining the quality of group judgment and assess the effects and limitations on judgmental quality of different strategies for combining opinions under a variety of circumstances .