Toward quality criteria for collective judgments

Abstract

While most organizational and social decision making is done in a group or collective mode, there are few guides or evaluative criteria for judging when a high-quality outcome has been reached. Most past studies of group decision making have been conducted in laboratories using student subjects and factual problems with correct answers as means for judging outcome quality. Such proxies are rough approximations at best of real-world conditions where value differences can be intense and problems have no correct or best answer. Drawing upon the existing literature, evaluative criteria are proposed including process, content, and outcome concerns. An instrument based upon these criteria is then applied retrospectively to six cases of ad hoc collective decision making. Using dimensional analyses, a set of more detailed evaluative factors is derived from actual participant responses. The results provide some insights into the nature of high-quality collective judgments as well as the most effective procedures for their achievement.