Overconfidence in Probability and Frequency Judgments: A Critical Examination

Abstract

The overconfidence observed in calibration studies has recently been questioned on both psychological and methodological grounds. In the first part of the article we discuss these issues and argue that overconfidence cannot be explained as a selection bias, and that it is not eliminated by random sampling of questions. In the second part of the article, we compare probability judgments for single events with judgments of relative frequency. Subjects received a target individual's personality profile and then predicted the target's responses to a series of binary questions. One group predicted the responses of an individual target, while a second group estimated the relative frequency of responses among all target subjects who shared a given personality profile. Judgments of confidence and estimates of relative frequency were practically indistinguishable; both exhibited substantial overconfidence and were highly correlated with independent judgments of representativeness.