Evaluation of a decision based on subjective probability estimates

Abstract

In numerous decision analysis problems the decision maker must make estimates of various probabilities, such as the probability of completion of a project by a given time, the probability of receiving a specific order, the probability that market demand for a product will be at a certain level, etc. Only infrequently is it possible to perform an `after the fact' evaluation of the accuracy of such subjective probability estimates. This paper describes the results of a practical application where subjective probability estimates by a group of 436 qualified individuals could be compared with subsequent realizations. The reasons for the divergences that were found are analyzed and related to other problems where one must rely on subjective probability estimates. Thus, although the specific survey described here is of a specialized nature, many of the findings have general applicability.