Belief Elicitation with Binary Outcomes: A Comparison of Quadratic and Binarized Scoring Rules

Abstract

It is becoming increasingly common for researchers to elicit individuals’ beliefs in controlled experiments to understand the underlying motivations of decision makers. We evaluate the performance of the quadratic scoring rule (QSR) and the binarized scoring rule (BSR) in an environment where subjects report probabilistic beliefs over binary outcomes when the probabilities are objectively known to them. We find that reported beliefs are less accurate under the QSR than the BSR at the aggregate level. Consistent with theoretical predictions, risk-averse subjects tend to distort their reported beliefs under the QSR.