Multiple Switching and Data Quality in the Multiple Price List

Abstract

Researchers working with the widely used Multiple Price List (MPL) instrument have found that a substantial proportion of subjects switch back and forth between the safe and the risky choice columns in the instrument, which is behavior believed to indicate low quality decision-making (e.g. Charness et al. (2013)). In this study we develop a conceptual framework defining decision-making quality to test three leading explanations for the nature of low quality decision-making in the MPL using the covariance of responses in the MPL with a second, simpler risk instrument. We finding evidence in support of task-specific miscomprehension of the MPL. Our novel nudge treatment reduced multiple switching behavior (MSB) from 31% to 10% (p-value $<$ 0.001) and increased the proportion of high quality responses by 152%, according to the data quality metric we propose. We find that MSB does not capture the full extent of low quality decision-making, as even non-multiple switchers generate relatively low data quality under the standard protocol. We also find that while cognitive ability explains MSB, it does not predict higher data quality. This suggests that comprehension of the MPL can be a problem even with high cognitive ability subjects exhibiting low multiple switching rates. We further suggest several general applications of our framework and experimental design:a test of relative complexity between two tasks and a test of the effectiveness of various devices for improving task comprehension.