Quadratic voting in the wild: real people, real votes

Abstract

Since their introduction in 1932, Likert and other continuous, independent rating scales have become the de facto toolset for survey research. Scholars have raised significant reliability and validity problems with these types of scales, and alternative methods for capturing perceptions and preferences have gained traction within specific domains. In this paper, we evaluate a new broadly applicable approach to opinion measurement based on quadratic voting (QV), a method in which respondents express preferences by ‘buying’ votes for options using a fixed budget from which they pay a quadratic price for votes. Comparable QV-based and Likert-based survey instruments designed by Collective Decision Engines LLC were experimentally evaluated by randomly assigning potential respondents to one or the other method. Using a host of metrics, including respondent engagement and process-based metrics, we provide some initial evidence that the QV-based instrument provides a clearer measure of the preferences of the most intense respondents than the Likert-based instrument does. We consider the implications for survey satisfying, a key threat to the continued value of survey research, and reveal the mechanisms by which QV differentiates itself from Likert-based scales, thus establishing QV as a promising alternative survey tool for further political and commercial research. We also explore key design issues within QV-based surveys to extend these promising results.