Abstract
Bayesian truth serum (BTS) is an exciting new method for improving honesty and information quality in multiple-choice survey, but, despite the method’s mathematical reliance on large sample sizes, existing literature about BTS only focuses on small experiments. Combined with the prevalence of online survey platforms, such as Amazon’s Mechanical Turk, which facilitate surveys with hundreds or thousands of participants, BTS must be effective in large-scale experiments for BTS to become a readily accepted tool in real-world applications. We demonstrate that BTS quantifiably improves honesty in large-scale online surveys where the “honest” distribution of answers is known in expectation on aggregate. Furthermore, we explore a marketing application where “honest” answers cannot be known, but find that BTS treatment impacts the resulting distributions of answers.
Highlights
Subjective judgements play an important role in several areas of polling [1, 2] and research [3, 4]
The base assumption of Bayesian truth serum (BTS) asserts that participants will disproportionately predict endorsements of their own beliefs, and this assumption holds for participants in all experimental treatments
Our experiments demonstrate that Bayesian truth serum (BTS) improves responses to large-scale online survey
Summary
Subjective judgements play an important role in several areas of polling [1, 2] and research [3, 4]. One plausible explanation from research suggests that response to political polls is swayed by social influence [9] or hidden political agendas [10]. As another example, customer satisfaction is an area relying heavily on subjective response; responders to these surveys may experience pressure from external media or their peers that alter their otherwise true responses [11,12,13]. Researchers have investigated the effects of social influence on cultural labor markets, and there is evidence that subjective participants in both artificial [14, 15] and real-world markets are not immune to influenced opinions [16,17,18]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
