The Bots Ruining Social Science Are Not Bots at All.

Abstract

Translate Article

Take Notes

Researchers who employ online data collection from human subjects currently face a conundrum: It is both essential to how behavioral science functions and threatened by low-quality data. It is often assumed that random, inconsistent, and otherwise incomprehensible data in online surveys comes mainly from bots. Despite this assumption, few studies have directly examined where problematic data comes from, even though identifying the source has important implications for creating the right solutions. We examined this issue on several popular participant-recruitment platforms, including Mechanical Turk (MTurk) and Lucid. Across four studies spanning 5 years using multiple methods, we here provide evidence that most of the data-quality problems affecting online research using online panels can be tied to fraudulent users from outside of the United States-not bots. We identify many of the telltale signs that humans leave behind and describe the most effective ways of blocking problematic human responses to address the online data-quality problem.