Would I Lie to You? Using Markov Chains to Detect Careless Responding in Survey Data

Abstract

Although careless response behaviors by survey participants bear considerable threats to the quality of the data, scholars often do not sufficiently address this issue in their analyses. This might be partially because existing approaches to detect careless responding are specialized to detect a certain type of careless response patterns, but fail to detect others. We develop a post-hoc measure, the Lazy Respondents (Laz.R) index, that is based on first-order Markov chains and the idea that careless respondents’ last response can be used to predict their next response. Using large, publicly available datasets to compare our Laz.R index to the most common precautionary and post-hoc measures, we find that (1) the Laz.R index correctly identifies patterns of careless respondents that some or all of the other indices overlook and (2) the Laz.R index avoids false positives (i.e., patterns that are flagged by single other indices, but which seem to capture truthful answering behavior). Furthermore, we suggest an approach to find a sample-specific cutoff value for careless respondents via kneedle algorithm. Finally, we introduce an R Shiny application that provides scholars a simple and readily available means to implement the Laz.R index.