Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants

Abstract

ABSTRACT Although careless respondents have wreaked havoc on research for decades, the prevalence and implications of these participants has likely increased due to many new methodological techniques currently in use. Across three studies, we examined the prevalence of careless responding in participants, several means of predicting careless respondents, and the implications of careless respondents on data quality and recruitment attempts. At the same time, we sought to examine the geographic differences of careless responding and provide psychometric evidence for using embedded questions to detect these participants. In Study 1, we found over 1/3rd of participants showed some evidence of careless responding and that careless respondents exhibited certain personality and demographic characteristics. In particular, carelessness seemed more prevalent in Asian samples than in North American samples. In Study 2, we found nearly 1/4th of participants showed evidence of careless responding and that conclusions based on data including versus excluding these participants differed in significant yet unpredictable ways. Finally, in Study 3, we found that nearly 2/3rd of participants who signed up for the study did not meet advertised study requirements for participation and including these participants changed the structure of the data obtained.