Using Crowdsourcing for Alcohol and Nicotine Use Research: Prevalence, Data Quality, and Attrition on Amazon Mechanical Turk

Abstract

BackgroundGauging the feasibility of using Amazon Mechanical Turk ® (MTurk) for various types of substance use research is precluded by a lack of information pertaining to the recruitment process in published studies utilizing it and concurrent information on data quality. ObjectiveThe present report addressed this gap by documenting the prevalence of alcohol and nicotine use, self-reported major health conditions, and information on data quality and retention on MTurk. Individuals 21 to 90 years old (N = 1101, Mdn age = 30) with United States-based MTurk accounts completed a stand-alone screening survey. The screening consisted of basic demographic, substance use, and physical/mental health questions, as well as items to gauge language proficiency/attention (i.e., data quality). ResultsPoor quality data was infrequent (6.5% of participants) and associated with self-reported non-United States residence, affirmative responding (e.g., currently pregnant, using both alcohol and nicotine), and other response characteristics (e.g., not disclosing health conditions). Among those passing quality checks, alcohol and nicotine use were relatively common (71.5% and 24.8%). Major physical (6.3%) and mental health conditions (14.8%) were less common. Despite not sending direct invitations, most eligible participants returned to and completed the main study (81.7%). Conclusions/Importance: Alcohol and nicotine use were relatively common among MTurk workers and retention rates were high. Together with the low prevalence of poor quality data, MTurk appears to remain a fruitful platform for substance use research; although researchers must be diligent in using appropriate screening tools, as substance use was sometimes associated with poor data quality and MTurk account information may not be reliable.