The relationship between nonsystematic delay discounting and low-quality survey responses in a sample of smokers: ROC curve analysis.

Abstract

Delay discounting (DD), the decrease of the subjective value of a reward as the delay to its receipt increases, is a crucial aspect of decision-making processes. As evidence continues to mount, additional attention needs to be given to nonsystematic DD, a response pattern that has been reported in the literature but rarely investigated. We noticed in our recent online research an increase in the proportion of nonsystematic DD responses across samples, consistent with the so-called Amazon Mechanical Turk (MTurk) data quality crisis. The significant proportion of nonsystematic responses created an opportunity to investigate its association with data quality in the present study. In a sample of smokers recruited from MTurk (n = 210), three independent quality check indexes evaluated participants' response quality. The degree of nonsystematic DD was quantified by the algorithms developed by Johnson and Bickel (2008). The area under the receiver operating characteristic curve (AUC) predicting response quality by nonsystematic DD was obtained. The observed AUC values were at the extreme of the null distributions (ps < .001) in a permutation test. Furthermore, the nonsystematic DD cutoffs provided in Johnson and Bickel (2008) showed good sensitivity (0.77-0.93), albeit low-moderate specificity (0.42-0.74), in detecting low-quality responses. The findings showed that nonsystematic DD was associated with low-quality responses, although other factors contributing to the nonsystematic responses remain to be identified. (PsycInfo Database Record (c) 2023 APA, all rights reserved).