Using mixture modeling to examine differences in perceptual decision-making as a function of the time and method of participant recruitment

Abstract

We examine whether perceptual decision-making differs as a function of the time in the academic term and whether the participant is an undergraduate participating for course credit, a paid in-person participant, or a paid online participant recruited via Amazon Mechanical Turk. We use a mixture modeling approach within an evidence accumulation framework that separates stimulus-driven responses from contaminant responses, allowing us to distinguish between performance when a participant is engaged in the task and the consistency in this task focus. We first report a survey showing cognitive psychologists expect performance and response caution to be lower among undergraduate participants recruited at the end of the academic term compared to those recruited near the start, and highest among paid in-person participants. The findings from two experiments using common paradigms revealed very little evidence of time-of-semester effects among course credit participants on accuracy, response time, efficiency of information processing (when engaged in the task), caution, and non-decision time, or consistency in task focus. However, paid in-person participants did tend to be more accurate than the other two groups. Groups showed similar effects of speed/accuracy emphasis on response caution and of discrimination difficulty on information processing efficiency, but the effect of speed/accuracy emphasis on information processing efficiency was less consistent among groups. We conclude that online crowdsourcing platforms can provide quality perceptual decision-making data, but recommend that mixture modeling be used to adequately account for data generated by processes other than the psychological phenomena under investigation.