Abstract
Instructional manipulation checks (IMCs) have become popular tools for identifying inattentive participants in online studies. IMCs function by attempting to trick inattentive participants into responding incorrectly. However, from a conversational perspective, question characteristics are part of the researcher’s contribution to the conversation, and IMCs may teach participants that there is “more than meets the eye,” prompting systematic thinking on subsequent tricky-seeming questions in an attempt to avoid being tricked. In two online studies, participants responded to a simple task either before or after completing an IMC. As expected, answering an IMC prior to the task improved performance on items that benefit from increased systematic thinking—namely, the Cognitive Reflection Test (Study 1), and a probabilistic reasoning task (Study 2). We conclude that IMCs change attention rather than merely measure attention and discuss implications for their use in online studies.
Highlights
Instructional manipulation checks (IMCs) were designed to address the issue of inattentive responding in surveys and psychological tasks
Studies that use IMCs typically include only the participants who respond correctly to the IMC, and answering the IMC should only affect participants who recognize that the question is intended to trick inattentive participants
Our findings converge to show that an IMC is more than merely a measure of attention—It is an intervention that increases systematic thinking on subsequent tricky-seeming tasks
Summary
If the IMC teaches participants that there is more than meets the eye to the questions, varying the order of the IMC and the CRT should affect participants’ CRT scores. We predicted that participants who receive an IMC question prior to the CRT will score better on the CRT than participants who complete the tasks in the reverse order.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
