How Randomly are Students Random Responding to Your Questionnaire? Within-Person Variability in Random Responding Across Scales in the TIMSS 2015 Eighth-Grade Student Questionnaire

Abstract

ABSTRACT Questionnaires in educational research assessing students’ attitudes and beliefs are low-stakes for the students. As a consequence, students might not always consistently respond to a questionnaire scale but instead provide more random response patterns with no clear link to items’ contents. We study inter-individual differences in students’ intra-individual random responding profile across 19 questionnaire scales in the TIMSS 2015 eighth-grade student questionnaire in seven countries. A mixture IRT approach was used to assess students’ random responder status on a questionnaire scale. A follow-up latent class analysis across the questionnaire revealed four random responding profiles that generalized across countries: A majority of consistent nonrandom responders, intermittent moderate random responders, frequent random responders, and students that were exclusively triggered to respond randomly on the confidence scales in the questionnaire. We discuss the implications of our findings in light of general data-quality concerns and the potential ineffectiveness of early-warning monitoring systems in computer-based surveys.