Who responds inconsistently to mixed-worded scales? Differences by achievement, age group, and gender

Abstract

ABSTRACT We investigated two research questions: which students are more likely to respond inconsistently to mixed-worded questionnaire scales, and which country samples have larger shares of inconsistent respondents? We defined an inconsistent response as strongly agreeing or disagreeing with both positively and negatively worded items of the same scale. Since we assumed that inconsistent responding occurs due to a lack of carefulness, reading, or cognitive skills, we expected to find that inconsistent responding was associated with lower achievement, younger age, being a nonnative speaker, and being a boy. We used data from all 38 countries that participated in the fourth- and eighth-grade assessments of TIMSS (Trends in International Mathematics and Science Study) 2019. Using the mean absolute difference method, we identified shares of 1‒21% inconsistent respondents across samples. The results generally supported our hypotheses, especially the hypothesis that inconsistent responding is more common among students and countries with lower mathematics achievement levels.