Where’s the difficulty in standardized reading tests: The passage or the question?

Abstract

The purpose of this investigation was to examine the extent to which item and text characteristics predict item difficulty on the comprehension portion of the Gates-MacGinitie Reading Tests for the 7th-9th and 10th-12th grade levels. Detailed item-based analyses were performed on 192 comprehension questions on the basis of the cognitive processing model framework proposed by Embretson and colleagues (Embretson & Wetzel, 1987). Item difficulty was analyzed in terms of various passage features (e.g., word frequency and number of propositions) and individual-question characteristics (e.g., abstractness and degree of inferential processing), using hierarchical linear modeling. The results indicated that the difficulty of the items in the test for the 7th-9th grade level is primarily influenced by text features--in particular, vocabulary difficulty--whereas the difficulty of the items in the test for the 10th-12th grade level is less systematically influenced by text features.