Defining and Comparing the Reading Comprehension Construct: A Cognitive-Psychometric Modeling Approach

Abstract

As a construct definition, the current study develops a cognitive model describing the knowledge, skills, and abilities measured by critical reading test items on a high-stakes assessment used for selection decisions in the United States. Additionally, in order to establish generalizability of the construct meaning to other similarly structured tests designed for international populations and distinct uses, the skills invoked during a reading comprehension test for English learners from a previous study are contrasted to those in the present study. The results obtained using rule-space methodology suggest that the most difficult skills on reading comprehension items pertain to complex cognitive processes (e.g., understanding implicit ideas), while skills tapping into basic cognitive processes (e.g., word meaning) are mastered with ease by both populations. However, variations across tests in the impact of various cognitive skills on test scores suggest that the differences in construct meaning be considered when interpreting and comparing test scores. Cognitive-psychometric modeling approaches such as those applied in this study prove to be useful in substantively examining score interpretation and construct generalizability.