Linking Not-Quite-Vertical Scales Through Multidimensional Item Response Theory

Abstract

ABSTRACTThis research concerns the application of multidimensional item response theory (e.g., Reckase, 2009) to link not-quite-vertical scales across age/grade levels. The approach allows scores to be related across levels without requiring the assumption of a vertical scale. The multidimensional framework is applied to an English language proficiency assessment system used in grades K-12. Separate item pools were developed for six gradebands (K, 1, 2–3, 4–5, 6–8, and 9–12) and for four domains of language use (listening, reading, speaking, and writing). Although the skills/practices underlying language proficiency overlap across these gradebands, it is expected that the constructs measured in adjacent bands are sufficiently distinct to call in to question the use of a vertical scale. We illustrate three uses of multidimensional item response theory: (1) to test the assumption of a common, vertical scale within domain; (2) to estimate student ability with respect to the scale for a gradeband adjacent to the test administered; and (3) to evaluate cut scores across grade levels, so that performance expectations increase in rigor across gradebands, even as the scale changes.