Applying Evidence‐Centered Design in the Development of a Multidimensional Adaptive Reading Motivation Measure

Abstract

AbstractAdaptive tests are more efficient than fixed‐length tests through the use of item response theory; adaptive tests also present students questions that are tailored to their proficiency level. Although the adaptive algorithm is straightforward, developing a multidimensional computer adaptive test (MCAT) measure is complex. Evidence‐centered design (ECD) can provide a highly structured framework to guide the development of a MCAT. In this paper, the development of the adaptive reading motivation measure (ARMM) demonstrates the process of applying ECD to the development of a MCAT measure. This paper focuses on the conceptual assessment framework layer of the ECD that guides the technical aspects of MCAT development. The five models of conceptual assessment framework layer are student model, task model, evidence model, assembly model, and presentation model. How each model guided the development of the ARMM is described in detail. In this paper, we demonstrate how the conceptual assessment framework in ECD can provide a useful structure for developing MCAT measures.