Developing a Trajectory Deviance Index for Dynamic Measurement Modeling

Abstract

Dynamic Measurement Modeling (DMM) is a recently-developed measurement framework for gauging developing constructs (e.g., learning capacity) that conventional single-timepoint tests cannot assess. The current project developed a person-specific DMM Trajectory Deviance Index (TDI) that captures the aberrance of an individual’s growth from the model-implied trajectory. Four TDI candidates were formulated, and two simulation studies were conducted to investigate the distributional properties and effectiveness of those four TDI candidates. Consequently, the best functioning index was determined as the final formulation of the TDI, to be recommended for use in DMM research. The data generation model in the simulation study was based on the parameter estimates from the Technology-enhanced, Research-based, Instruction, Assessment, and professional Development (TRIAD) cluster-randomized experiment data, which contains seven waves of mathematics test scores for students from preschool through Grade 5, and which has been modeled by DMM in previous research. In addition, an empirical study was also conducted to demonstrate the uses of the developed TDI within those real-world data. Incorporating TDI into DMM analysis strengthened the validity of score use and interpretation and offered a quantitative means of determining which students in the dataset were not adequately served by the dynamic measurement model.