The Efficacy of the IRTree Framework for Detecting Missing Data Mechanisms in Educational Assessments

Abstract

The effectiveness of methods for handling missing data in educational assessments depends on understanding the underlying missing mechanisms. This study investigates the performance of the IRTree framework in detecting missing data mechanisms using a Monte Carlo simulation. Omitted responses were simulated at varying proportions according to three mechanisms: MCAR, MAR, and MNAR, across tests with different lengths and sample sizes. The IRTree was employed to model the omitted responses and detect the mechanisms based on the correlations between the propensity to omit and proficiency. Results indicate that the IRTree accurately identifies all three missing data mechanisms, with no relationship between propensity to omit and proficiency under MCAR, and negative correlations for MAR, reaching up to -0.3, and for MNAR, as high as -0.8. Furthermore, the detection of MAR and MNAR mechanisms became more pronounced with higher proportions of omitted responses, longer tests, and larger sample sizes. IRTree framework not only enables educators and researchers to accurately understand the nature of missing data but also guides them in using appropriate methods for handling it.