Examining cognitive diagnostic modeling in classroom assessment conditions

Abstract

Cognitive diagnostic models (CDMs) are a family of psychometric models designed to provide categorical classifications for multiple latent attributes. CDMs provide more granular evidence than other psychometric models and have potential for guiding teaching and learning decisions in the classroom. However, CDMs have primarily been conducted using large samples. This study examines estimating CDMs in small sample conditions to aid formative learning. Three CDMs were compared across simulated classroom assessment conditions: deterministic input, noisy "and" gate (DINA) model, non-parametric cognitive diagnosis (NPCD), and supervised artificial neural network (SANN). We found all models estimated examinee classifications at the smallest sample size. Accuracy of individual attribute mastery classifications was acceptably high for the models under certain conditions. Effective item discrimination was the most important factor to accurately classify. The DINA and NPCD models were more resilient to measurement error than the SANN. Recommendations for application of CDMs in the classroom are provided.