Retrofitting Non-cognitive-diagnostic Reading Assessment Under the Generalized DINA Model Framework

Abstract

ABSTRACTCognitive diagnosis models (CDMs) are psychometric models developed mainly to assess examinees’ specific strengths and weaknesses in a set of skills or attributes within a domain. By adopting the Generalized-DINA model framework, the recently developed general modeling framework, we attempted to retrofit the PISA reading assessments, a non-cognitive-diagnostic assessment, in three steps: (a) Q-matrices construction based on item content analysis; (b) Q-matrices validation and refinement based on a saturated CDM; and (c) reduced CDMs evaluations based on acceptable Q-matrix. Our research showed that (a) the practice of retrofitting non-cognitive-diagnostic assessment for cognitive diagnostic purpose was feasible; (b) Q-matrix construction and CDM selection were especially important when retrofitting approach was adopted; (c) compensatory and saturated CDMs could fit the PISA reading assessment better than other CDMs; and (d) the application of CDMs to reading assessments may promote research on reading construct.