Cognitive Diagnostic Testing

Abstract

The objective is to present a concise description of cognitive diagnostic testing, including its definition, components, examples of current application, and future directions. Cognitive diagnostic testing or assessment is defined as a test ‘designed to measure specific knowledge structures and processing skills in students so as to provide information about their [academic] cognitive strengths and weaknesses’ (Leighton & Gierl, 2007, p. 3). Cognitive diagnostic tests are developed based on evidence-based frameworks of how learners conceptualise and execute their understanding of academic subject matter, and require some type of psychometric model to summarise student test-based performance. Diagnostic classification models or DCMs often serve this psychometric purpose. DCMs facilitate in the accurate and reliable analysis, summary, and interpretation of students’ cognitive diagnostic test performance. Challenges exist for the development and application of cognitive diagnostic tests, namely: (1) translating cognitive frameworks into test design, (2) applying DCMs given sample size requirements, and (3) ensuring the reliability and validity of subsequent student skill classifications. While research on DCMs has generally outpaced the creation of operational cognitive diagnostic tests, testing companies and other organisations are increasing their investment into developing cognitive diagnostic-style tests to support student learning inside and outside the classroom for workforce success.