Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Knowledge and Skills in Mathematics: An Operational Implementation of Cognitive Diagnostic Assessment

Abstract

The purpose of this study is to apply the attribute hierarchy method in an operational diagnostic mathematics program at Grades 3 and 6 to promote cognitive inferences about students’ problem-solving skills. The attribute hierarchy method is a psychometric procedure for classifying examinees’ test item responses into a set of structured attribute patterns associated with a cognitive model. Principled test design procedures were used to design the exam and evaluate the student response data. To begin, cognitive models were created by content specialists outlining the knowledge and skills required to solve mathematical tasks in Grades 3 and 6. Then, items were written specifically to measure the skills in the cognitive models. Finally, confirmatory psychometric analyses were used to evaluate the student response data by estimating model-data fit, attribute probabilities for diagnostic score reporting, and attribute reliabilities. Future directions for score reporting and adaptive diagnostic testing are discussed.