Iterative Attribute Hierarchy Exploration Methods for Cognitive Diagnosis Models

Abstract

Attribute hierarchies are commonly assumed to exist in many fields of psychological and educational assessment. Several theory-driven and data-driven approaches have been used to validate or explore attribute hierarchies, such as validating attribute hierarchies in the cognitive diagnostic model (CDM) through statistical hypothesis testing or even learning attribute hierarchies directly from data. A class of structural parameter standard error estimation methods for CDMs is useful for exploring attribute hierarchies, with the limitation that the information matrices of some model parameters may be unstable or singular, leading to biased hypothesis testing. An iterative method of attribute hierarchy testing was proposed to modify the original z-statistic method. The simulation study systematically compares the performance of the z-statistic and the iterative z-statistic in exploring the attribute hierarchy. The results show that the iterative z-statistic provides a better Type I error control rate and statistical power, and it partially solves the problem that the z-statistic is too conservative. In addition, the iterative z-statistic method also achieves satisfactory results on real data.