Abstract
Cognitive diagnostic test design (CDTD) has a direct impact on the pattern match ratio (PMR) of the classification of examinees. It is more helpful to know the quality of a test during the stage of the test design than after the examination is taken. The theoretical construct validity (TCV) is an index of the test quality that can be calculated without testing, and the relationship between the PMR and the TCV will be revealed. The TCV captures the three aspects of the appeal of the test design as follows: (1) the TCV is a measure of test construct validity, and this index will navigate the processes of item construction and test design toward achieving the goal of measuring the intended objectives, (2) it is the upper bound of the PMR of the knowledge states of examinees, so it can predict the PMR, and (3) it can detect the defects of test design, revise the test in time, improve the efficiency of test design, and save the cost of test design. Furthermore, the TCV is related to the distribution of knowledge states and item categories and has nothing to do with the number of items.
Highlights
Cognitive diagnosis (CD) has received much attention, providing diagnostic information of knowledge or skills to the examinees (de la Torre and Douglas, 2004; de la Torre, 2008; DeCarlo, 2011; Liu et al, 2012; Kang et al, 2017; Huebner et al, 2018)
The first column shows the different distribution of examinees, and the other columns show the results of the different Q matrix (Qt) matrices
The theoretical construct validity (TCV) was superior: the TCV was uniformly higher than the pattern match ratio (PMR) regardless of the distribution of examinees and the Qt matrices
Summary
Cognitive diagnosis (CD) has received much attention, providing diagnostic information of knowledge or skills (often called “attributes” in the CD literature) to the examinees (de la Torre and Douglas, 2004; de la Torre, 2008; DeCarlo, 2011; Liu et al, 2012; Kang et al, 2017; Huebner et al, 2018). It is critical to ensure that high-quality cognitive diagnostic tests can accurately diagnose the knowledge state (KS, i.e., the latent cognitive states) of examinees. Cognitive diagnostic test design (CDTD) is the design of a Q matrix, called Qt, i.e., rows representing attributes and columns representing attribute vectors, namely, items. By anchoring the items with attribute vectors, proposition experts and measurement experts transform items into measurable forms and diagnose examinees. The design of the Qt matrix is the problem of how to match the attribute vectors to achieve a certain predetermined goal
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
