Development and Validation of a Cognitive Diagnostic Assessment with Ordered Multiple-Choice Items for Addition of Time

Abstract

Cognitive Diagnostic Assessment (CDA) is an alternative assessment which can give a clear picture of pupils’ learning process and cognitive structures to education stakeholders so that appropriate instructional strategies can be designed to tailored pupils’ needs. Coincide with this function, the Ordered Multiple-Choice (OMC) items were incorporated into the CDA developed in this study. This paper describes the process of developing and validating a Cognitive Diagnostic Assessment with OMC items for assessing pupils’ attribute mastery on ‘Addition of Time’ and reports the psychometric properties of the items, the model-data fit, the attribute reliability and the overall reliability of the assessment. The sample of the study consisted of 30 Year Four National School (SK) pupils, 48 Year Four National-Type Chinese School (SJKC) pupils and 12 Year Four National-Type Tamil School (SJKT) pupils in Penang state, Malaysia. The data was analysed using two measurement models, namely Classical Test Theory and Attribute Hierarchy Method (AHM). The findings of the study indicated that the instrument developed consisted of good-quality OMC items with appropriate difficulty and high discrimination power. With the satisfactory model-data fit in AHM, the inference made about pupils’ attributes mastery based on their performance in CDA with OMC items was valid. The CDA with OMC items developed in this study was found to be reliable at both attribute level and assessment level. Perhaps, this instrument could be used in the mathematics classrooms for supporting teachers in diagnosing pupils’ cognitive strengths and weaknesses for ‘Addition of Time’.