Cognitive Diagnostic Assessment of IELTS Listening: Providing Feedback from its Internal Structure

Abstract

High stakes testing, such as IELTS, is designed to select individuals for decision-making purposes (Fulcher, 2013b). Hence, there is a slow-growing stream of research investigating the subskills of IELTS listening and, in feedback terms, its effects on individuals and educational programs. Here, cognitive diagnostic assessment (CDA) performs it through breaking down the abstract listening concepts into fine-grained subcomponents, and applying it to in-class assessment and teaching. Therefore, the strengths and weaknesses of language learners concerning various subsections of the IELTS listening test were explored with use of three CDMs, i.e., DINA, DINO and mixed DINA-DINO. As a result, the analysis of the participants’(N=463) performances revealed that the mixed-DINA-DINO model using the original Q-matrix was the most effective model. On the other hand, all three models indicated that gap-filling was cognitively less demanding than the other subsections. However, they did not show any agreement on multiple-choice sub-competency. Accordingly, it can be argued that the CDM-driven diagnostic information related to the sub-sections of the IELTS listening test can be used in educational systems to explore the underlying structure of a test and detect the learners’ potential strengths and weaknesses with mastery and non-mastery of the intended items.