A cognitive diagnostic approach to IELTS speaking test: unveiling the subskills and test-takers’ perceptions

Abstract

Although the IELTS speaking test assesses English verbal-communication competence accurately and reliably, its traditional, retroactive scoring framework stops short of providing detailed diagnostic information about individual test-takers’ specific domains of proficiency or deficiency. This study employed a cognitive diagnostic assessment (CDA) framework, grounded in cognitive psychology and diagnostic, formative feedback theories, to identify the specific cognitive (sub)skills and attributes required to successfully perform on the IELTS speaking test and examine the test-takers’ perceptions regarding the CDA and IELTS-speaking-assessment interface. The purpose was to integrate CDA and feedback literacy into the IELTS assessment. Adopting a cognitive/psychometric perspective, we conceptualized language proficiency as encompassing separate cognitive subskills that can be systematically measured and diagnosed. To this end, we analyzed speaking data from 500 pre-IELTS candidates and conducted 20 expert interviews. The thematic analysis employed through the MAXQDA software helped identify five specific subskills related to the IELTS speaking test. Using a similar thematic analysis procedure, we extracted five perceptive themes regarding an integrated IELTS-CDA framework from the interview data of 151 participants. The results have important implications for the practicality and efficacy of employing CDA to improve IELTS speaking interpretation and feedback.