Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2 reading ability

Abstract

Previous research in cognitive diagnostic assessment (CDA) of L2 reading ability has been frequently conducted using large-scale English proficiency exams (e.g., TOEFL, MELAB). Using CDA, it is possible to analyze individual learners’ strengths and weaknesses in multiple attributes (i.e., knowledge, skill, strategy) measured at the item level. This study explored how a placement test score could be used for diagnosing the L2 reading ability of incoming students to an adult ESL program. Five content experts completed a reading placement test and identified the attributes required for successfully completing each item on the test, while referring to the list of L2 reading attributes. Then, content experts’ codings were analyzed and developed into an item-by-attribute Q-matrix. The Fusion model, a type of cognitive diagnostic model (CDM), was used for refining the Q-matrix and diagnosing 1982 learners’ strengths and weaknesses in L2 reading. Results suggest that 10 major L2 reading attributes were involved in the reading test. In addition, examinees’ strengths and weaknesses were identified for the overall group, three reading proficiency groups (i.e., beginner, intermediate, and advanced), and individual learners. Such information could be provided to ESL program administrator and teachers for enhancing the reading curriculum and developing instructional materials.