The Impact of Communication and Collaboration Between Test Developers and Teachers on a High-Stakes ESL Exam: Aligning External Assessment and Classroom Practices

Abstract

In Quebec the high-stakes Secondary Five ESL exit writing exam developed by the Education Ministry (MELS) is administered and corrected by classroom teachers. In this distinctive situation, the MELS works toward aligning classroom-based assessment (CBA) and the writing exam by making ongoing teacher involvement part of its development and validation process. This article presents data from Year 1 (2010) and Year 2 (2011) of a three-year study using a mixed-methods design to examine the consequences of this unique process. The participants from each administration of the exam include 2 MELS evaluation officials, 11 MELS-trained raters from the research marking centers, and approximately 500 students. Data consist of student scores rated by raters at the marking centers, interviews with raters, teacher and rater survey information, and notes from the marking centers. Comparative analyses of student scores, Rasch analyses of rater behavior, and content analyses of interviews, surveys, and notes were conducted. The findings show two-way benefits for teachers and MELS administrators, creating a value-added validation process: teacher comments for rubric and task revisions improved scoring and exam construct validity; increased teacher exam knowledge helped align CBA practices, enhancing student learning and enabling mentoring of colleagues in interpreting rubric criteria.