Teachers in automated writing evaluation (AWE) system-supported ESL writing classes: Perception, implementation, and influence

Abstract

Automated writing evaluation (AWE) systems have attracted much attention for their efficiency in feedback provision and automated essay scoring in language classrooms. However, teachers’ roles in this technology-enhanced environment and their potential impact on students’ writing development are still under-investigated. To fill this gap, this classroom-based study examined three English as a second language (ESL) teacher’s perceptions and uses of Criterion®, an AWE system developed by Educational Testing Service (ETS), in four ESL writing classes at a large Midwestern U.S. university. Student’s writing performance data, including submission behaviors, revision types, and progress in grammatical accuracy, were then analyzed in light of teachers’ perceptions and reported uses of Criterion. The results indicate that the teachers took different approaches to integrating Criterion in their classes, which, in turn, was reflected in observable differences in students’ essay submission frequencies on Criterion, revision types, and changes in error rates. The findings shed light on the importance of teacher agency and cognition in technology-supported ESL classrooms. Implications for teaching English writing with AWE systems are discussed.