Teacher implementation of Self-Regulated Strategy Development with an automated writing evaluation system: Effects on the argumentative writing performance of middle school students

Abstract

This study examined the effects of teacher implementation of (1) Self-Regulated Strategy Development (SRSD) instruction or (2) traditional writing instruction, combined with an Automated Writing Evaluation (AWE) system called NC Write, on students’ argumentative writing performance. The SRSD model was adapted to a lower-intensity format with the goal of supporting teacher implementation and reducing professional development. This study is the first to examine the effectiveness of an intervention that combined SRSD for writing with an AWE system. Middle school students (N = 829) participated in one of three conditions: NC Write + SRSD instruction, NC Write + traditional writing instruction, or a comparison condition. Results of multilevel models showed that students in the NC Write + traditional writing instruction condition produced higher-quality essays than comparison students at posttest. Students in the NC Write + SRSD instruction condition produced posttest essays that were of a higher quality, longer, and included more basic elements of argumentative essays than students in the other two conditions. Social validity data from surveys and interviews showed that students and teachers rated NC Write and SRSD instruction favorably. Overall study results suggest that incorporating AWE into a program of writing instruction supports improvements in students’ writing quality. Findings provide initial evidence that when supported by AWE, SRSD may be implemented by teachers at a lower than normal treatment intensity and still have a strong, positive impact on students’ writing quality.