Memorizing and fabricating? Uncovering online writing test preparation

Abstract

With high-stakes decisions relying on standardized language tests (Chapelle, 2021a), many test takers seek test preparation from commercial ‘cram schools’, including TOEFL iBT writing (E.-Y. J. Kim, 2017; S. Kim, 2021). Despite a $1.2 billion market for online test preparation courses in Korea, previous studies have primarily focused on in-person classes through interviews and surveys, neglecting online courses. This study analyzed ten prerecorded online lectures from two popular cram schools, applying Messick’s (1982) taxonomy of test preparation to code instructional content. The analysis revealed that three types of instructional activities: three Type 1 (enhancing construct-relevant factors), four Type 2 (reducing construct-irrelevant factors), and four Type 3 (enhancing construct-irrelevant factors) test preparation activities were identified. Type 1 activities primarily covered vocabulary, grammar, and organizational patterns. Type 2 focused on teaching test formats, TOEFL trends, time management, and topic choice. Type 3 addressed templates, fabricating examples, gaming the scoring system, and cramming tips. Notably, students were encouraged to utilize memorized templates and fabricate examples. However, these practices may undermine support for the explanation and consequence implication inferences of the validity argument of TOEFL iBT. This study provides valuable insights into language testing preparation, applicable to various contexts with different tests.