Investigating the ecological validity of second language writing assessment tasks

Abstract

Nowadays, writing in academic settings is mediated by digital technology in essential ways, yet many writing test tasks deny access to the digital resources common to everyday practice (e.g., spell-check, electronic dictionaries). This discrepancy potentially threatens the ecological validity of such tasks and raises questions about construct (under) representation. Informed by an ecological perspective on language and communication (van Lier, 2004), a study was carried out to better understand how access to writing resources impacts writing test task performance. Twenty-two international students enrolled in a graduate program in the United States completed two argumentative writing tasks. On one of the tasks, participants were given full access to writing resources; on the other task, no access to writing resources was given. Average scores on participants' essays were analyzed quantitatively to determine the degree to which, and in what specific domains, performance differed under each condition. In addition, participants’ self-assessments of performance on the tasks were analyzed quantitatively and qualitatively. Results suggest that writers benefitted from having access to writing resources, but the perceived impact of resource accessibility differed from the scores given by raters. These findings have implications for writing test task design and may help inform L2 writing instruction.