An Experiment in the Ability of Raters to Evaluate Lexis in Writing

Abstract

Abstract For many years, people have considered lexis and grammar separately in the context of teaching and learning English. In the assessment of second language writing, lexis and grammar continue to be considered separately. However, recent corpus studies have questioned this approach and argued that lexis and grammar are fundamentally inseparable. While the assessment of lexis and grammar as two distinct qualities lends face validity to assessment criteria, the corpus literature suggests that raters may not be able to accurately distinguish the two. The current study examines the ability of raters to separate lexis and grammar when using an analytic rating scale to assess timed essays. In this experiment, the lexical content of 27 essays was manipulated before rating in order to determine the effect of lexical accuracy, lexical variation and lexical richness on lexis and grammar scores. From the results, it seems that raters are sensitive to lexical accuracy, but not lexical variation nor lexical richness. In addition, the manipulation of lexical qualities had a significant effect on grammar scores but not on lexis scores, supporting the idea that raters find it challenging to distinguish lexis from grammar