Assessing C2 writing ability on the Certificate of English Language Proficiency: Rater and examinee age effects

Abstract

Abstract Differentiating between advanced L2 writers at the higher levels of the Common European Framework of Reference (CEFR) presents a challenge in assessment. The distinction between descriptors at the C1 and C2 levels are fine-grained, and even native speakers of a language may not consistently achieve them. At the same time, the CEFR has generally been conceived with the language abilities and contexts of use of adults in mind, thus making CEFR-based interpretations of young language learner’s abilities problematic. This study examines two issues in the assessment of C2-level writing in the context of the Certificate of English Language Proficiency (CELP) writing task: rater effects and examinee age. Interrater reliability and many-facet Rasch analysis showed that raters varied substantially in severity. CELP scoring procedures for rater disagreement partially mitigated severity differences. Contrary to expectations, age differentiated examinee abilities minimally and defied hypothesized ordering (i.e., that writing ability would increase with age). Additionally, some raters were found to demonstrate bias towards the youngest examinees. Specific implications for the CELP’s validity argument and broader implications for assessing young writers in CEFR terms are discussed.