Linguistic complexity in teachers' assessment of German essays in high stakes testing

Abstract

In national educational standards, linguistic complexity is one of the core aspects characterizing the demands of reading material and student writing quality. In Germany, teachers are supposed to consider linguistic complexity in their assessment of language arts Abitur examinations, the high stakes test qualifying for higher education admission in Germany. In the present study, we investigated if and how teachers consider linguistic complexity in their assessment of writing quality. To systematically identify a range of linguistic complexity aspects, we conducted computational linguistic analyses of student essays (N = 344) written during Abitur examinations in the subject German. Experienced teachers (N = 33) then regraded a subsample of the essays (n = 16) and rated their linguistic complexity. We used mixed-effects models to study the relationships between computational measures of linguistic complexity, teachers’ assessment of writing quality (language, content, and overall grade), and their linguistic complexity rating. Results confirm that teachers’ complexity rating and their assessment of the language aspect of writing quality is related to the computed linguistic complexity features, primarily measures of syntactic complexity and language use (word frequency measures). But findings also show that teachers have difficulties in separating linguistic complexity from aspects of content and language accuracy.