Comparability of Writing Assessment Scores Across Languages: Searching for Evidence of Valid Interpretations

Abstract

Very few empirically based studies have denied or confirmed the validity of holistic score interpretations and the validity of French-English writing scores comparisons. The present study addresses these important issues. Part I investigates if adjacent holistic scores represent different writing skills. Part II evaluates if variations exposed in Part I differ across languages. Part III examines if the writing skills associated with a holistic score vary across languages. Data comes from a sample of 3,116 students' essays. Part I confirm that each holistic score represents different writing skills. Part II reveals negligible language effects. Part III reports that some writing skills vary according to language, but the size effect is very small. Results are discussed in relation to criteria clarity, test adaptation, score comparisons, and interpretations, and the need to search for more evidence supporting the validity of cross-language comparisons of holistic scores.