Examining rating criteria used to assess U.S. college students’ Chinese oral performance

Abstract

This study empirically examines the rating criteria used to assess U.S. college students’ CSL (Chinese as a Second Language) oral performance by analyzing teachers’ assessment of these performances at different proficiency levels. The researcher videotaped ten speeches, and three ACTFL-trained raters assessed oral performance in these samples. The researcher then selected three samples (Samples 1, 2, and 3) to represent Novice High, Intermediate High, and Advanced Low levels. The researcher developed 20 rating items through interviewing ten experienced CSL teachers and running an Exploratory Factor Analysis (EFA) on teachers’ assessments of speech samples. After that, 104 CSL teachers used these rating items to assess the aforementioned samples. The EFAs of teachers’ assessments led to three corresponding rating criteria models (Models 1, 2, and 3). Both Models 2 and 3 for Samples 2 and 3, respectively, were five-criterion models, consisting of fluency, conceptual understanding, content richness, communication appropriateness, and communication clarity. Model 1 for Sample 1 was a four-criterion model, in which the items in communication appropriateness and content richness showed high correlations, and therefore were merged into one category; the other three criteria remained the same. Comparisons of the three models demonstrated that the criteria were constant. The ANOVAs showed that the proficiency levels of these oral performances differed significantly across all five rating criteria. This study empirically supports CSL teachers’ use of constant rating criteria to assess different levels of oral performance. It also provides Chinese teachers with rating criteria they can use to assess U.S. college students’ CSL oral performance.