An investigation into native and non-native teachers' judgments of oral English performance: A mixed methods approach

Abstract

This study used a mixed methods research approach to examine how native English-speaking (NS) and non-native English-speaking (NNS) teachers assess students' oral English performance. The evaluation behaviors of two groups of teachers (12 Canadian NS teachers and 12 Korean NNS teachers) were compared with regard to internal consistency, severity, and evaluation criteria. Results of a Many-faceted Rasch Measurement analysis showed that most of the NS and NNS teachers maintained acceptable levels of internal consistency, with only one or two inconsistent raters in each group. The two groups of teachers also exhibited similar severity patterns across different tasks. However, substantial dissimilarities emerged in the evaluation criteria teachers used to assess students' performance. A qualitative analysis demonstrated that the judgments of the NS teachers were more detailed and elaborate than those of the NNS teachers in the areas of pronunciation, specific grammar use, and the accuracy of transferred information. These findings are used as the basis for a discussion of NS versus NNS teachers as language assessors on the one hand and the usefulness of mixed methods inquiries on the other.