Assessing Lexical Proficiency Using Analytic Ratings: A Case for Collocation Accuracy

Abstract

This study analyzes lexical proficiency in oral and written texts produced by second language (L2) learners of English. The purpose of the study is to examine relationships between analytic scores of depth of lexical knowledge, breadth of lexical knowledge, and access to core lexical items and holistic scores of lexical proficiency. A corpus of 240 spoken texts and 240 written texts produced by beginning L2 learners, intermediate L2 learners, advanced L2 learners, and native speakers were scored for both analytic and holistic features of lexical proficiency by trained raters. Using a multiple regression analysis, the study found that collocation accuracy, lexical diversity, and word frequency are significant predictors of human evaluations of lexical proficiency. Collocation accuracy explained the greatest amount of variance in the holistic scores (84% in the written samples and 89% in the spoken samples). The authors discuss the importance of collocation accuracy in predicting human judgments of lexical proficiency.