Automatically Assessing Lexical Sophistication: Indices, Tools, Findings, and Application

Abstract

This study explores the construct of lexical sophistication and its applications for measuring second language lexical and speaking proficiency. In doing so, the study introduces the Tool for the Automatic Analysis of LExical Sophistication (TAALES), which calculates text scores for 135 classic and newly developed lexical indices related to word frequency, range, bigram and trigram frequency, academic language, and psycholinguistic word information. TAALES is freely available; runs on Windows, Mac, and Linux operating systems; and has a simple graphic user interface that allows for batch processing of .txt files. The tool is fast, reliable, and outputs results to a comma‐separated value file that can be accessed using spreadsheet software. The study examines the ability of TAALES indices to explain the variance in human judgments of lexical proficiency and speaking proficiency for second language (L2) learners. Overall, these indices were able to explain 47.5% of the variance in holistic scores of lexical proficiency and 48.7% of the variance in holistic scores of speaking proficiency. This study has important implications for second language acquisition, for assessing L2 learners' productive skills (writing and speaking), and for L2 pedagogy. Limitations and future directions are also discussed.