Machine recognition of non-native speech: Task-specific language models versus large language models

Abstract

Automatic speech recognition (ASR) has offered a reliable foundation for measurement of young children’s reading skills and of second-language (L2) speaking skills. This is because well-fit task-specific language models (LMs) enable recognition that supports accurate scoring of pronunciation, fluency, vocabulary, usage, and grammar (Bernstein and Cheng, 2023). ASR works well in these measurement tasks because measurement of word production, disfluencies, and pronunciation errors is not very sensitive to moderate differences in word-error-rate (WER) accuracy, and because speech-interactive tasks appropriate for reading instruction or L2 assessment elicit relatively predictable responses, for which task-specific low-perplexity ASR systems achieve sufficiently accurate speech recognition (Cheng and Townshend, 2003). In the work reported here, we compared the accuracy of two English ASR systems on a set of 718 extended spontaneous speech recordings from 77 adult non-native speakers of English speaking from six countries under uncontrolled recording conditions. A Kaldi-based ASR system with well-fit task-specific LMs achieved WER 17%, while USM, a general-purpose mSLAM recognizer with an RNN-T decoder, achieved 11% WER, which is a 34% relative improvement. The mSLAM + RNN-T technology will be briefly described and an analysis of results in three different open-response interactive speaking tasks will be presented.