Real-time visual acoustic feedback for non-native vowel production training

Abstract

Computer-Assisted Pronunciation Training (CAPT) software which include visual feedback have shown potential for second language (L2) learning (Mehta & Katz, 2018). However, real-time formant visualization has yet to be used for L2 learners' production of speech sounds. This project investigates the effects of real-time formant visualization on L2 vowel production training. First, a vowel visualization system is presented. In contrast with other similar tools which use a regression approach (Frostel et al., 2011), this real-time formant (F1 and F2) extraction is performed using the Parselmouth library (Jadoul et al., 2018), which then uses Praat's formant extraction algorithm (Gray & Wong, 1980) to ensure accuracy. Then, the results from a preliminary user study are reported. L2 learners (n = 5) participated in six 30-min training sessions, during which they used the system to practice their production of eight vowels. Pronunciation accuracy was analyzed by calculating Mahalanobis distances between participants' productions (produced in pre-test and post-test) and native exemplars. The analysis of this preliminary dataset showed that the use of real-time visual acoustic feedback resulted in L2 vowel production improvement, which suggests that this system could, therefore, be used as a pedagogical tool for L2 learners.