Development of a visual app for improving learner's pronunciation with ultrasound and the speech accent archive

Abstract

Although many language learners desire to improve their pronunciation of a foreign language, there are not many apps to help them do so. Most commercial apps for pronunciation evaluation and training focus on only the acoustic signal. However, few systems give visual movements of native speakers’ lips, tongue, and jaw. In this paper, we describe the ongoing development of and app that is programmed in Swift for iPhone. The app incorporates and links together different kinds of phonetic data for the pronunciation learner—for example, recorded frontal and side videos of a native speaker’s face during pronunciation with an ultrasound movie of the tongue moving in the mouth overlaid. The training text is a paragraph from the Speech Accent Archive. The initial version of this app has two systems. First, listening to English sentences by a native speaker and checking tongue movement with ultrasound. Second, this app has buttons that play from user-chosen words and it also plays in slow motion. The method of the...