Speech-Training Aid with Time-Scaled Audiovisual Feedback of Articulatory Efforts

Abstract

Hearing-impaired children lack auditory feedback and experience difficulty in acquiring speech production. They can benefit from speech training aids providing visual feedback of key articulatory efforts. Requirements for such aid are developed through extended interaction with speech therapists and special education teachers. The aid is developed as a PC-based app for ease of distribution and use. It has two panels to enable comparison between the articulatory efforts of the learner and a teacher or a pre-recorded reference speaker. The visual feedback for an utterance is based on the information obtained from its audiovisual recording. The speech signal is processed to obtain time-varying vocal tract shape, level, and pitch. The vocal tract shape estimation uses LP-based inverse filtering, and the pitch estimation uses glottal epoch detection using Hilbert envelope for excitation enhancement. Visual feedback comprises a variable-rate animation of the lateral vocal tract shape, level, and pitch, and time-aligned display of the frontal view of the speaker's face along with playback of time-scaled speech signal. The graphical user interface and modules for signal acquisition, speech analysis, and time-scaled animation are developed and integrated using Python. The app has been tested for its functionalities and user interface and needs to be evaluated for speech training of hearing-impaired children. It may also be useful to second-language learners in improving the pronunciation of unfamiliar sounds.