Speech interfaces based upon surface electromyography

Abstract

This paper discusses the use of surface electromyography (EMG) to recognize and synthesize speech. The acoustic speech signal can be significantly corrupted by high noise in the environment or impeded by garments or masks. Such situations occur, for example, when firefighters wear pressurized suits with self-contained breathing apparatus (SCBA) or when astronauts perform operations in pressurized gear. In these conditions it is important to capture and transmit clear speech commands in spite of a corrupted or distorted acoustic speech signal. One way to mitigate this problem is to use surface electromyography to capture activity of speech articulators and then, either recognize spoken commands from EMG signals or use these signals to synthesize acoustic speech commands. We describe a set of experiments for both speech recognition and speech synthesis based on surface electromyography and discuss the lessons learned about the characteristics of the EMG signal for these domains. The experiments include speech recognition in high noise based on 15 commands for firefighters wearing self-contained breathing apparatus, a sub-vocal speech robotic platform control experiment based on five words, a speech recognition experiment testing recognition of vowels and consonants, and a speech synthesis experiment based on an articulatory speech synthesizer.