Analysis of acoustic to articulatory speech inversion for natural speech

Abstract

Speech inversion is a technique to estimate vocal tract configurations from speech acoustics. We constructed two such systems using feedforward neural networks. One was trained using natural speech data from the XRMB database and the second using synthetic data generated by the Haskins Laboratories TADA model that approximated the XRMB data. XRMB pellet trajectories were first converted into vocal tract constriction variables (TVs), providing a relative measure of constriction kinematics (location and degree) and synthetic TV data was obtained directly using TADA. The natural and synthetic speech inversion systems were trained as TV estimators using these respective sets of acoustic and TV data. TV-estimators were first tested using previously collected acoustic data on the utterance “perfect memory” spoken at slow, normal, and fast rates. The TV estimator trained on XRMB data (but not on TADA data) was able to recover the tongue tip gesture for /t/ in the fast utterance despite the gesture occurring partly during the acoustic silence of the closure. Further, the XRMB system (but not the TADA system) could distinguish between bunched and retroflexed /r/. Finally, we compared the performance of the XRMB system with a set of independently trained speaker-dependent systems (using the XRMB database) to understand the role of speaker-specific differences in the partitioning of variability across acoustic and articulatory spaces.