Learning the association between speech acoustics and articulatory trajectories with a neural network

Abstract

Because the relationship between the motion of the speech articulators and the acoustic signal produced is nonlinear and of great complexity, it has previously proved impossible to accurately infer the articulatory motions that produced natural continuous speech. However, a carefully tuned back‐propagation neural network has the capability of learning such complex relationships. The present results show that such a network can learn the map between a female speaker's continuous speech and the x‐ray microbeam record of the associated articulator movements. The discovered map can then be used to successfully infer the movements of a male speaker saying the same thing (r=0.82 between inferred and actual tongue tip vertical position). The Bark scaled spectrum of counting at a normal rate from 1 to 10 was used as input, and separate nets learned the position of the tongue tip, body, dorsum, and lower lip. Alternative input normalization schemes, network configurations, and training procedures were evaluated. [...