Finding phoneme trajectories in a feature space of sound and midsagittal ultrasound tongue images

Abstract

Supporting the development of a pronunciation learning system, this paper reports an inspection of the trajectory of speech sentences in a feature space that is constructed from midsagittal tongue images and frame-wise speech sounds. One objective of this research is to estimate tongue shape and position from speech sounds, so we focus on determining how best to construct and interpret a feature space we call MUTIS (midsagittal ultrasound tongue image space). Experimental results indicate that higher dimensions of MUTIS are most effective for separating people, and that primarily the lower dimensions of VSS (vocal sound space) data are most effective for separating phonemes. Also, the trajectories within only the VSS data indicate clear differences between first language and second language speakers, but they do not do so within only the MUTIS data. These results indicate that the ultrasound tongue image expresses individual oral cavity over a wide area, and specific tongue shape has a lower contribution in ultrasound tongue images.