Diphone subspace mixture trajectory models for HMM complementation

Abstract

This paper describes an extension of the previously reported attempt of capturing segmental transition information for speech recognition tasks [Speech Communication 27 (1) (1999) 19]. Representations in the subspace with multiple projected trajectories are discussed, employing EM-based methods to find optimal anchor points. Experimental work is carried out to illustrate that useful discriminant information is preserved in the subspace trajectories. These experiments include the development of “matched filters” to spot particular diphones in continuous speech, and the inclusion of diphone-based discriminant information into a phone-based HMM recognition framework to rerank multiple hypotheses. The difficulties in constructing the models due to the limited coverage of a sufficient amount of tokens within the phone balanced TIMIT database are discussed. The influence of the restricted diphone coverage on the rescoring results is reported. Improvements in phone recognition accuracy have been obtained on a speaker-by-speaker basis. Obtained improvements over baseline HMMs augmented with first-order derivatives suggest the importance of explicitly modelled between-phone information.