Automatic generation of non-uniform HMM structures based on variational Bayesian approach

Abstract

We propose using the variational Bayesian (VB) approach for automatically creating nonuniform, context-dependent HMM topologies in speech recognition. The maximum likelihood (ML) criterion is generally used to create HMM topologies. However, it has an over-fitting problem. Information criteria have been used to overcome this problem, but theoretically they cannot be applied to complicated models like HMM. Recently, to avoid these problems, the VB approach has been developed in the machine-learning field. We introduce the VB approach to the successive state splitting (SSS) algorithm, which can create both contextual and temporal variations for HMM. We define the prior and posterior probability densities and free energy with latent variables as split and stop criteria. Experimental results show that the proposed method can automatically create a more efficient model and obtain better performance, especially for vowels, than the original method.