Variational Bayesian approach for automatic generation of HMM topologies

Abstract

We propose a new method of automatically creating non-uniform, context-dependent HMM topologies by using the variational Bayesian (VB) approach. The maximum likelihood (ML) criterion is generally used to create HMM topologies. However, it has an overfitting problem. Information criteria have been used to overcome this problem, but, theoretically, they cannot be applied to complicated models like HMMs. Recently, to avoid these problems, a VB approach has been developed in the machine-learning field. The successive state splitting (SSS) algorithm is a method of creating contextual and temporal variations for HMMs. We introduce the VB approach to the SSS algorithm, and define the prior and posterior probability densities and free energy as split and stop criteria. Experimental results show that the proposed method can automatically create the proper model and obtain better performance, especially for vowels, than the original method.