Improved training procedures for hidden Markov models

Abstract

Techniques for training hidden Markov model (HMM) parameters from a labeled training set of data are well established and include the forward-backward algorithm as well as the segmental K-means algorithm. These algorithms have been shown to be capable of estimating the parameters of an HMM based on mathematically well-founded techniques. In practice, however, difficulties are often encountered when estimating some of the HMM parameters. These difficulties are generally the result of having insufficient training data to give robust and reliable parameter estimates. Typically, the model parameters most affected by having insufficient training data are the spectral parameter variance estimates, and the estimates of parameters related to the modeling of state duration. Although techniques have been proposed for improving estimates of the variances due to the effects of insufficient training data, the results have not proven adequate in some cases. As such, improved training techniques (which give better recognition performance) have been devised for controlling the minimum variance estimate of any spectral parameter, and for thresholding and clipping state duration parameter estimates. These improved training methods have been tested on several databases with good success. In addition, advanced techniques for creating multiple HMMs from the training data (i.e., for speaker independent recognition) have been devised and have proven successful for modeling large databases of training material.