The semi-relaxed algorithm for estimating parameters of hidden Markov models

Abstract

For hidden Markov model (HMM) based speech recognition where the basic speech unit is smaller than the recognizer's output unit, the standard full Baum-Welch re-estimation procedure for the HMM training is very costly in computation. This is because it requires evaluation of the HMM output densities and of the forward/backward probabilities in the entire region of the state-frame trellis. In this paper, we present an algorithm which exploits the fact that the entries of the trellis are essentially zero except near the block diagonal and hence achieves significant computational saving. The algorithm is evaluated in experiments with a large vocabulary word recognizer based on mixture-density HMM representation of phonemes. The HMM parameters trained with the new algorithm are essentially identical to those trained with the full Baum-Welch algorithm in that the resulting HMMs have nearly the same likelihood values on the same set of training data. Identical word recognition accuracies are yielded using the HMMs trained with the two algorithms. However, the new algorithm is shown to be about an order of magnitude faster than the full Baum-Welch algorithm.