A Discriminative Training Algorithm for Hidden Markov Models

Abstract

We introduce a discriminative training algorithm for the estimation of hidden Markov model (HMM) parameters. This algorithm is based on an approximation of the maximum mutual information (MMI) objective function and its maximization in a technique similar to the expectation-maximization (EM) algorithm. The algorithm is implemented by a simple modification of the standard Baum-Welch algorithm, and can be applied to speech recognition as well as to word-spotting systems. Three tasks were tested: isolated digit recognition in a noisy environment, connected digit recognition in a noisy environment and word-spotting. In all tasks a significant improvement over maximum likelihood (ML) estimation was observed. We also compared the new algorithm to the commonly used extended Baum-Welch MMI algorithm. In our tests the algorithm showed advantages in terms of both performance and computational complexity.