Ergodic hidden Markov models and polygrams for language modeling

Abstract

Presents two new techniques for language modeling in speech recognition. The first technique is based on ergodic discrete density hidden Markov models (HMM) which can be applied to bigrams based on word categories. This statistical approach of the so-called Markov bigrams enables an efficient unsupervised learning procedure for the bigram probabilities with the well-known Baum-Welch algorithm. Furthermore, maximizing the model-conditional probability is equivalent to minimizing the perplexity of the training corpus. The second technique is based on polygrams which are an extension of the bigram (n=2) or trigram (n=3) grammars to any possible value of n. According to the smoothing techniques for bigram or trigram models, the probabilities of the n-grams in the polygram model are interpolated using the relative frequencies of all n'-grams with n'/spl les/n. Both techniques were evaluated on the ATIS corpus by computing the test set perplexity. Furthermore the authors integrated the Markov bigrams as well as the polygrams into their word recognizer for continuous speech. Experimental results on a German database are discussed using the N-best paradigm to reorder the generated word sequences according to the sentence probability of the language model. >