Inversion-based nonlinear adaptation of noisy acoustic parameters for a neural/HMM speech recognizer

Abstract

Spoken human–machine interaction in real-world environments requires acoustic models that are robust to changes in acoustic conditions, e.g. presence of noise. Unfortunately, the popular hidden Markov models (HMM) are not noise tolerant. One way to increase recognition performance is to acquire a small adaptation set of noisy utterances, which is used to estimate a normalization mapping between noisy and clean features to be fed into the acoustic model. This paper proposes an unsupervised maximum-likelihood gradient-ascent training algorithm (instead of the usual least squares regression) for a neural feature adaptation module, properly combined with a hybrid connectionist/HMM speech recognizer. The algorithm is inspired by the so-called “inversion principle”, that prescribes the optimization of the input features instead of the model parameters. Simulation results on a real-world speaker-independent continuous speech corpus of connected Italian digits, corrupted by noise, validate the approach. A small neural net (13 hidden neurons) trained over a single adaptation utterance for one iteration yields a 18.79% relative word error rate (WER) reduction over the bare hybrid, and a 65.10% relative WER reduction over the Gaussian-based HMM.