Automatic Model Complexity Control Using Marginalized Discriminative Growth Functions

Abstract

Selecting the model structure with the "appropriate" complexity is a standard problem for training large-vocabulary continuous-speech recognition (LVCSR) systems. State-of-the-art LVCSR systems are highly complex. A wide variety of techniques may be used which alter the system complexity and word error rate (WER). Explicitly evaluating systems for all possible configurations is infeasible; hence, an automatic model complexity control criterion is highly desirable. Most existing complexity control schemes can be classified into two types, Bayesian learning techniques and information theory approaches. An implicit assumption is made in both, that increasing the likelihood on held-out data decreases the WER. However, this correlation has been found quite weak for current speech recognition systems. This paper presents a novel discriminative complexity control technique, the marginalization of a discriminative growth function. This is a closer approximation to the true WER than standard approaches. Experimental results on a standard LVCSR Switchboard task showed that marginalized discriminative growth functions outperforms manually tuned systems and conventional complexity control techniques, such as Bayesian information criterion (BIC), in terms of WER