Minimum Classification Error Training Research Articles

Robust and Efficient Pattern Classification using Large Geometric Margin Minimum Classification Error Training

Recently, one of the standard discriminative training methods for pattern classifier design, i.e., Minimum Classification Error (MCE) training, has been revised, and its new version is called Large Geometric Margin Minimum Classification Error (LGM-MCE) training. It is formulated by replacing a conventional misclassification measure, which is equivalent to the so-called functional margin, with a geometric margin that represents the geometric distance between an estimated class boundary and its closest training pattern sample. It seeks the status of the trainable classifier parameters that simultaneously correspond to the minimum of the empirical average classification error count loss and the maximum of the geometric margin. Experimental evaluations showed the fundamental utility of LGM-MCE training. However, to increase its effectiveness, this new training required careful setting for hyperparameters, especially the smoothness degree of the smooth classification error count loss. Exploring the smoothness degree usually requires many trial-and-error repetitions of training and testing, and such burdensome repetition does not necessarily lead to an optimal smoothness setting. To alleviate this problem and further increase the effect of geometric margin employment, we apply in this paper a new idea that automatically determines the loss smoothness of LGM-MCE training. We first introduce a new formalization of it using the Parzen estimation of error count risk and formalize LGM-MCE training that incorporates a mechanism of automatic loss smoothness determination. Importantly, the geometric-margin-based misclassification measure adopted in LGM-MCE training is directly linked with the geometric margin in a pattern sample space. Based on this relation, we also prove that loss smoothness affects the production of virtual samples along the estimated class boundaries in pattern sample space. Finally, through experimental evaluations and in comparisons with other training methods, we elaborate the characteristics of LGM-MCE training and its new function that automatically determines an appropriate loss smoothness degree.

Minimum Classification Error Training Incorporating Automatic Loss Smoothness Determination

Minimum Classification Error (MCE) training, which has been widely used as one of the recent standards of discriminative training for classifiers, is characterized by a smooth sigmoidal-form classification error count loss. The smoothness of this loss function effectively increases training robustness to unseen samples, well approximates the ultimate, minimum classification error probability status, and leads to accurate classification over unseen samples. However, few rational methods have been developed for controlling the smoothness, which is often determined through many repetitions of the experimental setting; this empirical approach has been a disincentive to the increased popularization of MCE training. To alleviate this long-standing problem, we propose a new MCE training method that automatically determines loss smoothness. The proposed method is based on Parzen-estimation-based MCE re-formalization, and the loss smoothness degree is determined so that Parzen distribution can be an accurate approximation to the unknown true distribution, whose positive-domain integration corresponds to classification error probability, in one-dimensional misclassification measure space. Through systematic experiments, we show that the proposed method efficiently yields a classification accuracy that nearly matches the best accuracy obtained by the conventional, trial-and-error-mode repetitions of smoothness setting.

Optimization of Spoken Term Detection System

Generally speaking, spoken term detection system will degrade significantly because of mismatch between acoustic model and spontaneous speech. This paper presents an improved spoken term detection strategy, which integrated with a novel phoneme confusion matrix and an improved word‐level minimum classification error (MCE) training method. The first technique is presented to improve spoken term detection rate while the second one is adopted to reject false accepts. On mandarin conversational telephone speech (CTS), the proposed methods reduce the equal error rate (EER) by 8.4% in relative.

Preference Music Ratings Prediction Using Tokenization and Minimum Classification Error Training

In order to address two main limitations of current content-based music recommendation approaches, an ordinal regression algorithm for music recommendation that incorporates dynamic information is presented. Instead of assuming that local spectral features within a song are identically and independently distributed examples of an underlying probability density, music is characterized by a vocabulary of acoustic segment models (ASMs), which are found with an unsupervised process. Further, instead of classifying music based on subjective classes, such as genre, or trying to find a universal notion of similarity, songs are classified based on personal preference ratings. The ordinal regression approach to perform the ratings prediction is based on the discriminative-training algorithm known as minimum classification error (MCE) training. Experimental results indicate that improved temporal modeling leads to superior performance over standard spectral-based music representations. Further, the MCE-based preference ratings algorithm is shown to be superior over two other systems. Analysis demonstrates that the superior performance is due to MCE being a non-conservative algorithm that demonstrates immunity to outliers.

MCE Training Techniques for Topic Identification of Spoken Audio Documents

In this paper, we discuss the use of minimum classification error (MCE) training as a means for improving traditional approaches to topic identification such as naive Bayes classifiers and support vector machines. A key element of our new MCE training techniques is their ability to efficiently apply jackknifing or leave-one-out training to yield improved models which generalize better to unseen data. Experiments were conducted using recorded human-human telephone conversations from the Fisher Corpus using feature vector representations from word-based automatic speech recognition lattices. Sizeable improvements in topic identification accuracy using the new MCE training techniques were observed.

Building compact recognizers of handwritten Chinese characters using precision constrained Gaussian model, minimum classification error training and parameter compression

In our previous work, a so-called precision constrained Gaussian model (PCGM) was proposed for character modeling to design compact recognizers of handwritten Chinese characters. A maximum likelihood training procedure was developed to estimate model parameters from training data. In this paper, we extend the above-mentioned work by using minimum classification error (MCE) training to improve recognition accuracy and using both split vector quantization and scalar quantization techniques to further compress model parameters. Experimental results on a handwritten character recognition task with a vocabulary of 2,965 Kanji characters demonstrate that MCE-trained and compressed PCGM-based classifiers can achieve much higher recognition accuracies than their counterparts based on traditional modified quadratic discriminant function (MQDF) when the footprint of the classifiers has to be made very small, e.g., less than 2 MB.

An Approach Using Combination of Multiple Features through Sigmoid Function for Speech-Presence/Absence Discrimination

In this paper, we present an approach of detecting speech presence for which the decision rule is based on a combination of multiple features using a sigmoid function. A minimum classification error (MCE) training is used to update the weights adjustment for the combination. The features, consisting of three parameters: the ratio of ZCR, the spectral energy, and spectral entropy, are combined linearly with weights derived from the sub-band domain. First, the Bark-scale wavelet decomposition (BSWD) is used to split the input speech into 24 critical sub-bands. Next, the feature parameters are derived from the selected frequency sub-band to form robust voice feature parameters. In order to discard the seriously corrupted frequency sub-band, a strategy of adaptive frequency sub-band extraction (AFSE) dependant on the sub-band SNR is then applied to only the frequency sub-band used. Finally, these three feature parameters, which only consider the useful sub-band, are combined through a sigmoid type function incorporating optimal weights based on MSE training to detect either a speech present frame or a speech absent frame. Experimental results show that the performance of the proposed algorithm is superior to the standard methods such as G.729B and AMR2.

Minimum Classification Error Training of Hidden Markov Models for Sequential Data in the Wavelet Domain

In the last years there has been increasing interest in developing discriminative training methods for hidden Markov models, with the aim to improve their performance in classi cation and pattern recognition tasks. Although several advances have been made in this area, they have been targeted almost exclusively to standard models whose conditional observations are given by a Gaussian mixture density. In parallel with this development, a special kind of hidden Markov models de ned in the wavelet domain has found wide-spread use in the signal and image processing community. Nevertheless, these models have been typically restricted to fully-tied parameter training using a single sequence and maximum likelihood estimates. This paper takes a step forward in the development of sequential pattern recognizers based on wavelet-domain hidden Markov models by introducing a new discriminative training method. The learning strategy relies on the minimum classi cation error approach and provides reestimation formulas for fully non-tied models. Numerical experiments on a simple phoneme recognition task show important improvement over the recognition rate achieved by the same models trained under the maximum likelihood estimation approach.

Minimum classification error training in example based speech and pattern recognition using sparse weight matrices

The Minimum Classification Error (MCE) criterion is a well-known criterion in pattern classification systems. The aim of MCE training is to minimize the resulting classification error when trying to classify a new data set. Usually, these classification systems use some form of statistical model to describe the data. These systems usually do not work very well when this underlying model is incorrect. Speech recognition systems traditionally use Hidden Markov Models (HMM) with Gaussian (or Gaussian mixture) probability density functions as their basic model. It is well known that these models make some assumptions that are not correct. In example based approaches, these statistical models are absent and are replaced by the pure data. The absence of statistical models has created the need for parameters to model the data space accurately. For this work, we use the MCE criterion to create a system that is able to work together with this example based approach. Moreover, we extend the locally scaled distance measure with sparse, block diagonal weight matrices resulting in a better model for the data space and avoiding the computational load caused by using full matrices. We illustrate the approach with some example experiments on databases from pattern recognition and with speech recognition.

An Ensemble Speaker and Speaking Environment Modeling Approach to Robust Speech Recognition

We propose an ensemble speaker and speaking environment modeling (ESSEM) approach to characterizing environments in order to enhance performance robustness of automatic speech recognition systems under adverse conditions. The ESSEM process comprises two phases, the offline and the online. In the offline phase, we prepare an ensemble speaker and speaking environment space formed by a collection of super-vectors. Each super-vector consists of the entire set of means from all the Gaussian mixture components of a set of hidden Markov models that characterizes a particular environment. In the online phase, with the ensemble environment space prepared in the offline phase, we estimate the super-vector for a new testing environment based on a stochastic matching criterion. In this paper, we focus on methods for enhancing the construction and coverage of the environment space in the offline phase. We first demonstrate environment clustering and partitioning algorithms to structure the environment space well; then, we propose a minimum classification error training algorithm to enhance discrimination across environment super-vectors and therefore broaden the coverage of the ensemble environment space. We evaluate the proposed ESSEM framework on the Aurora2 connected digit recognition task. Experimental results verify that ESSEM provides clear improvement over a baseline system without environmental compensation. Moreover, the performance of ESSEM can be further enhanced by using well-structured environment spaces. Finally, we confirm that ESSEM gives the best overall performance with an environment space refined by an integration of all techniques.

Minimum Rank Error Language Modeling

Statistical language modeling has been successfully developed for speech recognition and information retrieval. The minimum classification error (MCE) training was undertaken to enhance speech recognition performance by minimizing the word error rate. This paper presents a new minimum rank error (MRE) algorithm for n-gram language model training. Rather than speech recognition, the proposed language models are estimated for information retrieval by considering the metric of average precision. However, the maximization of average precision is closely linked to minimizing the rank error or optimizing the order of the ranked documents. Accordingly, this paper calculates the rank error loss function from the misordering pairs of relevant and irrelevant documents in the rank list. The Bayes risk due to the expected rank loss is minimized to develop the Bayesian retrieval rule for ad-hoc information retrieval. Consequently, the discriminative training of language model is performed by integrating discrimination information from individual relevant documents relative to their corresponding irrelevant documents. Experimental results on TREC collections indicate that the proposed MRE language model improves the order of relevant documents, and degrades that of irrelevant documents. The MRE method achieves significantly higher average precision for test queries than the maximum likelihood and the MCE retrieval models.

Large-margin minimum classification error training: A theoretical risk minimization perspective

Large-margin discriminative training of hidden Markov models has received significant attention recently. A natural and interesting question is whether the existing discriminative training algorithms can be extended directly to embed the concept of margin. In this paper, we give this question an affirmative answer by showing that the sigmoid bias in the conventional minimum classification error (MCE) training can be interpreted as a soft margin. We justify this claim from a theoretical classification risk minimization perspective where the loss function associated with a non-zero sigmoid bias is shown to include not only empirical error rates but also a margin-bound risk. Based on this perspective, we propose a practical optimization strategy that adjusts the margin (sigmoid bias) incrementally in the MCE training process so that a desirable balance between the empirical error rates on the training set and the margin can be achieved. We call this modified MCE training process large-margin minimum classification error (LM-MCE) training to differentiate it from the conventional MCE. Speech recognition experiments have been carried out on two tasks. First, in the TIDIGITS recognition task, LM-MCE outperforms the state-of-the-art MCE method with 17% relative digit-error reduction and 19% relative string-error reduction. Second, on the Microsoft internal large vocabulary telephony speech recognition task (with 2000 h of training data and 120 K words in the vocabulary), significant recognition accuracy improvement is achieved, demonstrating that our formulation of LM-MCE can be successfully scaled up and applied to large-scale speech recognition tasks.

Minimum Classification Error Training for Choquet Integrals With Applications to Landmine Detection

A novel algorithm for discriminative training of Choquet-integral-based fusion operators is described. Fusion is performed by Choquet integration of classifier outputs with respect to fuzzy measures. The fusion operators are determined by the parameters of fuzzy measures. These parameters are found by minimizing a minimum classification error (MCE) objective function. The minimization is performed with respect to a special class of measures, the Sugeno lambda-measures. An analytic expression is derived for the gradient of the Choquet integral with respect to the Sugeno lambda-measure. The new algorithm is applied to a landmine detection problem, and compared to previous techniques.

Unsupervised Discriminative Training With Application to Dialect Classification

Automatic dialect classification has gained interest in the field of speech research because of its importance in characterizing speaker traits and knowledge estimation which could improve integrated speech technology (e.g., speech recognition, speaker recognition). This study addresses novel advances in unsupervised spontaneous dialect classification in English and Spanish. The problem considers the case where no transcripts are available for training and test data, and speakers are talking spontaneously. The Gaussian mixture model (GMM) is used for unsupervised dialect classification in our study. Techniques which aim to deal with confused acoustic regions in the GMMs are proposed, where confused regions in the GMMs are identified through data driven methods. The first technique excludes confused regions by finding dialect dependence in the untranscribed audio by selecting the most discriminative Gaussian mixtures [mixture selection (MS)]. The second technique includes the confused regions in the model, but the confused regions are balanced over all classes. This technique is implemented by identifying discriminative frames and confused frames in the audio data [frame selection (FS)]. The new confused regions contribute to model representation but does not impact classification performance. The third technique is to reduce the confused regions in the original model. Minimum classification error (MCE) is applied to achieve this objective. All three techniques implement discriminative training for GMM-based classification. Both the first technique (MS-GMM, GMM trained with mixture selection) and the second technique (FS-GMM, GMM trained with frame selection) improve dialect classification performance. Further improvement is achieved after applying the third technique (MCE training) before the first or second techniques. The system is evaluated using British English dialects and Latin American Spanish dialects. Measurable improvement is achieved in both corpora. Finally, the system is compared with human listener performance, and shown to outperform human listeners in terms of classification accuracy.

A new look at discriminative training for hidden Markov models

Discriminative training for hidden Markov models (HMMs) has been a central theme in speech recognition research for many years. One most popular technique is minimum classification error (MCE) training, with the objective function closely related to the empirical error rate and with the optimization method based traditionally on gradient descent. In this paper, we provide a new look at the MCE technique in two ways. First, we develop a non-trivial framework in which the MCE objective function is re-formulated as a rational function for multiple sentence-level training tokens. Second, using this novel re-formulation, we develop a new optimization method for discriminatively estimating HMM parameters based on growth transformation or extended Baum–Welch algorithm. Technical details are given for the use of lattices as a rich representation of competing candidates for the MCE training.

Discriminative Training for Large-Vocabulary Speech Recognition Using Minimum Classification Error

The minimum classification error (MCE) framework for discriminative training is a simple and general formalism for directly optimizing recognition accuracy in pattern recognition problems. The framework applies directly to the optimization of hidden Markov models (HMMs) used for speech recognition problems. However, few if any studies have reported results for the application of MCE training to large-vocabulary, continuous-speech recognition tasks. This article reports significant gains in recognition performance and model compactness as a result of discriminative training based on MCE training applied to HMMs, in the context of three challenging large-vocabulary (up to 100 k word) speech recognition tasks: the Corpus of Spontaneous Japanese lecture speech transcription task, a telephone-based name recognition task, and the MIT Jupiter telephone-based conversational weather information task. On these tasks, starting from maximum likelihood (ML) baselines, MCE training yielded relative reductions in word error ranging from 7% to 20%. Furthermore, this paper evaluates the use of different methods for optimizing the MCE criterion function, as well as the use of precomputed recognition lattices to speed up training. An overview of the MCE framework is given, with an emphasis on practical implementation issues

Approximate Test Risk Bound Minimization Through Soft Margin Estimation

Inspired by the great success of margin-based classifiers, there is a trend to incorporate the margin concept into hidden Markov modeling for speech recognition. Several attempts based on margin maximization were proposed recently. In this paper, a new discriminative learning framework, called soft margin estimation (SME), is proposed for estimating the parameters of continuous-density hidden Markov models. The proposed method makes direct use of the successful ideas of soft margin in support vector machines to improve generalization capability and decision feedback learning in minimum classification error training to enhance model separation in classifier design. SME is illustrated from a perspective of statistical learning theory. By including a margin in formulating the SME objective function, SME is capable of directly minimizing an approximate test risk bound. Frame selection, utterance selection, and discriminative separation are unified into a single objective function that can be optimized using the generalized probabilistic descent algorithm. Tested on the TIDIGITS connected digit recognition task, the proposed SME approach achieves a string accuracy of 99.43%. On the 5 k-word Wall Street Journal task, SME obtains relative word error rate reductions of about 10% over our best baseline results in different experimental configurations. We believe this is the first attempt to show the effectiveness of margin-based acoustic modeling for large vocabulary continuous speech recognition in a hidden Markov model framework. Further improvements are expected because the approximate test risk bound minimization principle offers a flexible and rigorous framework to facilitate incorporation of new margin-based optimization criteria into hidden Markov model training.

An Environment-Compensated Minimum Classification Error Training Approach Based on Stochastic Vector Mapping

A conventional feature compensation module for robust automatic speech recognition is usually designed separately from the training of hidden Markov model (HMM) parameters of the recognizer, albeit a maximum-likelihood (ML) criterion might be used in both designs. In this paper, we present an environment-compensated minimum classification error (MCE) training approach for the joint design of the feature compensation module and the recognizer itself. The feature compensation module is based on a stochastic vector mapping function whose parameters have to be learned from stereo data in a previous approach called SPLICE. In our proposed MCE joint design approach, by initializing the parameters with an approximate ML training procedure, the requirement of stereo data can be removed. By evaluating the proposed approach on Auroral connected digits database, a digit recognition error rate, averaged on all three test sets, of 5.66% is achieved for multicondition training. In comparison with the performance achieved by the baseline system using ETSI advanced front-end, our approach achieves an additional overall error rate reduction of 12.4%

Investigation of model‐based error rate estimation techniques for speaker‐independent large‐vocabulary recognition tasks

This paper proposes model‐based error rate estimation techniques for HMM‐based speech recognition systems and shows their effectiveness in several speaker‐independent task conditions. The proposed techniques are based on an HMM‐based error rate estimation method [C.‐S. Huang et al., IEEE Trans. SAP, 11, No. 6, 581–589 (2003)], which only depends on subword‐based pronunciation lexicon knowledge. This study assumes the need of the dynamic change of open vocabulary. The vocabulary size is varied between 5000 and 25 000 words. Two types of formalizations are investigated. They utilize the Bhattacharyya distance as a local distance measure between two competing word classes which are modeled by syllable‐unit HMMs. A misclassification measure, which is similar to the one used by minimum classification error (MCE) training methods, is used for estimating error rate. The correlation between the error rates for testing utterances by 20 speakers and the estimated error rates for varied vocabulary sizes are evalu...

Minimum classification error training for online handwriting recognition

This paper describes an application of the Minimum Classification Error (MCE) criterion to the problem of recognizing online unconstrained-style characters and words. We describe an HMM-based, character and word-level MCE training aimed at minimizing the character or word error rate while enabling flexibility in writing style through the use of multiple allographs per character. Experiments on a writer-independent character recognition task covering alpha-numerical characters and keyboard symbols show that the MCE criterion achieves more than 30 percent character error rate reduction compared to the baseline Maximum Likelihood-based system. Word recognition results, on vocabularies of 5k to 10k, show that MCE training achieves around 17 percent word error rate reduction when compared to the baseline Maximum Likelihood system.