Minimum Classification Error Learning Research Articles

Minimum classification error learning for sequential data in the wavelet domain

Wavelet analysis has found widespread use in signal processing and many classification tasks. Nevertheless, its use in dynamic pattern recognition have been much more restricted since most of wavelet models cannot handle variable length sequences properly. Recently, composite hidden Markov models which observe structured data in the wavelet domain were proposed to deal with this kind of sequences. In these models, hidden Markov trees account for local dynamics in a multiresolution framework, while standard hidden Markov models capture longer correlations in time. Despite these models have shown promising results in simple applications, only generative approaches have been used so far for parameter estimation. The goal of this work is to take a step forward in the development of dynamic pattern recognizers using wavelet features by introducing a new discriminative training method for this Markov models. The learning strategy relies on the minimum classification error approach and provides re-estimation formulas for fully non-tied models. Numerical experiments on phoneme recognition show important improvement over the recognition rate achieved by the same models trained using maximum likelihood estimation.

Deterministic Neural Classification

This letter presents a minimum classification error learning formulation for a single-layer feedforward network (SLFN). By approximating the nonlinear counting step function using a quadratic function, the classification error rate is shown to be deterministically solvable. Essentially the derived solution is related to an existing weighted least-squares method with class-specific weights set according to the size of data set. By considering the class-specific weights as adjustable parameters, the learning formulation extends the classification robustness of the SLFN without sacrificing its intrinsic advantage of being a closed-form algorithm. While the method is applicable to other linear formulations, our empirical results indicate SLFN's effectiveness on classification generalization.

Statistical Analysis of Minimum Classification Error Learning for Gaussian and Hidden Markov Model Classifiers

Minimum classification error learning realized via generalized probabilistic descent, usually referred to as (MCE/GPD), is a very popular and powerful framework for building classifiers. This paper first presents a theoretical analysis of MCE/GPD. The focus is on a simple classification problem for estimating the means of two Gaussian classes. For this simple algorithm, we derive difference equations for the class means and decision threshold during learning, and develop closed form expressions for the evolution of both the smoothed and true error. In addition, we show that the decision threshold converges to its optimal value, and provide an estimate of the number of iterations needed to approach convergence. After convergence the class means drift towards increasing their distance to infinity without contributing to the decrease of the classification error. This behavior, referred to as mean drift, is then related to the increase of the variance of the classifier. The theoretical results perfectly agree with simulations carried out for a two-class Gaussian classification problem. In addition to the obtained theoretical results we experimentally verify, in speech recognition experiments, that MCE/GPD learning of Gaussian mixture hidden Markov models qualitatively follows the pattern suggested by the theoretical analysis. We also discuss links between MCE/GPD learning and both batch gradient descent and extended Baum-Welch re-estimation. The latter two approaches are known to be popular in large scale implementations of discriminative training. Hence, the proposed analysis can be used, at least as a rough guideline, for better understanding of the properties of discriminative training algorithms for speech recognition.

Rapid discriminative learning based on misclassification measure

AbstractThe current research is based on Minimum Classification Error Learning (MCE/GPD) using Generalized Probabilistic Descent (GPD), which is known as a high‐performance discriminative learning method. MCE/GPD is an excellent recognition technique that has been applied to speech recognition because of its high recognition performance and its ability to deal with variable‐length vectors. However, like other recognition techniques, it suffers from the problem that recognition performance drops for untrained data (generalization ability problem). There is also the practical fault that training time is lengthy due to the complexity of the algorithm. In the current research, the authors propose a new learning method that improves the generalization ability by introducing regularized learning to avoid ill‐posed problems and increases learning speed according to a hierarchical model arrangement, which should solve these two problems. They used a hierarchical neural network for performance evaluation. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(3): 58–68, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20353

Pattern recognition using a family of design algorithms based upon the generalized probabilistic descent method

This paper provides a comprehensive introduction to a novel approach to pattern recognition which is based on the generalized probabilistic descent method (GPD) and its related design algorithms. The paper contains a survey of recent recognizer design techniques, the formulation of GPD, the concept of minimum classification error learning that is closely related to the GPD formalization, a relational analysis between GPD and other important design methods, and various embodiments of GPD-based design, including segmental-GPD, minimum spotting error training, discriminative utterance verification, and discriminative feature extraction. GPD development has its origins in basic pattern recognition and Bayes decision theory. It represents a simple but careful re-investigation of the classical theory and successfully leads to an innovative framework. For clarity of presentation, detailed discussions about its embodiments are provided for examples of speech pattern recognition tasks that use a distance-based classifier. Experimental results in speech pattern recognition tasks clearly demonstrate the remarkable utility of the family of GPD-based design algorithms.