Generalized Discriminant Analysis Research Articles

Pose-Robust Facial Expression Recognition Using View-Based 2D $+$ 3D AAM

This paper proposes a pose-robust face tracking and facial expression recognition method using a view-based 2D 3D active appearance model (AAM) that extends the 2D 3D AAM to the view-based approach, where one independent face model is used for a specific view and an appropriate face model is selected for the input face image. Our extension has been conducted in many aspects. First, we use principal component analysis with missing data to construct the 2D 3D AAM due to the missing data in the posed face images. Second, we develop an effective model selection method that directly uses the estimated pose angle from the 2D 3D AAM, which makes face tracking pose-robust and feature extraction for facial expression recognition accurate. Third, we propose a double-layered generalized discriminant analysis (GDA) for facial expression recognition. Experimental results show the following: 1) The face tracking by the view-based 2D 3D AAM, which uses multiple face models with one face model per each view, is more robust to pose change than that by an integrated 2D 3D AAM, which uses an integrated face model for all three views; 2) the double-layered GDA extracts good features for facial expression recognition; and 3) the view-based 2D 3D AAM outperforms other existing models at pose-varying facial expression recognition.

Support vector machine-based arrhythmia classification using reduced features of heart rate variability signal

This paper presents an effective cardiac arrhythmia classification algorithm using the heart rate variability (HRV) signal. The proposed algorithm is based on the generalized discriminant analysis (GDA) feature reduction scheme and the support vector machine (SVM) classifier. Initially 15 different features are extracted from the input HRV signal by means of linear and nonlinear methods. These features are then reduced to only five features by the GDA technique. This not only reduces the number of the input features but also increases the classification accuracy by selecting most discriminating features. Finally, the SVM combined with the one-against-all strategy is used to classify the HRV signals. The proposed GDA- and SVM-based cardiac arrhythmia classification algorithm is applied to input HRV signals, obtained from the MIT-BIH arrhythmia database, to discriminate six different types of cardiac arrhythmia. In particular, the HRV signals representing the six different types of arrhythmia classes including normal sinus rhythm, premature ventricular contraction, atrial fibrillation, sick sinus syndrome, ventricular fibrillation and 2 degrees heart block are classified with an accuracy of 98.94%, 98.96%, 98.53%, 98.51%, 100% and 100%, respectively, which are better than any other previously reported results. An effective cardiac arrhythmia classification algorithm is presented. A main advantage of the proposed algorithm, compared to the approaches which use the ECG signal itself is the fact that it is completely based on the HRV (R-R interval) signal which can be extracted from even a very noisy ECG signal with a relatively high accuracy. Moreover, the usage of the HRV signal leads to an effective reduction of the processing time, which provides an online arrhythmia classification system. A main drawback of the proposed algorithm is however that some arrhythmia types such as left bundle branch block and right bundle branch block beats cannot be detected using only the features extracted from the HRV signal.

Face authentication using a hybrid approach

We present a hybrid approach to face feature extraction based on the trace transform and the novel kernel partial leastsquares discriminant analysis (KPA). The hybrid approach, called trace kernel partial least-squares discriminant analysis (TKPA), first uses a set of 15 trace functionals to derive robust and discriminative facial features and then applies the KPA method to reduce their dimensionality. The feasibility of the proposed approach was successfully tested on the XM2VTS database, where a false rejection rate (FRR) of 1.25% and a false acceptance rate (FAR) of 2.11% were achieved in our best-performing face authentication experiment. The experimental results also show that the proposed approach can outperform kernel methods such as generalized discriminant analysis (GDA), kernel Fisher analysis (KFA), and complete kernel Fisher discriminant analysis (CKFA) as well as combinations of these methods with features extracted using the trace transform.

Uncooled infrared-imaging face recognition using kernel-based generalized discriminant analysis

We investigate the application of the kernel-based generalized discriminant analysis (GDA) approach to the recognition of infrared face images collected using a low-resolution uncooled IR camera. Results show that a low-cost, low-resolution IR camera combined with an efficient classifier is a usable tool in uncooled IR face recognition applications, and that best GDA-based recognition performance improves over that obtained with the fisherface approach by 3.96 percentage points, from 94.59% to 98.55%, on the data considered. This study also investigates the effects the number of projection vectors used in the GDA step, the kernel expression, and the specific distance type have on recognition performance. Results show the Mahalanobis angular distance to be the best choice, and that the recognizer computational load may be reduced by decreasing the number of eigenvectors selected in the GDA projection step without significant impact on recognition performance.

Recognition of Electromyographic Signals Using Cascaded Kernel Learning Machine

Electromyographic (EMG) signals recognition is a complex pattern recognition problem due to its property of large variations in signals and features. This paper proposes a novel EMG classifier called cascaded kernel learning machine (CKLM) to achieve the goal of high-accuracy EMG recognition. First, the EMG signals are acquired by three surface electrodes placed on three different muscles. Second, EMG features are extracted by autoregressive model (ARM) and EMG histogram. After the feature extraction, the CKLM is performed to classify the features. CKLM is composed of two different kinds of kernel learning machines: generalized discriminant analysis (GDA) algorithm and support vector machine (SVM). By using GDA, both the goals of the dimensionality reduction of input features and the selection of discriminating features, named kernel FisherEMG, can be reached. Then, SVM combined with one-against-one strategy is executed to classify the kernel FisherEMG. By cascading SVM with GDA, the input features will be nonlinearly mapped twice by radial-basis function (RBF). As a result, a linear optimal separating hyperplane can be found with the largest margin of separation between each pair of postures' classes in the implicit dot product feature space. In addition, we develop a digital signal processor (DSP)-based EMG classification system for the control of a multi-degrees-of-freedom prosthetic hand for the practical implementation. Based on the clinical experiments, the results show that the proposed CKLM is superior to other frequently used methods, such as k-nearest neighbor algorithm, multilayer neural network, and SVM. The best EMG recognition rate 93.54% is obtained by CKLM.

A cascade learning system for classification of diabetes disease: Generalized Discriminant Analysis and Least Square Support Vector Machine

The aim of this study is to diagnosis of diabetes disease, which is one of the most important diseases in medical field using Generalized Discriminant Analysis (GDA) and Least Square Support Vector Machine (LS-SVM). Also, we proposed a new cascade learning system based on Generalized Discriminant Analysis and Least Square Support Vector Machine. The proposed system consists of two stages. The first stage, we have used Generalized Discriminant Analysis to discriminant feature variables between healthy and patient (diabetes) data as pre-processing process. The second stage, we have used LS-SVM in order to classification of diabetes dataset. While LS-SVM obtained 78.21% classification accuracy using 10-fold cross validation, the proposed system called GDA-LS-SVM obtained 82.05% classification accuracy using 10-fold cross validation. The robustness of the proposed system is examined using classification accuracy, k-fold cross-validation method and confusion matrix. The obtained classification accuracy is 82.05% and it is very promising compared to the previously reported classification techniques.

An efficient algorithm for generalized discriminant analysis using incomplete Cholesky decomposition

Generalized discriminant analysis (GDA) has provided an extremely powerful approach to extracting nonlinear features via kernel trick. And it has been suggested for a number of applications, such as classification problem. Whereas the GDA could be solved by the utilization of Mercer kernels, a drawback of the standard GDA is that it may suffer from computational problem for large scale data set. Besides, there is still attendant problem of numerical accuracy when computing the eigenvalue problem of large matrices. Also, the GDA would occupy large memory (to store the kernel matrix). To overcome these deficiencies, we use Gram–Schmidt orthonormalization and incomplete Cholesky decomposition to find a basis for the entire training samples, and then formulate GDA as another eigenvalue problem of matrix whose size is much smaller than that of the kernel matrix by using the basis, while still working out the optimal discriminant vectors from all training samples. The theoretical analysis and experimental results on both artificial and real data set have shown the superiority of the proposed method for performing GDA in terms of computational efficiency and even the recognition accuracy, especially when the training samples size is large.

An approach for mechanical fault classification based on generalized discriminant analysis

To deal with pattern classification of complicated mechanical faults, an approach to multi-faults classification based on generalized discriminant analysis is presented. Compared with linear discriminant analysis (LDA), generalized discriminant analysis (GDA), one of nonlinear discriminant analysis methods, is more suitable for classifying the linear non-separable problem. The connection and difference between KPCA (Kernel Principal Component Analysis) and GDA is discussed. KPCA is good at detection of machine abnormality while GDA performs well in multi-faults classification based on the collection of historical faults symptoms. When the proposed method is applied to air compressor condition classification and gear fault classification, an excellent performance in complicated multi-faults classification is presented.

Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance

This paper presents a novel pattern recognition framework by capitalizing on dimensionality increasing techniques. In particular, the framework integrates Gabor image representation, a novel multiclass Kernel Fisher Analysis (KFA) method, and fractional power polynomial models for improving pattern recognition performance. Gabor image representation, which increases dimensionality by incorporating Gabor filters with different scales and orientations, is characterized by spatial frequency, spatial locality, and orientational selectivity for coping with image variabilities such as illumination variations. The KFA method first performs nonlinear mapping from the input space to a high-dimensional feature space, and then implements the multiclass Fisher discriminant analysis in the feature space. The significance of the nonlinear mapping is that it increases the discriminating power of the KFA method, which is linear in the feature space but nonlinear in the input space. The novelty of the KFA method comes from the fact that 1) it extends the two-class kernel Fisher methods by addressing multiclass pattern classification problems and 2) it improves upon the traditional Generalized Discriminant Analysis (GDA) method by deriving a unique solution (compared to the GDA solution, which is not unique). The fractional power polynomial models further improve performance of the proposed pattern recognition framework. Experiments on face recognition using both the FERET database and the FRGC (Face Recognition Grand Challenge) databases show the feasibility of the proposed framework. In particular, experimental results using the FERET database show that the KFA method performs better than the GDA method and the fractional power polynomial models help both the KFA method and the GDA method improve their face recognition performance. Experimental results using the FRGC databases show that the proposed pattern recognition framework improves face recognition performance upon the BEE baseline algorithm and the LDA-based baseline algorithm by large margins.

MutualBoost learning for selecting Gabor features for face recognition

This paper describes an improved boosting algorithm, the MutualBoost algorithm, and its application in developing a fast and robust Gabor feature based face recognition system. The algorithm uses mutual information to eliminate redundancy among Gabor features selected using the AdaBoost algorithm. Selected Gabor features are then subjected to Generalized Discriminant Analysis (GDA) for class separability enhancement before being used for face recognition. Compared with one of the top performers in the 2004 face verification competition, our method demonstrates clear advantages in classification accuracy, memory and computation. The method has been tested on the whole FERET database using the FERET evaluation protocol. Significant improvement in performance is observed. For example, existing Gabor based methods use a huge number of Gabor features, our method needs only hundreds of Gabor features to achieve very high classification accuracy. Due to substantially reduced feature dimension, memory and computation costs are reduced significantly – only 4 s are needed to recognize 200 face images.

Class-Incremental Generalized Discriminant Analysis

Generalized discriminant analysis (GDA) is the nonlinear extension of the classical linear discriminant analysis (LDA) via the kernel trick. Mathematically, GDA aims to solve a generalized eigenequation problem, which is always implemented by the use of singular value decomposition (SVD) in the previously proposed GDA algorithms. A major drawback of SVD, however, is the difficulty of designing an incremental solution for the eigenvalue problem. Moreover, there are still numerical problems of computing the eigenvalue problem of large matrices. In this article, we propose another algorithm for solving GDA as for the case of small sample size problem, which applies QR decomposition rather than SVD. A major contribution of the proposed algorithm is that it can incrementally update the discriminant vectors when new classes are inserted into the training set. The other major contribution of this article is the presentation of the modified kernel Gram-Schmidt (MKGS) orthogonalization algorithm for implementing the QR decomposition in the feature space, which is more numerically stable than the kernel Gram-Schmidt (KGS) algorithm. We conduct experiments on both simulated and real data to demonstrate the better performance of the proposed methods.

Class-Incremental Generalized Discriminant Analysis

Generalized discriminant analysis (GDA) is the nonlinear extension of the classical linear discriminant analysis (LDA) via the kernel trick. Mathematically, GDA aims to solve a generalized eigenequation problem, which is always implemented by the use of singular value decomposition (SVD) in the previously proposed GDA algorithms. A major drawback of SVD, however, is the difficulty of designing an incremental solution for the eigenvalue problem. Moreover, there are still numerical problems of computing the eigenvalue problem of large matrices. In this article, we propose another algorithm for solving GDA as for the case of small sample size problem, which applies QR decomposition rather than SVD. A major contribution of the proposed algorithm is that it can incrementally update the discriminant vectors when new classes are inserted into the training set. The other major contribution of this article is the presentation of the modified kernel Gram-Schmidt (MKGS) orthogonalization algorithm for implementing the QR decomposition in the feature space, which is more numerically stable than the kernel Gram-Schmidt (KGS) algorithm. We conduct experiments on both simulated and real data to demonstrate the better performance of the proposed methods.

Kernel Pooled Local Subspaces for Classification

We investigate the use of subspace analysis methods for learning low-dimensional representations for classification. We propose a kernel-pooled local discriminant subspace method and compare it against competing techniques: kernel principal component analysis (KPCA) and generalized discriminant analysis (GDA) in classification problems. We evaluate the classification performance of the nearest-neighbor rule with each subspace representation. The experimental results using several data sets demonstrate the effectiveness and performance superiority of the kernel-pooled subspace method over competing methods such as KPCA and GDA in some classification problems.

Identification of Single Bacterial Cells in Aqueous Solution Using Confocal Laser Tweezers Raman Spectroscopy

We report on a rapid method for reagentless identification and discrimination of single bacterial cells in aqueous solutions using a combination of laser tweezers and confocal Raman spectroscopy (LTRS). The optical trapping enables capturing of individual bacteria in aqueous solution in the focus of the laser beam and levitating the captured cell well off the cover plate, thus maximizing the excitation and collection of Raman scattering from the cell and minimizing the unwanted background from the cover plate and environment. Raman spectral patterns excited by a near-infrared laser beam provide intrinsic molecular information for reagentless analysis of the optically isolated bacterium. In our experiments, six species of bacteria were used to demonstrate the capability of the confocal LTRS in the identification and discrimination between the diverse bacterial species at various growth conditions. We show that synchronized bacterial cells can be well-discriminated among the six species using principal component analyses (PCA). Unsynchronized bacterial cells that are cultured at stationary phases can also be well-discriminated by the PCA, as well as by a hierarchical cluster analysis (HCA) of their Raman spectra. We also show that unsynchronized bacteria selected from random growth phases can be classified with the help of a generalized discriminant analysis (GDA). These findings demonstrate that the LTRS may find valuable applications in rapid sensing of microbial cells in diverse aqueous media.

Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image

We present a novel method of nonlinear discriminant analysis involving a set of locally linear transformations called "Locally Linear Discriminant Analysis (LLDA)." The underlying idea is that global nonlinear data structures are locally linear and local structures can be linearly aligned. Input vectors are projected into each local feature space by linear transformations found to yield locally linearly transformed classes that maximize the between-class covariance while minimizing the within-class covariance. In face recognition, linear discriminant analysis (LDA) has been widely adopted owing to its efficiency, but it does not capture nonlinear manifolds of faces which exhibit pose variations. Conventional nonlinear classification methods based on kernels such as generalized discriminant analysis (GDA) and support vector machine (SVM) have been developed to overcome the shortcomings of the linear method, but they have the drawback of high computational cost of classification and overfitting. Our method is for multiclass nonlinear discrimination and it is computationally highly efficient as compared to GDA. The method does not suffer from overfitting by virtue of the linear base structure of the solution. A novel gradient-based learning algorithm is proposed for finding the optimal set of local linear bases. The optimization does not exhibit a local-maxima problem. The transformation functions facilitate robust face recognition in a low-dimensional subspace, under pose variations, using a single model image. The classification results are given for both synthetic and real face data.

Face recognition using kernel direct discriminant analysis algorithms

Techniques that can introduce low-dimensional feature representation with enhanced discriminatory power is of paramount importance in face recognition (FR) systems. It is well known that the distribution of face images, under a perceivable variation in viewpoint, illumination or facial expression, is highly nonlinear and complex. It is, therefore, not surprising that linear techniques, such as those based on principle component analysis (PCA) or linear discriminant analysis (LDA), cannot provide reliable and robust solutions to those FR problems with complex face variations. In this paper, we propose a kernel machine-based discriminant analysis method, which deals with the nonlinearity of the face patterns' distribution. The proposed method also effectively solves the so-called "small sample size" (SSS) problem, which exists in most FR tasks. The new algorithm has been tested, in terms of classification error rate performance, on the multiview UMIST face database. Results indicate that the proposed methodology is able to achieve excellent performance with only a very small set of features being used, and its error rate is approximately 34% and 48% of those of two other commonly used kernel FR approaches, the kernel-PCA (KPCA) and the generalized discriminant analysis (GDA), respectively.

Generalized discriminant analysis using a kernel approach.

We present a new method that we call generalized discriminant analysis (GDA) to deal with nonlinear discriminant analysis using kernel function operator. The underlying theory is close to the support vector machines (SVM) insofar as the GDA method provides a mapping of the input vectors into high-dimensional feature space. In the transformed space, linear properties make it easy to extend and generalize the classical linear discriminant analysis (LDA) to nonlinear discriminant analysis. The formulation is expressed as an eigenvalue problem resolution. Using a different kernel, one can cover a wide class of nonlinearities. For both simulated data and alternate kernels, we give classification results, as well as the shape of the decision function. The results are confirmed using real data to perform seed classification.