Extension Of Linear Discriminant Analysis Research Articles

Double Nearest Proportion Feature Extraction for Hyperspectral-Image Classification

For the classification among different land-cover types in a hyperspectral image, particularly in the small-sample-size situation, a feature-extraction method is an approach for reducing the dimensionality and increasing the classification accuracy. Fisher's linear discriminant analysis (LDA) is one of the most popular feature-extraction methods. However, it cannot be applied directly to the classification of hyperspectral image because of several natures of Fisher's criterion. Nonparametric discriminant analysis (NDA) and nonparametric weighted feature extraction, on the other hand, are two extensions of LDA with a creative idea about emphasizing the boundary structure of class distributions. However, the overlap situation was not considered in these methods and thus decreased the robustness of these methods. In this paper, a new feature-extraction method is introduced based on a structure named double nearest proportion. This structure enables the proposed method to reduce the effect of overlap, allows a new regularization technique to be embedded, and includes LDA and NDA as special cases. These properties enable the proposed method to be more robust and thus, generally, have better performance.

Generalized nonlinear discriminant analysis and its small sample size problems

This paper develops a generalized nonlinear discriminant analysis (GNDA) method and deals with its small sample size (SSS) problems. GNDA is a nonlinear extension of linear discriminant analysis (LDA), while kernel Fisher discriminant analysis (KFDA) can be regarded as a special case of GNDA. In LDA, an under sample problem or a small sample size problem occurs when the sample size is less than the sample dimensionality, which will result in the singularity of the within-class scatter matrix. Due to a high-dimensional nonlinear mapping in GNDA, small sample size problems arise rather frequently. To tackle this issue, this research presents five different schemes for GNDA to solve the SSS problems. Experimental results on real-world data sets show that these schemes for GNDA are very effective in tackling small sample size problems.

Exemplar based Laplacian Discriminant Projection

A new algorithm, exemplar based Laplacian Discriminant Projection (ELDP), is proposed in this paper for supervised dimensionality reduction. ELDP aims at learning a linear transformation which is an extension of Linear Discriminant Analysis combining with clustering technique. Specifically, we define three scatter matrices using similarities based on representative exemplars which are found by Affinity Propagation Clustering. After the transformation, the considered pair-wise samples within the same exemplar subset and the same class are as close as possible, while those exemplars between-classes are as far as possible. The structural information of classes is contained in the exemplar based Laplacian matrices. Thus the discriminant projection subspace can be derived by controlling the structural evolution of Laplacian matrices. The performance on several data sets demonstrates the competence of the proposed algorithm.

A Multifactor Extension of Linear Discriminant Analysis for Face Recognition under Varying Pose and Illumination

Linear Discriminant Analysis (LDA) and Multilinear Principal Component Analysis (MPCA) are leading subspace methods for achieving dimension reduction based on supervised learning. Both LDA and MPCA use class labels of data samples to calculate subspaces onto which these samples are projected. Furthermore, both methods have been successfully applied to face recognition. Although LDA and MPCA share common goals and methodologies, in previous research they have been applied separately and independently. In this paper, we propose an extension of LDA to multiple factor frameworks. Our proposed method, Multifactor Discriminant Analysis, aims to obtain multilinear projections that maximize the between-class scatter while minimizing the withinclass scatter, which is the same core fundamental objective of LDA. Moreover, Multifactor Discriminant Analysis (MDA), like MPCA, uses multifactor analysis and calculates subject parameters that represent the characteristics of subjects and are invariant to other changes, such as viewpoints or lighting conditions. In this way, our proposedMDA combines the best virtues of both LDA and MPCA for face recognition.

Feature extraction based on subspace methods for regression problems

In this paper, we propose a couple of new feature extraction methods for regression problems. The first one is closely related to the conventional principle component analysis (PCA) but unlike PCA, it incorporates target information in the optimization process and try to find a set of linear transforms that maximizes the distances between points with large differences in target values. On the other hand, the second one is a regressional version of linear discriminant analysis (LDA) which is very popular for classification problems. We have applied the proposed methods to several regression problems and compared the performance with the conventional feature extraction methods. The experimental results show that the proposed methods, especially the extension of LDA, perform well in many regression problems.

Generalized Discriminant Analysis: A Matrix Exponential Approach

Linear discriminant analysis (LDA) is well known as a powerful tool for discriminant analysis. In the case of a small training data set, however, it cannot directly be applied to high-dimensional data. This case is the so-called small-sample-size or undersampled problem. In this paper, we propose an exponential discriminant analysis (EDA) technique to overcome the undersampled problem. The advantages of EDA are that, compared with principal component analysis (PCA) + LDA, the EDA method can extract the most discriminant information that was contained in the null space of a within-class scatter matrix, and compared with another LDA extension, i.e., null-space LDA (NLDA), the discriminant information that was contained in the non-null space of the within-class scatter matrix is not discarded. Furthermore, EDA is equivalent to transforming original data into a new space by distance diffusion mapping, and then, LDA is applied in such a new space. As a result of diffusion mapping, the margin between different classes is enlarged, which is helpful in improving classification accuracy. Comparisons of experimental results on different data sets are given with respect to existing LDA extensions, including PCA + LDA, LDA via generalized singular value decomposition, regularized LDA, NLDA, and LDA via QR decomposition, which demonstrate the effectiveness of the proposed EDA method.

Tensor linear Laplacian discrimination (TLLD) for feature extraction

Discriminant feature extraction plays a central role in pattern recognition and classification. In this paper, we propose the tensor linear Laplacian discrimination (TLLD) algorithm for extracting discriminant features from tensor data. TLLD is an extension of linear discriminant analysis (LDA) and linear Laplacian discrimination (LLD) in directions of both nonlinear subspace learning and tensor representation. Based on the contextual distance, the weights for the within-class scatters and the between-class scatter can be determined to capture the principal structure of data clusters. This makes TLLD free from the metric of the sample space, which may not be known. Moreover, unlike LLD, the parameter tuning of TLLD is very easy. Experimental results on face recognition, texture classification and handwritten digit recognition show that TLLD is effective in extracting discriminative features.

Novel face recognition method based on KPCA plus KDA

Kernel Discriminant Anlaysis(KDA) and Kernel Principal Component Analysis(KPCA) are the nonlinear extensions of Linear Discriminant Analysis(LDA) and Principal Component Analysis(PCA) respectively.In this paper,we presented a feature extraction algorithm by combing KDA and KPCA to extract reliable and robust features for recognition.Furthermore,a generalized nearest feature line(GNFL) method was also presented for constructing powerful classifier.The performance of the proposed method was demonstrated through real data.

Identification of Influential Cases in Kernel Fisher Discriminant Analysis

We study the influence of a single data case on the results of a statistical analysis. This problem has been addressed in several articles for linear discriminant analysis (LDA). Kernel Fisher discriminant analysis (KFDA) is a kernel based extension of LDA. In this article, we study the effect of atypical data points on KFDA and develop criteria for identification of cases having a detrimental effect on the classification performance of the KFDA classifier. We find that the criteria are successful in identifying cases whose omission from the training data prior to obtaining the KFDA classifier results in reduced error rates.

A two-stage linear discriminant analysis via QR-decomposition

Linear Discriminant Analysis (LDA) is a well-known method for feature extraction and dimension reduction. It has been used widely in many applications involving high-dimensional data, such as image and text classification. An intrinsic limitation of classical LDA is the so-called singularity problems; that is, it fails when all scatter matrices are singular. Many LDA extensions were proposed in the past to overcome the singularity problems. Among these extensions, PCA+LDA, a two-stage method, received relatively more attention. In PCA+LDA, the LDA stage is preceded by an intermediate dimension reduction stage using Principal Component Analysis (PCA). Most previous LDA extensions are computationally expensive, and not scalable, due to the use of Singular Value Decomposition or Generalized Singular Value Decomposition. In this paper, we propose a two-stage LDA method, namely LDA/QR, which aims to overcome the singularity problems of classical LDA, while achieving efficiency and scalability simultaneously. The key difference between LDA/QR and PCA+LDA lies in the first stage, where LDA/QR applies QR decomposition to a small matrix involving the class centroids, while PCA+LDA applies PCA to the total scatter matrix involving all training data points. We further justify the proposed algorithm by showing the relationship among LDA/QR and previous LDA methods. Extensive experiments on face images and text documents are presented to show the effectiveness of the proposed algorithm.

Extensions of LDA by PCA mixture model and class-wise features

Linear discriminant analysis (LDA) is a data discrimination technique that seeks transformation to maximize the ratio of the between-class scatter and the within-class scatter. While it has been successfully applied to several applications, it has two limitations, both concerning the underfitting problem. First, it fails to discriminate data with complex distributions since all data in each class are assumed to be distributed in the Gaussian manner. Second, it can lose class-wise information, since it produces only one transformation over the entire range of classes. We propose three extensions of LDA to overcome the above problems. The first extension overcomes the first problem by modelling the within-class scatter using a PCA mixture model that can represent more complex distribution. The second extension overcomes the second problem by taking different transformation for each class in order to provide class-wise features. The third extension combines these two modifications by representing each class in terms of the PCA mixture model and taking different transformation for each mixture component. It is shown that all our proposed extensions of LDA outperform LDA concerning classification errors for synthetic data classification, hand-written digit recognition, and alphabet recognition.

Extensions of linear discriminant analysis for statistical classification of remotely sensed satellite imagery

Linear discriminant analysis is a commonly used statistical tool for the classification of surface features using satellite surface reflectance data. Extensions of this basic tool promise substantial improvements. In particular, we examine the added effectiveness of the integration of spatial autocorrelation into the discriminant model, the resolution of nonhomogeneous pixels, and data based prior probability estimates of class membership.