Discriminant Projection Method Research Articles

A Promising Nonlinear Dimensionality Reduction Method: Kernel-Based Within Class Collaborative Preserving Discriminant Projection

High-dimensional small sample size problems exist in the real world, which significantly increases the difficulty of data processing. In this paper, we propose a kernel-based within class collaborative preserving discriminant projection method to reduce data dimensionality. In order to deal with nonlinear problems and improve the discrimination of the projection subspace, the proposed method preserves the collaborative reconstruction relationship of the same class samples in the kernel space, and pursuits maximizing the between class scatter. A two-step eigenvalue decomposition method is used to stably obtain the optimal discriminant projection matrix. Moreover, simulation experiments show that, even in low dimensions and small sample size, the proposed method can achieve high recognition accuracy.

Multi-Kernel Coupled Projections for Domain Adaptive Dictionary Learning

Dictionary learning has produced state-of-the-art results in various classification tasks. However, if the training data have a different distribution than the testing data, the learned sparse representation might not be optimal. Recently, several domain-adaptive dictionary learning (DADL) methods and kernels have been proposed and have achieved impressive performance. However, the performance of these single kernel-based methods heavily depends heavily on the choice of the kernel, and the question of how to combine multiple kernel learning (MKL) with the DADL framework has not been well studied. Motivated by these concerns, in this paper, we propose a multi-kernel domain-adaptive sparse representation-based classification (MK-DASRC) and then use it as a criterion to design a multi-kernel sparse representation-based domain-adaptive discriminative projection method, in which the discriminative features of the data in the two domains are simultaneously learned with the dictionary. The purpose of this method is to maximize the between-class sparse reconstruction residuals of data from both domains, and minimize the within-class sparse reconstruction residuals of data in the low-dimensional subspace. Thus, the resulting representations can satisfactorily fit MK-DASRC and simultaneously display discriminability. Extensive experimental results on a series of benchmark databases show that our method performs better than the state-of-the-art methods.

Feature extraction based on Low-rank affinity matrix for biological recognition

The low-rank representation (LRR) was presented recently and demonstrated its effectiveness for robust subspace segmentation. This paper presents a discriminative projection method based on Low-rank affinity matrix (LRA-DP) for robust feature extraction. The affinity matrix is designed to better preserve the underlying low-rank structure of data representation revealed by LRR. The experiments on the Yale, Extended Yale B, AR face image databases and the PolyU palmprint database showed LRA-DP is always better than or comparable to other state-of-the-art methods, which means underlying low-rank structure of data representation preserved by LRA-DP is helpful for classification problem.

Kernel collaborative representation based dictionary learning and discriminative projection

Sparse representation based classification (SRC) has been developed and shown great potential due to its effectiveness in various classification tasks. But how to determine appropriate features that can best work with SRC remains an open question. Based on SRC and dimensionality reduction (DR) techniques, a simultaneous discriminative projection and dictionary learning method (DSRC) is proposed. However, as a linear algorithm, DSRC cannot handle the data with highly nonlinear distribution. Recently research has shown that the collaborative representation mechanism is more important to the success of SRC. Motivated by these concerns, in this paper, we propose a novel kernel collaborative representation based classifier (KCRC), and then we use it as a criterion to design a kernel collaborative representation based dictionary learning and discriminant projection method (KDL-DP). The proposed method aims at learning a projection matrix and a dictionary such that in the low dimension subspace the between-class reconstruction residual of a given data set is maximized and the within-class reconstruction residual is minimized. Extensive experimental results validate the superiority of the proposed approach when compared with the state-of-the-art methods.

Multiple Kernel Sparse Representation-Based Orthogonal Discriminative Projection and Its Cost-Sensitive Extension.

Sparse representation-based classification (SRC) has been developed and shown great potential for real-world application. Based on SRC, Yang et al. devised an SRC steered discriminative projection (SRC-DP) method. However, as a linear algorithm, SRC-DP cannot handle the data with highly nonlinear distribution. Kernel sparse representation-based classifier (KSRC) is a non-linear extension of SRC and can remedy the drawback of SRC. KSRC requires the use of a predetermined kernel function and selection of the kernel function and its parameters is difficult. Recently, multiple kernel learning for SRC (MKL-SRC) has been proposed to learn a kernel from a set of base kernels. However, MKL-SRC only considers the within-class reconstruction residual while ignoring the between-class relationship, when learning the kernel weights. In this paper, we propose a novel multiple kernel sparse representation-based classifier, and then we use it as a criterion to design a multiple kernel sparse representation-based orthogonal discriminative projection method. The proposed algorithm aims at learning a projection matrix and a corresponding kernel from the given base kernels such that in the low dimension subspace the between-class reconstruction residual is maximized and the within-class reconstruction residual is minimized. Furthermore, to achieve a minimum overall loss by performing recognition in the learned low-dimensional subspace, we introduce cost information into the dimensionality reduction method. The solutions for the proposed method can be efficiently found based on trace ratio optimization method. Extensive experimental results demonstrate the superiority of the proposed algorithm when compared with the state-of-the-art methods.

Max-Margin Discriminant Projection via Data Augmentation

In this paper, we introduce a new max-margin discriminant projection method, which takes advantage of the latent variable representation for support vector machine (SVM) as the classification criterion. Specifically, the proposed model jointly learns the discriminative subspace and classifier in a Bayesian framework by conditioning on augmented variables. Moreover, an extended nonlinear model is developed based on the kernel trick, where the similar model can be used in this setting with few modifications. To explore the sparsity in the kernel expansion, we use the spike-and-slab prior to seek basis vectors (BVs) from the corresponding candidates. Unlike existing methods, which employ BVs to approximate the original feature space, in our method BVs are sought to associate the final classification task. Thanks to the conditionally conjugate property, the parameters in our models can be inferred via the simple and efficient Gibbs sampler. Finally, we test our methods on synthesized and real-world data, including large-scale data sets to demonstrate their efficiency and effectiveness.

Adaptive Neighborhood-Preserving Discriminant Projection Method for HRRP-Based Radar Target Recognition

A new manifold learning algorithm named adaptive neighborhood preserving discriminant projection method is proposed for the feature extraction of high-range resolution profile (HRRP)-based radar target recognition. By utilizing the objective functions of both neighborhood-preserving projection (NPP) and adaptive maximum margin criterion (AMMC), the proposed method can not only preserve the neighborhood structure of original data in the dimensionality reduced space, but also exhibit good classification performance. The proposed method is applied to the feature extraction of HRRP-based radar target recognition. Numerical experiments show that the proposed method can effectively reduce the dimensionality of HRRP and give satisfactory recognition rate.

Low-rank representation based discriminative projection for robust feature extraction

The low-rank representation (LRR) was presented recently and showed effective and robust for subspace segmentation. This paper presents a LRR-based discriminative projection method (LRR-DP) for robust feature extraction, by virtue of the underlying low-rank structure of data represesntation revealed by LRR. LRR-DP seeks a linear transformation such that in the transformed space, the between-class scatter (i.e. the distance between a sample and its between-class representation prototype) is as large as possible and simultaneously, the combination of the within-class scatter (i.e. the distance between a sample and its within-class representation prototype) and the scatter of noises is as small as possible. Our experiments were done using the Yale, Extended Yale B, AR face image databases and the PolyU palmprint database, and the results show that LRR-DP is always better than or comparable to other state-of-the-art methods.

A New Local Discriminant Projection Method

To solve the problem of multimodal data classification,the idea of localization is introduced into discriminant analysis,known as local discriminant analysis.In this paper,we first illustrated,by some synthetic data as examples,the drawbacks of LFDA and MFA,two recently proposed and successfully used local linear discriminant analysis methods.We then proposed a new local discriminant projection method to overcome the drawback of LFDA and MFA that neighbor relationships between samples of different classes are not fully taken into consideration.The underlying idea of the new method,different from LFDA and MFA,is that the desired projection should make neighbors of the same class close and neighbors of different classes apart.Based on this idea,we proposed two optimal functions,one is represented by the sum of squared distance between samples,the other is represented by within class scatter and between class scatter.Analyses and comparisons on the two optimal functions are also included in this paper.Experiment results show that the new method overcomes the shortcomings of LFDA and MFA,and achieves good performance on the synthetic data,USPS,UCI standard handwriting digital data sets and IDA standard data sets.

A Local Discriminant Projection Method Based on Objective Space

A Local Discriminant Projection Method Based on Objective Space