Sparsity Preserving Projections Research Articles

Discriminative sparsity graph embedding based on histogram of rotated princial orientation gradients

The unconstrained face images collected in the real environments include many complicated and changeable interference factors, and sparsity preserving projections (SPP) cannot well obtain the low-dimensional intrinsic structure embedded in the high-dimensional samples, which is important for subsequent sparse representation classifier (SRC). To deal with this problem, in this paper we propose a new method named discriminative sparsity graph embedding based on histogram of rotated principal orientation gradients (DSGE-HRPOG). Firstly, it extracts multi-scale and multi-directional gradient features of unconstrained face images by HRPOG feature descriptor and incorporates them into a discriminative feature dictionary of sparse representation classifier. Secondly, it seeks an optimal subspace of HRPOG feature dictionary in which the atoms in intra-classes are as compact as possible, while the atoms in inter-classes are as separable as possible by adopting the proposed DSGE dimensionality reduction method. Finally, an optimal algorithm is presented in which the low-dimensional projection and the sparse graph construction are iteratively updated, and the accuracy of unconstrained face recognition is further improved. Extensive experimental results on AR, Extended Yale B, LFW and PubFig databases demonstrate the effectiveness of our proposed method.

Linear regression based projections for dimensionality reduction

In graph embedding based dimensionality reduction methods, the number of K-nearest neighbors is usually needed to be manually chosen in high dimensional space. Graph construction by different number of K-nearest neighbor changes dramatically and seriously affects the performance of graph embedding based dimensionality reduction. How to automatically construct a graph is very important. In this paper, first, a discriminative L2-graph is investigated. It computes the edge weights using the class-specific samples and weighted ridge regression, avoiding manually choosing the K-nearest neighbors in traditional graph construction. Second, a discriminative L2-graph based dimensionality reduction method is proposed, named Linear Regression based Projections (LRP). LRP minimizes the ratio between the local compactness information and the total separability information to seek the optimal projection matrix. LRP is much faster than its counterparts, Sparsity Preserving Projections (SPP) and Collaborative Representation based Projections (CRP), since LRP is supervised and computes edge weights using class-specific samples while SPP and CRP are unsupervised and compute edge weights using all samples. The experimental results on benchmark face image databases show that the proposed LRP outperforms many existing representative linear dimensionality reduction methods.

2D-LPCCA and 2D-SPCCA: Two new canonical correlation methods for feature extraction, fusion and recognition

Abstract Two-dimensional canonical correlation analysis (2D-CCA) is an effective and efficient method for two-view feature extraction and fusion. Since it is a global linear method, it fails to find the nonlinear correlation between different features. In contrast, in this paper we propose a novel two-view method named as two-dimensional locality preserving canonical correlation analysis (2D-LPCCA), which uses the neighborhood information to discover the intrinsic structure of data. In other words, it uses many local linear problems to approximate the global nonlinear case. In addition, inspired by sparsity preserving projections (SPP), the two-dimensional sparsity preserving canonical correlation analysis (2D-SPCCA) framework is also developed, which consists of three models. Experimental results on real world databases demonstrate the viability of the formulation, they also show that the classification results of our methods are higher than the other’s.

A Weighted Sparse Neighbourhood-Preserving Projections for Face Recognition

ABSTRACTDimensionality reduction algorithms are widely applied to high-dimensional data pre-processing, especially for face images. In this paper, we propose an unsupervised sparse subspace learning approach called weighted sparse neighbourhood-preserving projections (WSNPP) for face recognition. Unlike many existing approaches such as sparsity-preserving projections (SPP), where the constructive weights are computed by the classical sparse representation (SR), WSNPP utilizes a weighted SR model to represent samples. The obtained projections can contain more local discriminant information than classical sparse subspace learning methods. Moreover, WSNPP puts a constraint on the number of nonzero reconstruction coefficients and hence is more robust to global noises and time saving. Experiments on AR, Yale-B and ORL image datasets demonstrate its effectiveness.

Regularized coplanar discriminant analysis for dimensionality reduction

The dimensionality reduction methods based on linear embedding, such as neighborhood preserving embedding (NPE), sparsity preserving projections (SPP) and collaborative representation based projections (CRP), try to preserve a certain kind of linear representation for each sample after projection. However, in the transformed low-dimensional space, the linear relationship between the samples may be changed, which cannot make the linear representation-based classifiers, such as sparse representation-based classifier (SRC), to achieve higher recognition accuracy. In this paper, we propose a new linear dimensionality reduction algorithm, called Regularized Coplanar Discriminant Analysis (RCDA) to address this problem. It simultaneously seeks a linear projection matrix and some linear representation coefficients that make the samples from the same class coplanar and the samples from different classes not coplanar. The proposed regularization term balances the bias from the optimal linear representation and that from the class mean to avoid overfitting the training data, and overcomes the matrix singularity in solving the linear representation coefficients. An alternative optimization approach is proposed to solve the RCDA model. Experiments are done on several benchmark face databases and hyperspectral image databases, and results show that RCDA can obtain better performance than other dimensionality reduction methods.

Flexible constrained sparsity preserving embedding

In this paper, two semi-supervised embedding methods are proposed, namely Constrained Sparsity Preserving Embedding (CSPE) and Flexible Constrained Sparsity Preserving Embedding (FCSPE). CSPE is a semi-supervised embedding method which can be considered as a semi-supervised extension of Sparsity Preserving Projections (SPP) integrated with the idea of in-class constraints. Both the labeled and unlabeled data can be utilized within the CSPE framework. However, CSPE does not have an out-of-sample extension since the projection of the unseen samples cannot be obtained directly. In order to have an inductive semi-supervised learning, i.e. being able to handle unseen samples, we propose FCSPE which can simultaneously provide a non-linear embedding and an approximate linear projection in one regression function. FCSPE simultaneously achieves the following: (i) the local sparse structures is preserved, (ii) the data samples with a same label are mapped onto one point in the projection space, and (iii) a linear projection that is the closest one to the non-linear embedding is estimated. Experimental results on eight public image data sets demonstrate the effectiveness of the proposed methods as well as their superiority to many competitive semi-supervised embedding techniques.

Discriminant Sparsity Preserving Analysis for Face Recognition

Sparse subspace learning has drawn more and more attentions recently, however, most of them are unsupervised and unsuitable for classification tasks. In this paper, a new discriminant sparsity preserving analysis (DSPA) method by integrating sparse reconstructive weighting into Fisher criterion is proposed for face recognition. We first get sparsity preserving space spanned by the eigenvectors of sparsity preserving projections (SPP). Then, the optimal projection can be obtained by solving an eigenvalue and eigenvector problem of the between-class scatter matrix in sparsity preserving space. The method not only preserves the sparse reconstructive relationship of the data, but also encodes the discriminant information. Extensive experiments on four face image datasets (Yale, ORL, AR and CMU PIE) demonstrate the effectiveness of the proposed DSPA method.

Dimensionality Reduction of Hyperspectral Images With Sparse Discriminant Embedding

Sparse manifold learning has drawn more and more attentions recently, and sparsity preserving projections (SPP) has been proposed, which inherits the advantages of sparse reconstruction. However, SPP only focuses on the sparse structure, ignoring the discriminant information of labeled samples. In this paper, we proposed a new supervised dimensionality reduction method, which is called sparse discriminant embedding (SDE), for hyperspectral image (HSI) classification. SDE utilizes the merits of both intermanifold structure and sparsity property. It not only preserves the sparse reconstructive relations through $\ell_{1} $ -graph but also enhances the intermanifold separability of data, and the discriminating power of SDE is further improved than SPP. Experiments on two real HSIs collected by the Airborne Visible/Infrared Imaging Spectrometer and Reflective Optics System Imaging Spectrometer sensors are performed to demonstrate the effectiveness of the proposed SDE method.

Sparse dimensionality reduction of hyperspectral image based on semi-supervised local Fisher discriminant analysis

This paper presents a novel sparse dimensionality reduction method of hyperspectral image based on semi-supervised local Fisher discriminant analysis (SELF). The proposed method is designed to be especially effective for dealing with the out-of-sample extrapolation to realize advantageous complementarities between SELF and sparsity preserving projections (SPP). Compared to SELF and SPP, the method proposed herein offers highly discriminative ability and produces an explicit nonlinear feature mapping for the out-of-sample extrapolation. This is due to the fact that the proposed method can get an explicit feature mapping for dimensionality reduction and improve the classification performance of classifiers by performing dimensionality reduction. Experimental analysis on the sparsity and efficacy of low dimensional outputs shows that, sparse dimensionality reduction based on SELF can yield good classification results and interpretability in the field of hyperspectral remote sensing.

Classification of hyperspectral remote-sensing images based on sparse manifold learning

Sparsity preserving projections (SPP) has drawn more and more attention recently. However, the SPP only focuses on the sparse structure but ignores the discriminant information of labeled samples. We proposed a new sparse manifold learning method, called sparse discriminant embedding (SDE), for hyperspectral image (HSI) classification. The SDE utilizes the merits of both sparsity property and manifold structure. It not only preserves the sparse reconstructive relations but also explicitly boosts the discriminant manifold structure of the data, and the discriminating power of the SDE is further improved than the SPP. Experiments on the Flightline C1, Washington DC Mall, and Botswana HSI datasets are performed to demonstrate the effectiveness of the proposed SDE method.

Pairwise constraint-guided sparsity preserving projections

Concerning the deficiency of supervision information in the process of sparse reconstruction in Sparsity Preserving Projection(SPP),Pairwise Constraint-guided Sparsity Preserving Projection(PCSPP) was proposed,which introduced supervision information of must-link constraints and cannot-link constraints to guide sparse reconstruction in the process of sparsity reconstruction of training samples,making SPP fuse constraint supervise information efficiently.The experimental results in UMIST,YALE and AR face datasets show,in contrast to unsupervised sparsity preserving projections,our algorithm achieves approximately 5%~15% increase in recognition accuracy based on the nearest neighbor classifier and promotes efficiently the performance of dimensionality reduction classification.

Sparsity Preserving Canonical Correlation Analysis with Application in Feature Fusion

摘要: 稀疏保持投影(Sparsity preserving projections, SPP)由于保持了数据间的稀疏重构性, 因而获取的投影向量满足旋转、尺度和平移的不变性, 并能够在无标签的情况下提取样本的自然鉴别信息, 在人脸识别领域取得了较为成功的应用. 本文在典型相关分析(Canonical correlation analysis, CCA)的基础上引入稀疏保持项, 提出一种稀疏保持典型相关分析(Sparsity preserving canonical correlation analysis, SPCCA). 该方法不仅实现了两组特征集鉴别信息的有效融合, 同时对提取特征间的稀疏重构性加以约束, 增强了特征的表示和鉴别能力. 在多特征手写体字符集与人脸数据集上的实验结果表明, SPCCA比CCA具有更优的识别性能. 关键词: 典型相关分析 (CCA) / 稀疏保持投影(SPP) / 稀疏保持典型相关分析(SPCCA) / 特征融合

Local sparse representation projections for face recognition

How to define the sparse affinity weight matrices is still an open problem in existing manifold learning algorithm. In this paper, we propose a novel supervised learning method called local sparse representation projections (LSRP) for linear dimensionality reduction. Differing from sparsity preserving projections (SPP) and the recent manifold learning methods such as locality preserving projections (LPP), LSRP introduces the local sparse representation information into the objective function. Although there are no labels used in the local sparse representation, it still can provide better measure coefficients and significant discriminant abilities. By combining the local interclass neighborhood relationships and sparse representation information, LSRP aims to preserve the local sparse reconstructive relationships of the data and simultaneously maximize the interclass separability. Comprehensive comparison and extensive experiments show that LSRP achieves higher recognition rates than principle component analysis, linear discriminant analysis and the state-of-the-art techniques such as LPP, SPP and maximum variance projections.

Fast Fisher Sparsity Preserving Projections

Recently, there has been a lot of interest in the underlying sparse representation structure in high-dimensional data such as face images. In this paper, we propose two novel efficient dimensionality reduction methods named Fast Sparsity Preserving Projections (FSPP) and Fast Fisher Sparsity Preserving Projections (FFSPP), respectively, which aim to preserve the sparse representation structure in high-dimensional data. Unlike the existing Sparsity Preserving Projections (SPP), where the sparse representation structure is learned through resolving n (the number of samples) time-consuming \( \ell^{ 1} \) norm optimization problems, FSPP constructs a dictionary through classwise PCA decompositions and learns the sparse representation structure under the constructed dictionary through matrix–vector multiplications, which is much more computationally tractable. FFSPP takes into consideration both the sparse representation structure and the discriminating efficiency by adding the Fisher constraint to the FSPP formulation to improve FSPP’s discriminating ability. Both of the proposed methods can boil down to a generalized eigenvalue problem. Experimental results on three publicly available face data sets (Yale, Extended Yale B and ORL), and a standard document collection (Reuters-21578) validate the feasibility and effectiveness of the proposed methods.

Discover latent discriminant information for dimensionality reduction: Non-negative Sparseness Preserving Embedding

How to define sparse affinity weight matrices is still an open problem in existing manifold learning algorithms. In this paper, we propose a novel unsupervised learning method called Non-negative Sparseness Preserving Embedding (NSPE) for linear dimensionality reduction. Differing from the manifold learning-based subspace learning methods such as Locality Preserving Projections (LPP), Neighbor Preserving Embedding (NPE) and the recently proposed sparse representation based Sparsity Preserving Projections (SPP); NSPE preserves the non-negative sparse reconstruction relationships in low-dimensional subspace. Another novelty of NSPE is the sparseness constraint, which is directly added to control the non-negative sparse representation coefficients. This gives a more ground truth model to imitate the actions of the active neuron cells of V1 of the primate visual cortex on information processing. Although labels are not used in the training steps, the non-negative sparse representation can still discover the latent discriminant information and thus provides better measure coefficients and significant discriminant abilities for feature extraction. Moreover, NSPE is more efficient than the recently proposed sparse representation based SPP algorithm. Comprehensive comparison and extensive experiments show that NSPE has the competitive performance against the unsupervised learning algorithms such as classical PCA and the state-of-the-art techniques: LPP, NPE and SPP.

Discriminative learning by sparse representation for classification

Recently, sparsity preserving projections (SPP) algorithm has been proposed, which combines l 1-graph preserving the sparse reconstructive relationship of the data with the classical dimensionality reduction algorithm. However, when applied to classification problem, SPP only focuses on the sparse structure but ignores the label information of samples. To enhance the classification performance, a new algorithm termed discriminative learning by sparse representation projections or DLSP for short is proposed in this paper. DLSP algorithm incorporates the merits of both local interclass geometrical structure and sparsity property. That makes it possess the advantages of the sparse reconstruction, and more importantly, it has better capacity of discrimination, especially when the size of the training set is small. Extensive experimental results on serval publicly available data sets show the feasibility and effectiveness of the proposed algorithm.

Sparsity preserving projections with applications to face recognition

Dimensionality reduction methods (DRs) have commonly been used as a principled way to understand the high-dimensional data such as face images. In this paper, we propose a new unsupervised DR method called sparsity preserving projections (SPP). Unlike many existing techniques such as local preserving projection (LPP) and neighborhood preserving embedding (NPE), where local neighborhood information is preserved during the DR procedure, SPP aims to preserve the sparse reconstructive relationship of the data, which is achieved by minimizing a L1 regularization-related objective function. The obtained projections are invariant to rotations, rescalings and translations of the data, and more importantly, they contain natural discriminating information even if no class labels are provided. Moreover, SPP chooses its neighborhood automatically and hence can be more conveniently used in practice compared to LPP and NPE. The feasibility and effectiveness of the proposed method is verified on three popular face databases (Yale, AR and Extended Yale B) with promising results.