Sparse Locality Preserving Embedding

Abstract

Principal component analysis (PCA) is an unsupervised linear method of variables technique used in data compression, classification, and visualization[1]. The essence of PCA is to extract principal components, linear combinations of input variables that together best account for the variance in a data set. Linear discriminant analysis (LDA) is a favored tool for supervised linear classification in many areas because of its simplicity and robustness[2]. The goal of LDA is to provide low dimensional projections of data onto the most discriminative directions. Locality preserving projection (LPP) is a recently proposed method[3], which can be regarded as the linearization of Laplacian EigenMap[4]. When applied to face recognition tasks, LPP is also called LaplacianFaces. The idea behind LPP is that it considers the manifold structure of the data set, and preserves the locality of data in the embedding space. LPP has shown the superiority in terms of image indexing and face recognition. One of the major disadvantages of these three methods is that the derived projections are linear combinations of all the original features. Hence the learned results are difficult to interpret. In this paper, we propose a novel algorithm, called sparse locality preserving embedding (SpLPE), which is based on lasso regression framework for learning sparse projections by incorporating c 1 penalty with conventional locality preserving projections. The affinity graph constructed in LPP encodes both discriminant and geometrical structure in the data[3]. Once the Laplacian matrix is computed, we recast the generalized eigenvalue problem of LPP in the lasso regression framework to obtain sparse basis functions. The proposed SpLPE is a combination of locality preserving with sparsity. Additionally, our algorithm can be performed in either supervised or unsupervised mode.