Regression analysis of locality preserving projections via sparse penalty

Abstract

Recent studies have shown that linear subspace algorithms, such as Principal Component Analysis, Linear Discriminant Analysis and Locality Preserving Projections, have attracted tremendous attention in many fields of information processing. However, the projection results obtained by these algorithms are linear combination of the original features, which is difficult to be interpreted psychologically and physiologically. Motivated by Compressive Sensing theory, we formulate the generalized eigenvalue problem under CS framework, which then allows us to apply a sparsity penalty and minimization procedure to locality preserving projections. The proposed algorithm is called sparse locality preserving projections, which performs locality preserving projections in the lasso regression framework that dimensionality reduction, feature selection and classification are merged into one analysis. The method is also extended to its regularized form to improve its generalization. The proposed algorithm is a combination of locality preserving with sparse penalty. Additionally, the algorithm can be performed in either supervised or unsupervised tasks. Experimental results on toy and real data sets show that our methods are effective and demonstrate much higher performance.