Subspace Learning and Feature Selection via Orthogonal Mapping

Abstract

A plethora of dimensionality reduction (DR) techniques, stemming from statistics, machine learning, or graph theory, has been developed. Their ultimate objective is to eliminate data redundancy without a significant information loss. A general formulation, known as graph embedding, offers a unified framework for describing several well-known DR algorithms. In this paper, the inclusion of several DR algorithms within this unified framework is demonstrated. The enforcement of orthogonality to the projection matrix within this framework is proven to be of vital importance, since Orthogonal Neighborhood Preserving Projections, Orthogonal Locality Preserving Projections, Orthogonal Isometric Projection, Orthogonal Linear Discriminant Analysis, and Orthogonal Local Tangent Space Alignment algorithms outperform their non-orthogonal counterparts, e.g., Neighborhood Preserving Embedding, Locality Preserving Projections, Isometric Mapping, Linear Discriminant Analysis, and Local Tangent Space Alignment. One may simultaneously also impose the $\ell _{2,1}$ norm regularization on the projection matrix, seeking for row-sparsity. This leads to the known Joint Feature Selection and Subspace Learning (JFSSL) framework. All DR algorithms are employed within JFSSL. It is proven that the use of orthogonal mapping algorithms within JFSSL against their non-orthogonal counterparts does not improve the recognition rate of NN classifier.