Graph regularized locally linear embedding for unsupervised feature selection

Abstract

As one of the important dimensionality reduction techniques, unsupervised feature selection (UFS) has enjoyed amounts of popularity over the last few decades, which can not only improve learning performance, but also enhance interpretability and reduce computational costs. The existing UFS methods often model the data in the original feature space, which cannot fully exploit the discriminative information. In this paper, to address this issue, we investigate how to strengthen the relationship between UFS and the feature subspace, so as to select relevant features more straightforwardly and effectively. Methodologically, a novel UFS approach, referred to as Graph Regularized Local Linear Embedding (GLLE), is proposed by integrating local linear embedding (LLE) and manifold regularization constrained in feature subspace into a unified framework. To be more specific, we explicitly define a feature selection matrix composed of 0 and 1, which can realize the process of UFS. For the purpose of modelling the feature selection matrix, we propose to preserve the local linear reconstruction relationship among neighboring data points in the feature subspace, which corresponds to LLE constrained in the feature subspace. To make the feature selection matrix more accurate, we propose to use manifold regularization as an assistant of LLE to find the relevant and representative features such that the selected features can make each sample under the feature subspace be accordance with the manifold assumption. A tailored iterative algorithm based on Alternative Direction Method of Multipliers (ADMM) is designed to solve the proposed optimization problem. Extensive experiments on twelve real-world benchmark datasets are conducted, and the more promising results are achieved compared with the state-of-the-arts approaches.