Fast neighborhood reconstruction with adaptive weights learning

Abstract

Neighborhood reconstruction is one of the hot topics in data analysis, as it has a powerful ability to capture the local manifold information of data. However, one common drawback of existing discriminant neighborhood reconstruction methods is that they partition the reconstruction weights learning and feature extraction into two separate steps; due to the existence of noisy and redundant features, this affects the learning performance. In addition to this, each sample is expressed by the linear combination of all other samples, and the sparsity of the reconstruction coefficients is usually constrained by the sparse norm or regularization term. This will not only cause heterogeneous samples to own unnecessary weights, but also increases the time complexity. To address these issues, in this paper, we propose a Fast and Adaptive Neighborhood Reconstruction (FANR) model, where each sample is expressed by representative points from the same class. By replacing the fully connected graph with a bipartite graph, the time complexity can be reduced from the original O(n3) to O(n), while the adverse impact of heterogeneous samples can also be avoided. The weights and representative points of the bipartite graph in each class are auto-updated in the ideal subspace to eliminate the influence of noisy and redundant features. Furthermore, to reduce the sensitivity of the model to the reduced dimension, PCA is introduced to hold the main energy of the original data. An iterative algorithm is developed to solve the model effectively. The results of experiments conduced on toy data and benchmark datasets demonstrate the efficiency and superiority of our proposed model.