Scalable Semi-Supervised Learning by Graph Construction with Approximate Anchors Embedding

Abstract

Semi-supervised learning (SSL) generalizes and improves supervised learning using labeled data and unlabeled data. With the rapid development of the Internet as well as the increasing availability of data in the open web, collecting tremendous amount of unlabeled data has became more feasible. As the central notion in SSL, smoothness is often defined on a graph representation of the data. However, only few researches up to date adapt graph-based SSL approaches into the large scale Solution. Even if approximation approaches are commonly used in methods of the large-scale SSL, it is difficult to outperform original methods. In this paper, we propose an efficient method to construct a graph based on anchors. There are two major concerns in the anchor graph generally: how to learn better anchors and to present the input both in a more efficient and effective fashion. Intuitively, compared with sparse representation (e.g. local anchor embedding), a more straightforward approach is marginal regression with non-negative constraint. Rather than using clustering algorithms, the anchors are trained using dictionary learning with sparsity constraints. And thus in our approach, not only the relation between anchors and data is taken into consideration, the relation between anchors is also regarded. Beyond that, in the evaluation section, we demonstrate that our method outperforms other large scale SSL methods as well as traditional ones in classification performance according to several classical datasets. Further more, the proposed method solves the large scale SSL problem more efficient than current methods. Therefore, our method is an efficient and effective alternative to handle large scale SSL problem.