Unsupervised Large Graph Embedding Based on Balanced and Hierarchical K-means

Abstract

There are many successful spectral based unsupervised dimensionality reduction methods, including Laplacian Eigenmap (LE), Locality Preserving Projection (LPP), Spectral Regression (SR), etc. We discover that LPP and SR are equivalent if the symmetric similarity matrix is {doubly stochastic}, Positive Semi-Definite (PSD) and with rank p, where p is the reduced dimension. Since solving SR is believed faster than solving LPP based on some related literature, the discovery promotes us to seek to construct such specific similarity matrix to speed up LPP solving procedures. We then propose an unsupervised linear method called Unsupervised Large Graph Embedding (ULGE). ULGE starts with a similar idea as LPP but adopts an efficient approach to construct anchor-based similarity matrix and then performs spectral analysis on it. Moreover, since conventional anchor generation strategies suffer kinds of problems, we propose an efficient and effective anchor generation strategy, called Balanced K-means based Hierarchical K-means (BHKH). The computational complexity of ULGE can reduce to O(ndm), which is a significant improvement compared to conventional methods need $O(n^2d)$ at least, where n, d and m are the number of samples, dimensions, and anchors, respectively. Extensive experiments on several publicly available datasets demonstrate the efficiency and effectiveness of the proposed method.