A general framework for scalable spectral clustering based on document models

Abstract

We propose a new, general framework for fast, approximate spectral clustering on large data sets. We first consider the special setting of cosine similarity for clustering sparse data (e.g., documents under the bag-of-words model) or data of at most a few hundred dimensions (e.g., small images). We show that in those cases various versions of spectral clustering, such as the Ng-Jordan-Weiss algorithm (NIPS 2001), Normalized Cut (Shi and Malik, 2000), and Diffusion Maps (Coifman and Lafon, 2006), can be implemented solely based on three kinds of efficient operations on the data matrix – elementwise manipulation, matrix-vector multiplication and low-rank SVD, thus eliminating the need to compute the weight matrix. For general similarity and any kind of data, we present a landmark-based technique that first converts the given data (or a landmark set selected from them) to a collection of “documents” and then applies to them the scalable implementation of spectral clustering with cosine similarity. Our algorithm is simple to implement and fast to run, with additional benefits such as a naturally embedded outliers removal step. We conduct extensive experiments to compare our algorithm with a few existing methods to demonstrate its superior performance.