Hashing-based clustering in high dimensional data

Abstract

Clustering is one of the most important techniques for the design of intelligent systems, and it has been incorporated into a large number of real applications. However, classical clustering algorithms cannot process high-dimensional data, such as text, in a reasonable amount of time. To address this problem, we use techniques based on locality-sensitive hashing (LSH), which was originally designed as an efficient means of solving the near-neighbor search problem for high-dimensional data. We propose the use of two LSH strategies to group high-dimensional data: MinHash, which enables Jaccard similarity approximations, and SimHash, which approximates cosine similarity. Instead of creating a computational costly data structure for responding to queries from near neighbors, we use a low-dimensional Hamming embedding to approximate a pairwise similarity matrix using a single-pass procedure. This procedure does not require data storage. It requires only the maintenance of a low-dimensional embedding. Then, the clustering solution is found by applying the bisection method to the similarity matrix. In addition to the above, we propose an improvement to LSH that is beneficial for its use on high-dimensional data. This improvement introduces a penalty on the Hamming distance, which is used in conjunction with SimHash, thereby improving the cosine similarity approximation. Experimental results indicate that our proposal yields a solution that is very close to the one found by applying the bisection method to a matrix with complete information, with better running times and a lower use of memory.