Locality Sensitive Hashing Technique Research Articles

Cluster-discovery of Twitter messages for event detection and trending

Social media data carries abundant hidden occurrences of real-time events. In this paper, a novel methodology is proposed for detecting and trending events from tweet clusters that are discovered by using locality sensitive hashing (LSH) technique. Key challenges include: (1) construction of dictionary using incremental term frequency–inverse document frequency (TF–IDF) in high-dimensional data to create tweet feature vector, (2) leveraging LSH to find truly interesting events, (3) trending the behavior of event based on time, geo-locations and cluster size, and (4) speed-up the cluster-discovery process while retaining the cluster quality. Experiments are conducted for a specific event and the clusters discovered using LSH and K-means are compared with group average agglomerative clustering technique.

Fast Fuzzy Search for Mixed Data Using Locality Sensitive Hashing

The drastic increase in data volume strongly demands efficient search techniques for similar data to queries. It is sometimes useful to specify data of interest with fuzzy constraints. When data objects contain both numerical and categorical attributes, it is usually not easy to define commonly-accepted distance measures between data objects. With no efficient indexing structure, it costs much to search for specific data objects because a linear search needs to be conducted over the whole data set. This paper proposes a method to use locality sensitive hashing technique and fuzzy constrained queries to search for interesting ones from big data. The method builds up a locality sensitive hashing-based indexing structure only with constituting continuous attributes, collects a small number of candidate data objects to which query is examined, and then evaluates their satisfaction degree to fuzzy constrained query so that data objects satisfying the query are determined.

Locality-Sensitive Hashing Techniques for Nearest Neighbor Search

When the volume of data grows big, some simple tasks could become a significant concern. Nearest neighbor search is such a task which finds from a data set the k nearest data points to queries. Locality-sensitive hashing techniques have been developed for approximate but fast nearest neighbor search. This paper introduces the notion of locality-sensitive hashing and surveys the locality-sensitive hashing techniques. It categories them based on several criteria, presents their characteristics, and compares their performance.

A Projection-Based Locality-Sensitive Hashing Technique for Reducing False Negatives

It is challenging to efficiently find similar pairs of objects when the number of objects is huge. The locality-sensitive hashing techniques have been developed to address this issue. They employ the hash functions to map objects into buckets, where similar objects have high chances to fall into the same buckets. This paper is concerned with a locality-sensitive hashing technique, the projection-based method, which is applicable to the Euclidean distance-based similar pair identification problem. It proposes an extended method which allows an object to be hashed to more than one bucket by introducing additional hashing functions. From the experimental studies, it has been shown that the proposed method could provide better performance compared to the projection-based method.

A Locality Sensitive Hashing Technique for Categorical Data

The measured data may contain various types of attributes such as continuous, categorical, and set-valued attributes. Several locality-sensitive hashing techniques, which enable to find similar pairs of data in a fast and approximate way, have been developed for data with either numeric or set-valued attributes. This paper introduces a new locality sensitive-hashing technique applicable to data with categorical attributes.

Satellite image retrieval using low memory locality sensitive hashing in Euclidean space

This paper presents the use of the Low Memory Locality Sensitive Hashing (LMLSH) technique operating in Euclidean space to build a data structure for the Defense Meteorological Satellite Program (DMSP) satellite imagery database. The LMLSH technique finds satellite image matches in sublinear search time. The texture feature vectors of the images are extracted using pyramid-structured wavelet transform coupled with Gaussian central moment technique. These feature vectors and families of hash functions, drawn randomly and independently from a Gaussian distribution, are used to build hash tables. Given a query, the hash tables are used to pull out the best matches to that query and this is done in a sublinear search time complexity. When tested, our algorithm has proven to be approximately twenty six times faster than the Linear Search (LS) algorithm. In addition, the LMLSH algorithm searches about two percent of the entire database randomly to find the possible matches to any given query without loss of accuracy compared to the absolute best matches returned by its LS counterpart.