Neighbor Search In High Dimensional Space Research Articles

Towards Efficient Index Construction and Approximate Nearest Neighbor Search in High-Dimensional Spaces

The approximate nearest neighbor (ANN) search in high-dimensional spaces is a fundamental but computationally very expensive problem. Many methods have been designed for solving the ANN problem, such as LSH-based methods and graph-based methods. The LSH-based methods can be costly to reach high query quality due to the hash-boundary issues, while the graph-based methods can achieve better query performance by greedy expansion in an approximate proximity graph (APG). However, the construction cost of these APGs can be one or two orders of magnitude higher than that for building hash-based indexes. In addition, they fail short in incrementally maintaining APGs as the underlying dataset evolves. In this paper, we propose a novel approach named LSH-APG to build APGs and facilitate fast ANN search using a lightweight LSH framework. LSH-APG builds an APG via consecutively inserting points based on their nearest neighbor relationship with an efficient and accurate LSH-based search strategy. A high-quality entry point selection technique and an LSH-based pruning condition are developed to accelerate index construction and query processing by reducing the number of points to be accessed during the search. LSH-APG supports fast maintenance of APGs in lieu of building them from scratch as dataset evolves. Its maintenance cost and query cost for a point is proven to be less affected by dataset cardinality. Extensive experiments on real-world and synthetic datasets demonstrate that LSH-APG incurs significantly less construction cost but achieves better query performance than existing graph-based methods.

HCTree+: A workload-guided index for approximate [formula omitted]NN search

The problem of approximate k-nearest neighbor search in high-dimensional space is a fundamental problem in many applications. We have observed that most existing approaches are unable to leverage the query results to improve the search performance. To address this limitation, we present a new index, called HCTree+, that aims to improve the query performance based on incoming queries and their results. First, we adopt a simple yet effective index to support efficient search. Second, incoming queries and their results are used to optimize the index dynamically for future queries. The experimental study shows that HCTree+ outperforms the state-of-the-art algorithms in terms of accuracy while achieving the desired efficiency.

Distance Encoded Product Quantization for Approximate K-Nearest Neighbor Search in High-Dimensional Space.

Approximate K-nearest neighbor search is a fundamental problem in computer science. The problem is especially important for high-dimensional and large-scale data. Recently, many techniques encoding high-dimensional data to compact codes have been proposed. The product quantization and its variations that encode the cluster index in each subspace have been shown to provide impressive accuracy. In this paper, we explore a simple question: is it best to use all the bit-budget for encoding a cluster index? We have found that as data points are located farther away from the cluster centers, the error of estimated distance becomes larger. To address this issue, we propose a novel compact code representation that encodes both the cluster index and quantized distance between a point and its cluster center in each subspace by distributing the bit-budget. We also propose two distance estimators tailored to our representation. We further extend our method to encode global residual distances in the original space. We have evaluated our proposed methods on benchmarks consisting of GIST, VLAD, and CNN features. Our extensive experiments show that the proposed methods significantly and consistently improve the search accuracy over other tested techniques. This result is achieved mainly because our methods accurately estimate distances.

Effective optimizations of cluster-based nearest neighbor search in high-dimensional space

Nearest neighbor (NN) search in high-dimensional space plays a fundamental role in large-scale image retrieval. It seeks efficient indexing and search techniques, both of which are simultaneously essential for similarity search and semantic analysis. However, in recent years, there has been a rare breakthrough. Achievement of current techniques for NN search is far from satisfactory, especially for exact NN search. A recently proposed method, HB, addresses the exact NN search efficiently in high-dimensional space. It benefits from cluster-based techniques which can generate more compact representation of the data set than other techniques by exploiting interdimensional correlations. However, HB suffers from huge cost for lower bound computations and provides no further pruning scheme for points in candidate clusters. In this paper, we extend the HB method to address exact NN search in correlated, high-dimensional vector data sets extracted from large-scale image database by introducing two new pruning/selection techniques and we call it HB+. The first approach aims at selecting more quickly the subset of hyperplanes/clusters that must be considered. The second technique prunes irrelevant points in the selected subset of clusters. Performed experiments show the improvement of HB+ with respect to HB in terms of efficiency (I/O cost and CPU response time) and also demonstrate the superiority over other exact NN indexes.

An Error Minimizing Partitioning Method for the Nearest Neighbor Search

The nearest neighbor search in high-dimensional space is an important operation in many applications, such as data mining and multimedia databases. Evaluating similarity in high-dimensional space requires high computational cost; index-structures are frequently used for reducing computational cost. Most of these index-structures are built by partitioning the data set. However, the partitioning approaches potentially have the problem of failing to find the nearest neighbor that is caused by partitions. In this paper, we propose the Error Minimizing Partitioning (EMP) method with a novel tree structure that minimizes the failures of finding the nearest neighbors. EMP divides the data into subsets with considering the distribution of data sets. For partitioning a data set, the proposed method finds the line that minimizes the summation of distance to data points. The method then finds the median of the data set. Finally, our proposed method determines the partitioning hyper-plane that passes the median and is perpendicular to the line. We also make a comparative study between existing methods and the proposed method to verify the effectiveness of our method.

Efficient nearest neighbor query based on extended B +-tree in high-dimensional space

Nearest neighbor queries in high-dimensional space are important in various applications. One-dimensional mapping is an efficient indexing method to speed up the k-nearest neighbor search, which can transform a high-dimensional point into a single-dimensional value indexed by a B +-tree. In this paper, we present a new one-dimensional indexing scheme based on extended B +-tree for k-nearest neighbor search in high-dimensional space. We first partition the high-dimensional dataset and perform Principal Component Analysis on each partition. The distance of each point to the center of the partition is indexed using a B +-tree, and the projection on the first principal component of each point is embedded into leaf node of the B +-tree. In the query, a new filter strategy according to the spatial relationship between the query point and the axis determined by the first principal component is applied to improve the query performance. We also present a novel k-nearest neighbor search algorithm which can guarantee the accuracy of query results. Extensive experiments have been indicative of the effectiveness of our approach.

Effectiveness of optimal incremental multi-step nearest neighbor search

The development of techniques that facilitate effective similarity search is important for many applications such as multi-media databases, content-based image retrieval, molecular biology, medical imaging, and object recognition, among others. Two of the common operations in this context are range queries and k-nearest neighbor search in high-dimensional space. However, the distance measures used to determine the dissimilarities between high-dimensional feature vectors are often expensive to compute. To reduce the number of expensive distance calculations in the search process, Korn, Sidiropoulos, Faloutsos, Siegel, and Protopapas (1996) proposed a multi-step algorithm, which involves two stages: filtering and refinement. It employs an easily computable lower-bound distance measure to filter out a candidate set in the filtering stage and confine the expensive distance computation to a small candidate set in the refinement stage. This algorithm was later improved by Seidl and Kriegel (1998) to produce optimal-sized candidate set in the filtering stage; the improved algorithm is said to be filtering optimal. However, the improved algorithm cannot produce the result incrementally in the refinement stage. The improved algorithm can only start to produce results after the whole search process stops, which is a disadvantage in real applications. In this paper, we experimentally demonstrate the applicability and effectiveness of an extended version of the algorithm that can produce the nearest neighbors incrementally in an optimal way in the sense that a nearest neighbor is output as soon as it can be determined using the existing information; thus, nearest neighbors are produced in order. Our algorithm is both filtering and refinement optimal, and well serves real applications. We have already proved the optimality of the proposed extended algorithm ( Zhang, Alhajj, & Rokne, 2008), and in here we empirically demonstrate its independence on the number of nearest neighbors and its effectiveness in early retrieving results as compared to the previous algorithm.

Fast multidimensional nearest neighbor search algorithm using priority queue

AbstractNearest neighbor search in high‐dimensional spaces is an interesting and important problem which is relevant for a wide variety of applications, including multimedia information retrieval, data mining, and pattern recognition. For such applications, the curse of high dimensionality tends to be a major obstacle in the development of efficient search methods. This paper addresses the problem of designing an efficient algorithm for high‐dimensional nearest neighbor search using a priority queue. The proposed algorithm is based on a simple linear search algorithm and eliminates unnecessary arithmetic operations from distance computations between multidimensional vectors. Moreover, we propose two techniques, a dimensional sorting method and a PCA‐based method, to accelerate multidimensional search. Experimental results indicate that our scheme scales well even for a very large number of dimensions. © 2008 Wiley Periodicals, Inc. Electr Eng Jpn, 164(3): 69–77, 2008; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/eej.20502

Indexing the solution space: a new technique for nearest neighbor search in high-dimensional space

Similarity search in multimedia databases requires an efficient support of nearest-neighbor search on a large set of high-dimensional points as a basic operation for query processing. As recent theoretical results show, state of the art approaches to nearest-neighbor search are not efficient in higher dimensions. In our new approach, we therefore precompute the result of any nearest-neighbor search which corresponds to a computation of the Voronoi cell of each data point. In a second step, we store conservative approximations of the Voronoi cells in an index structure efficient for high-dimensional data spaces. As a result, nearest neighbor search corresponds to a simple point query on the index structure. Although our technique is based on a precomputation of the solution space, it is dynamic, i.e., it supports insertions of new data points. An extensive experimental evaluation of our technique demonstrates the high efficiency for uniformly distributed as well as real data. We obtained a significant reduction of the search time compared to nearest neighbor search in other index structures such as the X-tree.