Exact Nearest Neighbor Research Articles

Optimal Partition Trees

We revisit one of the most fundamental classes of data structure problems in computational geometry: range searching. Matoušek (Discrete Comput. Geom. 10:157–182, 1993) gave a partition tree method for d-dimensional simplex range searching achieving O(n) space and O(n 1−1/d ) query time. Although this method is generally believed to be optimal, it is complicated and requires O(n 1+ε ) preprocessing time for any fixed ε>0. An earlier method by Matoušek (Discrete Comput. Geom. 8:315–334, 1992) requires O(nlogn) preprocessing time but O(n 1−1/d log O(1) n) query time. We give a new method that achieves simultaneously O(nlogn) preprocessing time, O(n) space, and O(n 1−1/d ) query time with high probability. Our method has several advantages: Numerous consequences follow (e.g., improved results for computing spanning trees with low crossing number, ray shooting among line segments, intersection searching, exact nearest neighbor search, linear programming queries, finding extreme points, …).

Group Enclosing Queries

Given a set of points P and a query set Q, a group enclosing query (Geq) fetches the point p* ∈ P such that the maximum distance of p* to all points in Q is minimized. This problem is equivalent to the Min-Max case (minimizing the maximum distance) of aggregate nearest neighbor queries for spatial databases. This work first designs a new exact solution by exploring new geometric insights, such as the minimum enclosing ball, the convex hull, and the furthest voronoi diagram of the query group. To further reduce the query cost, especially when the dimensionality increases, we turn to approximation algorithms. Our main approximation algorithm has a worst case √2-approximation ratio if one can find the exact nearest neighbor of a point. In practice, its approximation ratio never exceeds 1.05 for a large number of data sets up to six dimensions. We also discuss how to extend it to higher dimensions (up to 74 in our experiment) and show that it still maintains a very good approximation quality (still close to 1) and low query cost. In fixed dimensions, we extend the √2-approximation algorithm to get a (1 + ε)-approximate solution for the Geq problem. Both approximation algorithms have O(log N + M) query cost in any fixed dimension, where N and M are the sizes of the data set P and query group Q. Extensive experiments on both synthetic and real data sets, up to 10 million points and 74 dimensions, confirm the efficiency, effectiveness, and scalability of the proposed algorithms, especially their significant improvement over the state-of-the-art method.

Adaptive Cluster Distance Bounding for High-Dimensional Indexing

We consider approaches for similarity search in correlated, high-dimensional data sets, which are derived within a clustering framework. We note that indexing by “vector approximation” (VA-File), which was proposed as a technique to combat the “Curse of Dimensionality,” employs scalar quantization, and hence necessarily ignores dependencies across dimensions, which represents a source of suboptimality. Clustering, on the other hand, exploits interdimensional correlations and is thus a more compact representation of the data set. However, existing methods to prune irrelevant clusters are based on bounding hyperspheres and/or bounding rectangles, whose lack of tightness compromises their efficiency in exact nearest neighbor search. We propose a new cluster-adaptive distance bound based on separating hyperplane boundaries of Voronoi clusters to complement our cluster based index. This bound enables efficient spatial filtering, with a relatively small preprocessing storage overhead and is applicable to euclidean and Mahalanobis similarity measures. Experiments in exact nearest-neighbor set retrieval, conducted on real data sets, show that our indexing method is scalable with data set size and data dimensionality and outperforms several recently proposed indexes. Relative to the VA-File, over a wide range of quantization resolutions, it is able to reduce random IO accesses, given (roughly) the same amount of sequential IO operations, by factors reaching 100X and more.

APPROXIMATE NEAREST NEIGHBOR SEARCH UNDER TRANSLATION INVARIANT HAUSDORFF DISTANCE

We present an embedding and search reduction which allow us to build the first data structure for the nearest neighbor search among small point sets with respect to the directed Hausdorff distance under translation. The search structure is non-heuristic in the sense that the quality of the results, the performance, and the space bound are guaranteed. Let n denote the number of point sets in the data base, s the maximal size of a point set, and d the dimension of the points. The nearest neighbor of a query point set under the translation invariant directed Hausdorff distance can be approximated by one or several nearest neighbor searches for single points in the Euclidean embedding space ℝd(s-1). The approximation factor is [Formula: see text] in case of even s and [Formula: see text] when s is odd. Depending on the direction of the Hausdorff distance either the number of queries or the space requirements are exponential in s. Furthermore it is shown how to find the exact nearest neighbor under the directed Hausdorff distance without transformation of the point sets within some weaker time and space bounds.

Cell-Probe Proofs

We study the nondeterministic cell-probe complexity of static data structures. We introduce cell-probe proofs (CPP), a proof system for the cell-probe model, which describes verification instead of computation in the cell-probe model. We present a combinatorial characterization of CPP. With this novel tool, we prove the following lower bounds for the nondeterministic cell-probe complexity of static data structures. –There exists a data structure problem with high nondeterministic cell-probe complexity. –For the exact nearest neighbor search (NNS) problem or the partial match problem in high dimensional Hamming space, for any data structure with Poly( n ) cells, each of which contains O ( n C ) bits where C < 1, the nondeterministic cell-probe complexity is at least Ω(log( d /log n )), where d is the dimension and n is the number of points in the data set. –For the polynomial evaluation problem of d -degree polynomial over finite field of size 2 k where d ≤ 2 k , for any data structure with s cells, each of which contains b bits, the nondeterministic cell-probe complexity is at least min (( k / b ( d − 1)), ( k −log( d −1)/log s )).

Approximate and probabilistic methods

Why? The metric search paradigm has been to this day successfully applied to several real-world problems, ranging from multimedia to data mining, from decision support to pattern recognition, to statistical and medical applications. Indeed, its simplicity makes it a perfect candidate for solving a variety of similarity problems arising in applications [4, 11]. The casual reader may wonder what prevents the metric space paradigm to become ubiquitously applicable to the ever-increasing range of applications that can benefit from it. The answer to this question is so dreadful that researchers have given it the hideous name of " curse of dimensionality" (an entry in this issue of the bulletin is devoted to this concept). In its essence, the curse of dimensionality says that, whenever the (intrinsic) dimensionality D of the metric space is high, an efficient solution to NN (nearest neighbor) queries. is impossible, and only a sequential scan of the whole dataset could guarantee that the correct result is found. This behavior is basically due to the fact that the variance of the distances to the query object q vanishes with increasing values of D , so that all data objects have almost the same distance to q . In such scenarios, one may however argue that NN queries lose of significance, since any data object would have a distance to the query object comparable to the minimal one [2]. On the other hand, in several real-world cases searching for the exact NN is still difficult, yet the distribution of distances exhibits a sufficiently high variance to make the problem worth solving. In such cases, it is also observed that locating the NN of a query point is, in itself, a relatively easy task, whose complexity indeed decreases with space dimensionality. As a matter of fact, the hard problem in high- D exact NN search is to determine how to stop, i.e., how to guarantee that the current result is the correct one. From this it follows that most of the time spent in an (exact) NN search is wasted time, during which little (or no) improvement is obtained [5].

Efficient and accurate nearest neighbor and closest pair search in high-dimensional space

Nearest Neighbor (NN) search in high-dimensional space is an important problem in many applications. From the database perspective, a good solution needs to have two properties: (i) it can be easily incorporated in a relational database, and (ii) its query cost should increase sublinearly with the dataset size, regardless of the data and query distributions. Locality-Sensitive Hashing (LSH) is a well-known methodology fulfilling both requirements, but its current implementations either incur expensive space and query cost, or abandon its theoretical guarantee on the quality of query results. Motivated by this, we improve LSH by proposing an access method called the Locality-Sensitive B-tree (LSB-tree) to enable fast, accurate, high-dimensional NN search in relational databases. The combination of several LSB-trees forms a LSB-forest that has strong quality guarantees, but improves dramatically the efficiency of the previous LSH implementation having the same guarantees. In practice, the LSB-tree itself is also an effective index which consumes linear space, supports efficient updates, and provides accurate query results. In our experiments, the LSB-tree was faster than: (i) iDistance (a famous technique for exact NN search) by two orders of magnitude, and (ii) MedRank (a recent approximate method with nontrivial quality guarantees) by one order of magnitude, and meanwhile returned much better results. As a second step, we extend our LSB technique to solve another classic problem, called Closest Pair (CP) search, in high-dimensional space. The long-term challenge for this problem has been to achieve subquadratic running time at very high dimensionalities, which fails most of the existing solutions. We show that, using a LSB-forest, CP search can be accomplished in (worst-case) time significantly lower than the quadratic complexity, yet still ensuring very good quality. In practice, accurate answers can be found using just two LSB-trees, thus giving a substantial reduction in the space and running time. In our experiments, our technique was faster: (i) than distance browsing (a well-known method for solving the problem exactly) by several orders of magnitude, and (ii) than D-shift (an approximate approach with theoretical guarantees in low-dimensional space) by one order of magnitude, and at the same time, outputs better results.

Approximate k-NN delta test minimization method using genetic algorithms: Application to time series

In many real world problems, the existence of irrelevant input variables (features) hinders the predictive quality of the models used to estimate the output variables. In particular, time series prediction often involves building large regressors of artificial variables that can contain irrelevant or misleading information. Many techniques have arisen to confront the problem of accurate variable selection, including both local and global search strategies. This paper presents a method based on genetic algorithms that intends to find a global optimum set of input variables that minimize the Delta Test criterion. The execution speed has been enhanced by substituting the exact nearest neighbor computation by its approximate version. The problems of scaling and projection of variables have been addressed. The developed method works in conjunction with MATLAB's Genetic Algorithm and Direct Search Toolbox. The goodness of the proposed methodology has been evaluated on several popular time series examples, and also generalized to other non-time-series datasets.

Towards Optimal Indexing for Relevance Feedback in Large Image Databases$^+$

Motivated by the need to efficiently leverage user relevance feedback in content-based retrieval from image databases, we propose a fast, clustering-based indexing technique for exact nearest-neighbor search that adapts to the Mahalanobis distance with a varying weight matrix. We derive a basic property of point-to-hyperplane Mahalanobis distance, which enables efficient recalculation of such distances as the Mahalanobis weight matrix is varied. This property is exploited to recalculate bounds on query-cluster distances via projection on known separating hyperplanes (available from the underlying clustering procedure), to effectively eliminate noncompetitive clusters from the search and to retrieve clusters in increasing order of (the appropriate) distance from the query. We compare performance with an existing variant of VA-File indexing designed for relevance feedback, and observe considerable gains.

Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions

In this article, we give an overview of efficient algorithms for the approximate and exact nearest neighbor problem. The goal is to preprocess a dataset of objects (e.g., images) so that later, given a new query object, one can quickly return the dataset object that is most similar to the query. The problem is of significant interest in a wide variety of areas.

Cell-probe lower bounds for the partial match problem

Given a database of n points in {0,1} d , the partial match problem is: In response to a query x in {0,1,∗} d , is there a database point y such that for every i whenever x i≠∗ , we have x i = y i . In this paper we show randomized lower bounds in the cell–probe model for this well-studied problem (Analysis of associative retrieval algorithms, Ph.D. Thesis, Stanford University, 1974; The Art of Computer Programming; Sorting and Searching, Addison-Wesley, Reading, MA, 1973; SIAM J. Comput. 5(1) (1976) 19; J. Comput. System Sci. 57(1) (1998) 37; Proceedings of the 31st Annual ACM Symposium on Theory of Computing, 1999; Proceedings of the 29th International Colloquium on Algorithms, Logic, and Programming, 1999). Our lower bounds follow from a near-optimal asymmetric communication complexity lower bound for this problem. Specifically, we show that either Alice has to send Ω(d/ log n) bits or Bob has to send Ω(n 1−o(1)) bits. When applied to the cell–probe model, it means that if the number of cells is restricted to be poly( n, d) where each cell is of size poly( log n,d) , then Ω(d/ log 2 n) probes are needed. This is an exponential improvement over the previously known lower bounds for this problem obtained by Miltersen et al. (1998) and Borodin et al. (1999). Our lower bound also leads to new and improved lower bounds for related problems including a lower bound for the ℓ ∞ c-nearest neighbor problem for c<3 and an improved communication complexity lower bound for the exact nearest neighbor problem.

On Approximate Nearest Neighbors under l∞ Norm

The nearest neighbor search (NNS) problem is the following: Given a set of n points P={p1, …, pn} in some metric space X, preprocess P so as to efficiently answer queries which require finding a point in P closest to a query point q∈X. The approximate nearest neighbor search (c-NNS) is a relaxation of NNS which allows to return any point within c times the distance to the nearest neighbor (called c-nearest neighbor). This problem is of major and growing importance to a variety of applications. In this paper, we give an algorithm for (4⌈log1+ρlog4d⌉+1)-NNS algorithm in ld∞ with O(dn1+ρlogO(1)n) storage and O(dlogO(1)n) query time. Moreover, we obtain an algorithm for 3-NNS for l∞ with nlogd+1 storage. The preprocessing time is close to linear in the size of the data structure. The algorithm can be also used (after simple modifications) to output the exact nearest neighbor in time bounded by O(dlogO(1)n) plus the number of (4⌈log1+ρlog4d⌉+1)-nearest neighbors of the query point. Building on this result, we also obtain an approximation algorithm for a general class of product metrics. Finally, we show that for any c<3 the c-NNS problem in l∞ is provably as hard as the subset query problem (also called the partial match problem). This indicates that obtaining a sublinear query time and subexponential (in d) space for c<3 might be hard.

Adsorption Layer of Charged Surfaces: An Exact Nearest-Neighbor Model for the Linear Lattice

The effects of finite ion size and polyion concentration on the adsorption isotherms are examined within the context of a one-dimensional lattice restricted to nearest-neighbor interactions. The nearest-neighbor interactions include that of the bound ion with adjacent lattice sites as well as with a neighbor bound ion. Both screened and unscreened Coulombic interactions are employed, as they are shown to exhibit different behaviors under high salt concentrations. For the screened interactions, previously adsorbed ions are released, whereas for the unscreened Coulombic interaction charge reversal is observed for sufficiently separated lattice sites. The effect of polyion concentration is to provide a constant degree of adsorption for added salt concentrations less than the concentration ofcounterions released by the polyion. The model is applied to experimental data on monovalent and divalent salts.

Exact nearest neighbor statistics for λ-bell particles on a one-dimensional lattice

Relationships are developed which describe exactly the degeneracies of those states specified by the number of either the occupied, mixed, or vacant nearest neighbor pairs which exist when indistinguishable λ-bell particles (occupying λ contiguous sites) are distributed on a one-dimensional lattice space.

Inapplicability of “Scatchard‐type” analysis to highly cooperative binding systems

AbstractThere are several studies of small molecule binding by biopolymers which use “Scatchard‐type” plots to determine the intrinsic binding constant (k) and pairwise interaction energy between bound sites (S). “Scatchard‐type” refers to the plot ln (r/m(n – r)) versus r in this text whereas “Scatchard plot” conventionally refers to the plot r/m versus r. The quantities r, n, and m are, respectively, the average bound ligand/site, ligand/site at saturation, and concentration of free ligand. Although these plots result from approximate solutions to the problem of binding small molecules on a general three‐dimensional lattice, they are often used to represent binding isotherms in linear systems for which the solutions are known exactly. A critical examination regarding the applicability of “Scatchard‐type” analysis to systems with a large number of binding sites is presented herein.Both experimental and simulated binding data are analyzed by “Scatchard‐type” and exact nearest neighbor plots (θ versus ln m, where θ = r/n). Numerical determination of k and S for the two theoretical approaches differ greatly in highly cooperative systems although their theoretical equivalence (interconversion) is demonstrated. A modified Scatchard theory, which correctly counts the degenerate energy states, is presented which reconciles this discrepancy. The “Scatchard‐type” plot for the modified theory in the highly cooperative limit is mathematically equivalent to the form ln (r/n – r) + p, where p contains all the molecular information and locates the r/n = 1/2 point. Contrary to the original theory, the r dependence in the “Scatchard‐type” representation is devoid of molecular interpretation and is characterized by curvature in the low and high‐binding regions with a (maximum) slope of 4 at r/n = 1/2. The modified theory is shown to reproduce accurately the poly(cytidylic acid)‐guanosine data of Sarocchi et al. and to account for the maxima exhibited in the conventional Scatchard plots.

Exact nearest neighbor degeneracy for dumbbells on a one-dimensional lattice space

Relationships are developed which describe exactly the degeneracy associated with nearest neighbor pairs of occupied sites (1–1), mixed sites (0–1), and vacant sites (0–0) for dumbbells distributed on a one-dimensional lattice space. The first moments of these statistics are calculated, thereby permitting an evaluation of the error inherent in the use of the Bragg-Williams approximation for this situation.