Exact Nearest Neighbor Research Articles

Is it possible to find the single nearest neighbor of a query in high dimensions?

We investigate an open question in the study of the curse of dimensionality: Is it possible to find the single nearest neighbor of a query in high dimensions? Using the notion of (in)distinguishability to examine whether the feature map of a kernel is able to distinguish two distinct points in high dimensions, we analyze this ability of a metric-based Lipschitz continuous kernel as well as that of the recently introduced Isolation Kernel. Between the two kernels, we show that only Isolation Kernel has distinguishability and it performs consistently well in four tasks: indexed search for exact nearest neighbor search, anomaly detection using kernel density estimation, t-SNE visualization and SVM classification in both low and high dimensions, compared with distance, Gaussian and three other existing kernels.

Efficient Approximate Nearest Neighbor Search in Multi-dimensional Databases

Approximate nearest neighbor (ANN) search is a fundamental search in multi-dimensional databases, which has numerous real-world applications, such as image retrieval, recommendation, entity resolution, and sequence matching. Proximity graph (PG) has been the state-of-the-art index for ANN search. However, the search on existing PGs either suffers from a high time complexity or has no performance guarantee on the search result. In this paper, we propose a novel τ-monotonic graph (τ- MG) to address the limitations. The novelty of τ-MG lies in a τ-monotonic property. Based on this property, we prove that if the distance between a query q and its nearest neighbor is less than a constant τ, the search on τ-MG guarantees to find the exact nearest neighbor of q and the time complexity of the search is smaller than all existing PG-based methods. For index construction efficiency, we propose an approximate variant of τ-MG, namely τ-monotonic neighborhood graph (τ- MNG), which only requires the neighborhood of each node to be τ-monotonic. We further propose an optimization to reduce the number of distance computations in search. Our extensive experiments show that our techniques outperform all existing methods on well-known real-world datasets.

Accelerating exact nearest neighbor search in high dimensional Euclidean space via block vectors

The nearest neighbor search is an essential operation for many computer vision, data mining, and machine learning problems. Since it is so widely used, the nearest neighbor search should be as fast as possible. This paper explores lower bound-based approaches to speed up the exact nearest neighbor search in high dimensional Euclidean space. We compute the lower bound of Euclidean Distance by using the block vectors and Cauchy–Schwartz inequality. The proposed lower bound is calculated efficiently and is close to the real Euclidean Distance. Besides, the preprocessing step of the proposal has linear time complexity. Given a query, during the procedure of identifying the nearest neighbor, our method can eliminate many expensive actual distance computations using the lower bound to approximate Euclidean Distance. In addition, we develop a multilevel lower bound strategy, which calculates the lower bound step by step and utilizes the multistep filtering mechanism to improve the searching process further. Theoretical analysis is provided to show that the proposals can guarantee to obtain the same result as the brute-force search. Comprehensive experiments on 16 public data sets collected from various domains demonstrate that our approach performs well in finding the exact nearest neighbor compared to related competitors. The experimental results also illustrate that the multilevel lower bound strategy is effective.

Approx-SMOTE: Fast SMOTE for Big Data on Apache Spark

One of the main goals of Big Data research, is to find new data mining methods that are able to process large amounts of data in acceptable times. In Big Data classification, as in traditional classification, class imbalance is a common problem that must be addressed, in the case of Big Data also looking for a solution that can be applied in an acceptable execution time. In this paper we present Approx-SMOTE, a parallel implementation of the SMOTE algorithm for the Apache Spark framework. The key difference with the original SMOTE, besides parallelism, is that it uses an approximated version of k-Nearest Neighbor which makes it highly scalable. Although an implementation of SMOTE for Big Data already exists (SMOTE-BD), it uses an exact Nearest Neighbor search, which does not make it entirely scalable. Approx-SMOTE on the other hand is able to achieve up to 30 times faster run times without sacrificing the improved classification performance offered by the original SMOTE.

CapsNet-based supervised hashing

With the development of Internet technology, an increasing amount of data enters people’s daily life, which brings great challenges when users quickly search for interesting images. The existing exact nearest neighbor retrieval methods often fail to obtain results within an acceptable retrieval time, so researchers have begun to focus on approximate nearest neighbor retrieval. Recently, the hashing-based approximate nearest neighbor retrieval method has attracted increasing attention because of its small storage space and high retrieval efficiency. At present, one of the most advanced hashing methods is to use deep neural networks, especially convolutional neural networks (CNN), to obtain image hash codes to achieve fast image retrieval. However, CNN needs a large number of images during training, so it takes a lot of time to obtain training samples. In addition, CNN cannot handle ambiguity well, and a lot of information is lost in the pooling layer; furthermore, CNN cannot learn the hierarchical structure of the image. Aiming to address these problems while making full use of data classification information to guide hash learning in a supervised form to improve retrieval efficiency, we introduce the capsule network into the hash learning and propose CapsNet-based supervised hashing (CSH) to preserve the effective information of an image as much as possible. CSH adds a hashing layer equivalent to hash mapping between the capsule network and the decoder to perform hash learning. By optimizing the objective function defined for capsule network loss, reconstruction loss and hashing quantization loss, feature representations, hashing functions and classification results can be learned from the input data at the same time. To verify the effectiveness of the method, we performed experiments on multiple datasets. The experimental results show that this method is superior to the existing hashing-based image retrieval methods and achieves satisfactory results in image classification performance.

A Fast Nearest Neighbor Search Scheme Over Outsourced Encrypted Medical Images

Medical imaging is crucial for medical diagnosis, and the sensitive nature of medical images necessitates rigorous security and privacy solutions to be in place. In a cloud-based medical system for Healthcare Industry 4.0, medical images should be encrypted prior to being outsourced. However, processing queries over encrypted data without first executing the decryption operation is challenging and impractical at present. In this paper, we propose a secure and efficient scheme to find the exact nearest neighbor over encrypted medical images. Instead of calculating the Euclidean distance, we reject candidates by computing the lower bound of the Euclidean distance that is related to the mean and standard deviation of data. Unlike most existing schemes, our scheme can obtain the exact nearest neighbor rather than an approximate result. We, then, evaluate our proposed approach to demonstrate its utility.

Autoencoder-based unsupervised clustering and hashing

Faced with a large amount of data and high-dimensional data information in a database, the existing exact nearest neighbor retrieval methods cannot obtain ideal retrieval results within an acceptable retrieval time. Therefore, researchers have begun to focus on approximate nearest neighbor retrieval. Recently, the hashing-based approximate nearest neighbor retrieval method has attracted increasing attention because of its small storage space and high retrieval efficiency. The development of neural networks has also promoted progress in hash learning. However, these methods are mostly supervised. In practical applications, annotating large amounts of data is a very time-consuming and laborious task. Furthermore, efficiently using a large amount of unlabeled data for hash learning is challenging. In this paper, we create a new autoencoder variant to efficiently capture the features of high-dimensional data, and propose an unsupervised deep hashing method for large-scale data retrieval, named as Autoencoder-based Unsupervised Clustering and Hashing (AUCH). By constructing a hashing layer as a hidden layer of the autoencoder, hash learning is performed together with unsupervised clustering by minimizing the overall loss. AUCH can unify unsupervised clustering and retrieval tasks into a single learning model. In addition, the method can use a deep neural network to simultaneously learn feature representations, hashing functions and cluster assignments. Experimental results on standard datasets indicate that AUCH achieves competitive results compared to state-of-the-art models for retrieval and clustering tasks.

Towards Large-Scale Object Instance Search: A Multi-Block N-Ary Trie

Object instance search is a challenging task with a wide range of applications, but the fast search with high accuracy has not been well solved yet. In this paper, we investigate the object instance search from a new perspective in terms of joint precision and computational cost optimization, and propose a novel index structure i.e., Multi-Block N-ary Trie (MBNT) to accelerate the exact r-neighbor search in the Hamming space. Comprehensive studies are first carried out to reveal the performance of exact and approximate nearest neighbor (NN) algorithms for object instance search. An interesting finding that the exact search is more promising for very compact binary codes (e.g., 64-bit and 128-bit) is analyzed. Along this vein, we introduce a Trie structure, i.e., MBNT, which is specifically designed for improving the exact NN search performance in the context of large-scale object instance search. To index the binary codes, a subset of continuous bits of a binary string, denoted as a block, is regarded as an atomic indexing element. As such, the problem of lookup misses can be addressed. Theoretical analyses are also provided to show that our MBNT scheme can incur less computational cost than other hash table-based methods. Extensive experimental results on the 100M dataset have demonstrated that our method achieves faster search speed while maintaining the promising search precision towards large-scale object instance search.

Scalable Approximate FRNN-OWA Classification

Fuzzy rough nearest neighbor classification with ordered weighted averaging operators (FRNN-OWA) is an algorithm that classifies unseen instances according to their membership in the fuzzy upper and lower approximations of the decision classes. Previous research has shown that the use of OWA operators increases the robustness of this model. However, calculating membership in an approximation requires a nearest neighbor search. In practice, the query time complexity of exact nearest neighbor search algorithms in more than a handful of dimensions is near linear, which limits the scalability of FRNN-OWA. Therefore, we propose approximate FRNN-OWA, a modified model that calculates upper and lower approximations of decision classes using the approximate nearest neighbors returned by hierarchical navigable small worlds (HNSW), a recent approximative nearest neighbor search algorithm with logarithmic query time complexity at constant near-100% accuracy. We demonstrate that approximate FRNN-OWA is sufficiently robust to match the classification accuracy of exact FRNN-OWA while scaling much more efficiently. We test four parameter configurations of HNSW and evaluate their performance by measuring classification accuracy and construction and query times for samples of various sizes from three large datasets. We find that with two of the parameter configurations, approximate FRNN-OWA achieves near-identical accuracy to exact FRNN-OWA for most sample sizes within query times that are up to several orders of magnitude faster.

A probabilistic molecular fingerprint for big data settings

BackgroundAmong the various molecular fingerprints available to describe small organic molecules, extended connectivity fingerprint, up to four bonds (ECFP4) performs best in benchmarking drug analog recovery studies as it encodes substructures with a high level of detail. Unfortunately, ECFP4 requires high dimensional representations (≥ 1024D) to perform well, resulting in ECFP4 nearest neighbor searches in very large databases such as GDB, PubChem or ZINC to perform very slowly due to the curse of dimensionality.ResultsHerein we report a new fingerprint, called MinHash fingerprint, up to six bonds (MHFP6), which encodes detailed substructures using the extended connectivity principle of ECFP in a fundamentally different manner, increasing the performance of exact nearest neighbor searches in benchmarking studies and enabling the application of locality sensitive hashing (LSH) approximate nearest neighbor search algorithms. To describe a molecule, MHFP6 extracts the SMILES of all circular substructures around each atom up to a diameter of six bonds and applies the MinHash method to the resulting set. MHFP6 outperforms ECFP4 in benchmarking analog recovery studies. By leveraging locality sensitive hashing, LSH approximate nearest neighbor search methods perform as well on unfolded MHFP6 as comparable methods do on folded ECFP4 fingerprints in terms of speed and relative recovery rate, while operating in very sparse and high-dimensional binary chemical space.ConclusionMHFP6 is a new molecular fingerprint, encoding circular substructures, which outperforms ECFP4 for analog searches while allowing the direct application of locality sensitive hashing algorithms. It should be well suited for the analysis of large databases. The source code for MHFP6 is available on GitHub (https://github.com/reymond-group/mhfp).

Fast feature matching based on r ‐nearest k ‐means searching

Feature matching has been frequently applied in computer vision and pattern recognition. In this paper, the authors propose a fast feature matching algorithm for vector-based feature. Their algorithm searches r-nearest neighborhood clusters for the query point after a k-means clustering, which shows higher efficiency in three aspects. First, it does not reformat the data into a complex tree, so it shortens the construction time twice. Second, their algorithm adopts the r-nearest searching strategy to increase the probability to contain the exact nearest neighbor (NN) and take this NN as the global one, which can accelerate the searching speed by 170 times. Third, they set up a matching rule with a variant distance threshold to eliminate wrong matches. Their algorithm has been tested on large SIFT databases with different scales and compared with two widely applied algorithms, priority search km-tree and random kd-tree. The results show that their algorithm outperforms both algorithms in terms of speed up over linear search, and consumes less time than km-tree. Finally, they carry out the CFI test based on ISKLRS database using their algorithm. The test results show that their algorithm can greatly improve the recognition speed without affecting the recognition rate.

A comparative study of hash based approximate nearest neighbor learning and its application in image retrieval

Plenty of data are available due to the growth of digital technology that creates a high expectation in retrieving the relevant images, accurately and efficiently for a given query image. For searching the relevant images efficiently for the Large Scale dataset, the searching algorithm should have fast access capability. The existing Exact Nearest Neighbor search performs in linear time and so it takes more time as both the dataset and data dimension increases. As a remedy to provide sub-linear/logarithmic time complexity, usage of Approximate Nearest Neighbor (ANN) algorithms is emerging at a rapid rate. This paper discusses about the importance of ANN and their general classification; the different categories involved in Learning to Hash has been analyzed with their pros and cons; different bit assignment types and methods to minimize the Quantization Errors have been reviewed along with its merits and demerits. Therefore, it serves to increase the efficiency of the Image Retrieval process in Large Scale.

BLBTree: an Efficient Index Structure for Fast Search

Due of its interest in different areas, similarity search in high-dimensional spaces is one of the principal research axes today for CBIR Systems. The properties for high-dimensional data demand some adequate search methods to have a research in optimal time. Due to the curse of dimensionality, search time in the index structure exponentially increases, according the dimension of the descriptor. However, the performances of classical index structures become less than the sequential scan of data in search time when answering exact nearest neighbor queries. To overcome this problematic, two major approaches for high-dimensional data in CBIR systems have been proposed: The first approach permits to speed up the search by using multiple levels of lower bound distances, the second approach exploits the BTree index structure to speed up the search. We propose to combine both techniques to search for large nearest neighbors in a high-dimensional space. We develop a new multidimensional index structure, called BLBTree. In BLBTree, instead of computing the distances in the high-dimensional original space to find the nearest neighbor, lot of candidates are to be rejected based on the lower bound distances: the research in BLBTree does not calculate the distances in the high-dimensional original space to find the nearest neighbor if the lower bounding distance exceeds the threshold, because the distances in the high-dimensional cannot be lower than the distances in the low-dimensional. We note that each object in BLBTree is described by a pyramid structure of histograms, with each histogram representing the lower resolution of the other histogram; due to this property, our index structure is well-suited for high-dimensional and large-scale problems. We have shown that our approach provides interesting and powerful experimental results.

HDIdx: High-dimensional indexing for efficient approximate nearest neighbor search

Fast Nearest Neighbor (NN) search is a fundamental challenge in large-scale data processing and analytics, particularly for analyzing multimedia contents which are often of high dimensionality. Instead of using exact NN search, extensive research efforts have been focusing on approximate NN search algorithms. In this work, we present “HDIdx”, an efficient high-dimensional indexing library for fast approximate NN search, which is open-source and written in Python. It offers a family of state-of-the-art algorithms that convert input high-dimensional vectors into compact binary codes, making them very efficient and scalable for NN search with very low space complexity.

Hashing-Based Scalable Remote Sensing Image Search and Retrieval in Large Archives

Large-scale remote sensing (RS) image search and retrieval have recently attracted great attention, due to the rapid evolution of satellite systems, that results in a sharp growing of image archives. An exhaustive search through linear scan from such archives is time demanding and not scalable in operational applications. To overcome such a problem, this paper introduces hashing-based approximate nearest neighbor search for fast and accurate image search and retrieval in large RS data archives. The hashing aims at mapping high-dimensional image feature vectors into compact binary hash codes, which are indexed into a hash table that enables real-time search and accurate retrieval. Such binary hash codes can also significantly reduce the amount of memory required for storing the RS images in the auxiliary archives. In particular, in this paper, we introduce in RS two kernel-based nonlinear hashing methods. The first hashing method defines hash functions in the kernel space by using only unlabeled images, while the second method leverages on the semantic similarity extracted by annotated images to describe much distinctive hash functions in the kernel space. The effectiveness of considered hashing methods is analyzed in terms of RS image retrieval accuracy and retrieval time. Experiments carried out on an archive of aerial images point out that the presented hashing methods are much faster, while keeping a similar (or even higher) retrieval accuracy, than those typically used in RS, which exploit an exact nearest neighbor search.

A novel supervised cluster adjustment method using a fast exact nearest neighbor search algorithm

Supervised clustering is a new research area that aims to improve unsupervised clustering algorithms exploiting supervised information. Today, there are several clustering algorithms, but the effective supervised cluster adjustment method which is able to adjust the resulting clusters, regardless of applied clustering algorithm has not been presented yet. In this paper, we propose a new supervised cluster adjustment method which can be applied to any clustering algorithm. Since the adjustment method is based on finding the nearest neighbors, a novel exact nearest neighbor search algorithm is also introduced which is significantly faster than the classic one. Several datasets and clustering evaluation metrics are employed to examine the effectiveness of the proposed cluster adjustment method and the proposed fast exact nearest neighbor algorithm comprehensively. The experimental results show that the proposed algorithms are significantly effective in improving clusters and accelerating nearest neighbor searches.

Effective optimizations of cluster-based nearest neighbor search in high-dimensional space

Nearest neighbor (NN) search in high-dimensional space plays a fundamental role in large-scale image retrieval. It seeks efficient indexing and search techniques, both of which are simultaneously essential for similarity search and semantic analysis. However, in recent years, there has been a rare breakthrough. Achievement of current techniques for NN search is far from satisfactory, especially for exact NN search. A recently proposed method, HB, addresses the exact NN search efficiently in high-dimensional space. It benefits from cluster-based techniques which can generate more compact representation of the data set than other techniques by exploiting interdimensional correlations. However, HB suffers from huge cost for lower bound computations and provides no further pruning scheme for points in candidate clusters. In this paper, we extend the HB method to address exact NN search in correlated, high-dimensional vector data sets extracted from large-scale image database by introducing two new pruning/selection techniques and we call it HB+. The first approach aims at selecting more quickly the subset of hyperplanes/clusters that must be considered. The second technique prunes irrelevant points in the selected subset of clusters. Performed experiments show the improvement of HB+ with respect to HB in terms of efficiency (I/O cost and CPU response time) and also demonstrate the superiority over other exact NN indexes.

SRS

Nearest neighbor searches in high-dimensional space have many important applications in domains such as data mining, and multimedia databases. The problem is challenging due to the phenomenon called "curse of dimensionality". An alternative solution is to consider algorithms that returns a c -approximate nearest neighbor ( c -ANN) with guaranteed probabilities. Locality Sensitive Hashing (LSH) is among the most widely adopted method, and it achieves high efficiency both in theory and practice. However, it is known to require an extremely high amount of space for indexing, hence limiting its scalability. In this paper, we propose several surprisingly simple methods to answer c -ANN queries with theoretical guarantees requiring only a single tiny index. Our methods are highly flexible and support a variety of functionalities, such as finding the exact nearest neighbor with any given probability. In the experiment, our methods demonstrate superior performance against the state-of-the-art LSH-based methods, and scale up well to 1 billion high-dimensional points on a single commodity PC.

Numerical Optimization of a Walk-on-Spheres Solver for the Linear Poisson-Boltzmann Equation

AbstractStochastic walk-on-spheres (WOS) algorithms for solving the linearized Poisson-Boltzmann equation (LPBE) provide several attractive features not available in traditional deterministic solvers: Gaussian error bars can be computed easily, the algorithm is readily parallelized and requires minimal memory and multiple solvent environments can be accounted for by reweighting trajectories. However, previously-reported computational times of these Monte Carlo methods were not competitive with existing deterministic numerical methods. The present paper demonstrates a series of numerical optimizations that collectively make the computational time of these Monte Carlo LPBE solvers competitive with deterministic methods. The optimization techniques used are to ensure that each atom’s contribution to the variance of the electrostatic solvation free energy is the same, to optimize the bias-generating parameters in the algorithm and to use an epsilon-approximate rather than exact nearest-neighbor search when determining the size of the next step in the Brownian motion when outside the molecule.

Bregman Hyperplane Trees for Fast Approximate Nearest Neighbor Search

This article presents a new approximate index structure, the Bregman hyperplane tree, for indexing the Bregman divergence, aiming to decrease the number of distance computations required at query processing time, by sacrificing some accuracy in the result. The experimental results on various high-dimensional data sets demonstrate that the proposed index structure performs comparably to the state-of-the-art Bregman ball tree in terms of search performance and result quality. Moreover, this method results in a speedup of well over an order of magnitude for index construction. The authors also apply their space partitioning principle to the Bregman ball tree and obtain a new index structure for exact nearest neighbor search that is faster to build and a slightly slower at query processing than the original.