Approximate Nearest Neighbor Research Articles

Product quantization with dual codebooks for approximate nearest neighbor search

Product quantization (PQ) is a powerful technique for approximate nearest neighbor (ANN) search. In this paper, to improve the accuracy of ANN search, we propose a new PQ-based method named product quantization with dual codebooks (DCPQ). Different from traditional PQ-based methods, we analyze quantization errors after learning the first PQ codebook, and then part of training vectors with larger quantization errors are found and selected to relearn a second PQ codebook. When encoding the database offline, all database vectors are firstly quantized using both of dual codebooks in each subspace, and the encoding mode of a database vector is determined after comparing the two quantization errors based on dual codebooks. Moreover, database vectors with the same encoding mode are grouped as a sub-database and can be more efficiently searched. Experimental results demonstrate that our proposed dual codebooks solution can achieve higher accuracy compared with the standard PQ and its variants.

Computational Geometry Column 70

In this column, we review the most recent results on processing persistence diagrams as purely geometric objects. (Yes, this is not a typo! While persistence diagrams originally come from computational topology, this column will mainly focus on the geometric parts and the minimal amount of knowledge on computational topology is needed to read this column.) We mainly focus on computing a center persistence diagram under the bottleneck and Wasserstein distances, and also the approximate nearest neighbor query under the bottleneck distance. At the end, we will go over a few open problems as well as some direction for future research, e.g., (approximate) farthest neighbor query under the bottleneck distance.

Minimal residual ordinal loss hashing with an adaptive optimization mechanism

The binary coding technique has been widely used in approximate nearest neighbors (ANN) search tasks. Traditional hashing algorithms treat binary bits equally, which usually causes an ambiguous ranking. To solve this issue, we propose an innovative bitwise weight method dubbed minimal residual ordinal loss hashing (MROLH). Different from a two-step mechanism, MROLH simultaneously learns binary codes and bitwise weights by a feedback mechanism. When the algorithm converges, the binary codes and bitwise weights can be well adaptive to each other. Furthermore, we establish the ordinal relation preserving constraint based on quartic samples to enhance the power of preserving relative similarity. To decrease the training complexity, we utilize a tensor ordinal graph to represent quartic ordinal relation, and the original objective function is approximated by the one based on triplet samples. In this paper, we also assign different weight values to training samples. During the training procedure, the weight of each data is initialized to the same value, and we iteratively boost the weight of the data whose relative similarity is not well preserved. As a result, we can minimize the residual ordinal loss. Experimental results on three large-scale ANN search benchmark datasets, i.e., SIFT1M, GIST1M, and Cifar10, show that the proposed method MROLH achieves a superior ANN search performance in both the Hamming space and the weighted Hamming space over the sate-of-the-art approaches.

Deep hashing for multi-label image retrieval: a survey

Content-based image retrieval (CBIR) aims to display, as a result of a search, images with the same visual contents as a query. This problem has attracted increasing attention in the area of computer vision. Learning-based hashing techniques are amongst the most studied search approaches for approximate nearest neighbors in large-scale image retrieval. With the advance of deep neural networks in image representation, hashing methods for CBIR have started using deep learning to build binary codes. Such strategies are generally known as deep hashing techniques. In this paper, we present a comprehensive deep hashing survey for the task of image retrieval with multiple labels, categorizing the methods according to how the input images are treated: pointwise, pairwise, tripletwise and listwise, as well as their relationships. In addition, we present discussions regarding the cost of space, efficiency and search quality of the described models, as well as open issues and future work opportunities.

Re-ranking via local embeddings: A use case with permutation-based indexing and the nSimplex projection

Approximate Nearest Neighbor (ANN) search is a prevalent paradigm for searching intrinsically high dimensional objects in large-scale data sets. Recently, the permutation-based approach for ANN has attracted a lot of interest due to its versatility in being used in the more general class of metric spaces. In this approach, the entire database is ranked by a permutation distance to the query. Typically, permutations allow the efficient selection of a candidate set of results, but typically to achieve high recall or precision this set has to be reviewed using the original metric and data. This can lead to a sizeable percentage of the database being recalled, along with many expensive distance calculations.To reduce the number of metric computations and the number of database elements accessed, we propose here a re-ranking based on a local embedding using the nSimplex projection. The nSimplex projection produces Euclidean vectors from objects in metric spaces which possess the n-point property. The mapping is obtained from the distances to a set of reference objects, and the original metric can be lower bounded and upper bounded by the Euclidean distance of objects sharing the same set of references.Our approach is particularly advantageous for extensive databases or expensive metric function. We reuse the distances computed in the permutations in the first stage, and hence the memory footprint of the index is not increased.An extensive experimental evaluation of our approach is presented, demonstrating excellent results even on a set of hundreds of millions of objects.

Soft biometrics‐based face image retrieval using improved grey wolf optimisation

Humans often use faces to recognise and identify individuals. Face recognition is one of the important tasks carried out by forensic examiners manually during their investigation, when there is an evidence image/video available from a crime scene. There is a growing demand for face recognition from unconstrained images, which is valuable for criminal investigators in identifying the victims. When an input face image is given to the proposed system, it filters from large scale face dataset to find the top-k similar faces. Deep convolutional neural network approach is employed to extract important features present in the input face image and improved grey wolf optimisation approach is proposed to select optimal features from the extracted features. It is then preceded by a soft biometric-based face matcher that helps in retrieving exact face image from the top-k similar faces matched using approximate nearest neighbour. The performance of the proposed system is evaluated using LFW, CASIA, Multi-pie and Color-Feret datasets. The proposed system addresses the challenge of searching face images from a large collection of unconstrained images by incorporating feature retrieval using DCNN and IGWO with soft biometric face matcher in cascaded framework which improves accuracy and reduces computation and retrieval time.

Towards Large-Scale Object Instance Search: A Multi-Block N-Ary Trie

Object instance search is a challenging task with a wide range of applications, but the fast search with high accuracy has not been well solved yet. In this paper, we investigate the object instance search from a new perspective in terms of joint precision and computational cost optimization, and propose a novel index structure i.e., Multi-Block N-ary Trie (MBNT) to accelerate the exact r-neighbor search in the Hamming space. Comprehensive studies are first carried out to reveal the performance of exact and approximate nearest neighbor (NN) algorithms for object instance search. An interesting finding that the exact search is more promising for very compact binary codes (e.g., 64-bit and 128-bit) is analyzed. Along this vein, we introduce a Trie structure, i.e., MBNT, which is specifically designed for improving the exact NN search performance in the context of large-scale object instance search. To index the binary codes, a subset of continuous bits of a binary string, denoted as a block, is regarded as an atomic indexing element. As such, the problem of lookup misses can be addressed. Theoretical analyses are also provided to show that our MBNT scheme can incur less computational cost than other hash table-based methods. Extensive experimental results on the 100M dataset have demonstrated that our method achieves faster search speed while maintaining the promising search precision towards large-scale object instance search.

Efficient plagiarism detection for software modeling assignments

ABSTRACT Background and Context Reports suggest plagiarism is a common occurrence in universities. While plagiarism detection mechanisms exist for textual artifacts, this is less so for non-code related ones such as software design artifacts like models, metamodels or model transformations. Objective To provide an efficient mechanism for the detection of plagiarism in repositories of Model-Driven Engineering (MDE) assignments. Method Our approach is based on the adaptation of the Locality Sensitive Hashing, an approximate nearest neighbor search mechanism, to the modeling technical space. We evaluate our approach on a real use case consisting of two repositories containing 10 years of student answers to MDE course assignments. Findings We have found that: effectively, plagiarism occurred on the aforementioned course assignments our tool was able to efficiently detect them. Implications Plagiarism detection must be integrated into the toolset and activities of MDE instructors in order to correctly evaluate students

Top Position Sensitive Ordinal Relation Preserving Bitwise Weight for Image Retrieval

In recent years, binary coding methods have become increasingly popular for tasks of searching approximate nearest neighbors (ANNs). High-dimensional data can be quantized into binary codes to give an efficient similarity approximation via a Hamming distance. However, most of existing schemes consider the importance of each binary bit as the same and treat training samples at different positions equally, which causes many data pairs to share the same Hamming distance and a larger retrieval loss at the top position. To handle these problems, we propose a novel method dubbed by the top-position-sensitive ordinal-relation-preserving bitwise weight (TORBW) method. The core idea is to penalize data points without preserving an ordinal relation at the top position of a ranking list more than those at the bottom and assign different weight values to their binary bits according to the distribution of query data. Specifically, we design an iterative optimization mechanism to simultaneously learn binary codes and bitwise weights, which makes their learning processes related to each other. When the iterative procedure converges, the binary codes and bitwise weights are effectively adapted to each other. To reduce the training complexity, we relax the discrete constraints of both the binary codes and the indicator function. Furthermore, we pretrain a tensor ordinal graph to decrease the time consumption of computing a relative similarity relationship among data points. Experimental results on three large-scale ANN search benchmark datasets, i.e., SIFT1M, GIST1M, and Cifar10, show that the proposed TORBW method can achieve superior performance over state-of-the-art approaches.

IPDH: An Improved Pairwise-Based Deep Hashing Method for Large-Scale Image Retrieval

Hashing technique has been extensively utilized in approximate nearest neighbor (ANN) search for large-scale image retrieval by virtue of its storage simplicity and computational efficiency. Recently, many researches show that hashing methods based. on deep neural networks (DNNs) can improve retrieval accuracy by simultaneously learning both deep feature representation and hashing functions in an end-to-end framework. Most deep supervised hashing methods aim to preserve the distance or similarity between data points using the similarity relationships constructed based on semantic labels of images, while ignoring the classification ability of the generated hash codes. However, the semantic labels themselves carry more information than the corresponding similarity labels. We propose an Improved Pairwise-based Deep Hashing (IPDH) method to generate hash codes with powerful classification ability by exploring the global distribution of semantic labels. Specifically, the proposed IPDH method aims to minimize the information loss generated during the process of classification prediction to ensure that the output predicted labels of the network model has a similar distribution with those from the original semantic labels. Comprehensive experiments show that the proposed IPDH method can obtain better improvement than other state-of-the-art algorithms.

Wind turbine tower detection using feature descriptors and deep learning

Wind Turbine Towers (WTTs) are the main structures of wind farms. They are costly devices that must be thoroughly inspected according to maintenance plans. Today, existence of machine vision techniques along with unmanned aerial vehicles (UAVs) enable fast, easy, and intelligent visual inspection of the structures. Our work is aimed towards developing a vision-based system to perform Nondestructive tests (NDTs) for wind turbines using UAVs. In order to navigate the flying machine toward the wind turbine tower and reliably land on it, the exact position of the wind turbine and its tower must be detected. We employ several strong computer vision approaches such as Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Features from Accelerated Segment Test (FAST), Brute-Force, Fast Library for Approximate Nearest Neighbors (FLANN) to detect the WTT. Then, in order to increase the reliability of the system, we apply the ResNet, MobileNet, ShuffleNet, EffNet, and SqueezeNet pre-trained classifiers in order to verify whether a detected object is indeed a turbine tower or not. This intelligent monitoring system has auto navigation ability and can be used for future goals including intelligent fault diagnosis and maintenance purposes. The simulation results show the accuracy of the proposed model are 89.4% in WTT detection and 97.74% in verification (classification) problems.

Distributed Discrete Hashing by Equivalent Continuous Formulation

Hashing based approximate nearest neighbor search has attracted considerable attention in various fields. Most of the existing hashing methods are centralized, which cannot be used for many large-scale applications with the data stored or collected in a distributed manner. In this article, we consider the distributed hashing problem. The main difficulty of hashing is brought by its inherent binary constraints, which makes the problem generally NP-hard. Most of the existing distributed hashing methods chose to relax the problem by dropping the binary constraints. However, such a manner will bring additional quantization error, which makes the binary codes less effective. In this paper, we propose a novel distributed discrete hashing method, which learns effective hash codes without using any relaxations. Specifically, we give a method to transform the discrete hashing problem into an equivalent distributed continuous optimization problem. After transformation, we devise a distributed discrete hashing (dDH) algorithm based on the idea of DC programming to solve the problem. To obtain more efficient hash codes, we further add bits balance and uncorrelation constraints to the hashing problem, and we also propose a distributed constrained discrete hashing algorithm (dCDH) to solve this problem. Extensive experiments are provided to show the superiority of the proposed methods.

Semi-Paired Asymmetric Deep Cross-Modal Hashing Learning

Hashing has been effectively applied in large-scale multimedia retrieval tasks due to the characteristics of fast calculation speed and low storage cost. However, most existing cross-view hashing methods require well paired views, where one sample in one view can always be associated with one sample in another view. Nevertheless, such fully-paired setting is difficult to hold in practice. The cross-view retrieval task for semi-paired data, in which only a portion of the sample is paired, remains more challenging and less explored. In this case, we design an unsupervised deep hashing for large-scale data, named semi-paired asymmetric deep cross-model hashing (SADCH) to address the challenging task. SADCH is a novel asymmetric end-to-end deep neural network model. Specifically, SADCH trains deep network by using query points to improve the training efficiency, and directly learns hash codes of database. A similarity matrix is constructed by a novel cross-view graph to explore the underlying data structures, such that the similarities of paired points and unpaired points can be preserved. As such, the deep features and similarity graph matrix can be jointly used to design an alternating algorithm to efficiently generate more differentiated hash codes. SADCH is evaluated for large-scale cross-view approximate nearest neighbor search on three benchmark datasets and compared against several state-of-the arts. The results of extensive experiments demonstrate the superiority of the proposed SADCH for semi-paired hashing in the unsupervised setting.

Attention-Aware Joint Location Constraint Hashing for Multi-Label Image Retrieval

Learning based hashing has been widely used in approximate nearest neighbor search for image retrieval. However, most of the existing hashing methods are designed to learn only simplex feature similarity while ignored the location similarity among multiple objects, thus cannot work well on multi-label image retrieval tasks. In this paper, we propose a novel supervised hashing method which fusions the two kinds of similarities together. First, we leverage an adjacency matrix to record the relative location relationship among multiple objects. Second, by incorporating matrix discretization difference and image label difference, we re-define the pairwise image similarity in a more meticulous way. Third, to learn more distinguishable hash codes, we leverage an attention sub-network to identify the approximate regions of the objects in an image so that the extracted features can mainly focus on the foreground objects and ignore the background clutter. The loss function in our method consists of a multi-categories classification loss which is used to learn the attention sub-network and a hash loss with a scaled sigmoid function which is used to learn the efficient hash codes. Experiment results show that our proposed method is effective in preserving high-level similarities and outperforms the baseline methods in multi-label image retrieval.

Unsupervised Ensemble Hashing: Boosting Minimum Hamming Distance

Hashing aims at learning discriminative binary codes of high-dimensional data for the approximate nearest neighbor searching. However, the distance ranking obtained by traditional methods is not optimum in the Hamming space, and it degrades the performance for retrieval tasks. To tackle the above problem, an unsupervised ensemble hashing is proposed to improve the ranking accuracy by assembling the diverse hash tables independently in this paper. We observe that the higher the accuracy is the larger diversity the base learner has, and the more effective the ensemble method is. Based on this principle, two special ensembles hashing approaches are proposed to increase diversity by bootstrap sampling with data-dependent methods. Especially, the results are better when the minimum Hamming distance is large and the variance of the Hamming distance is small. This proposed method is conducted in the experiments and the results show that it can achieve about 10%-25% performance compared with the baseline algorithm, which achieves competitive results with the state-of-the-art methods on the CIFAR-10 and LabelMe benchmarks.

PM-LSH

Nearest neighbor (NN) search in high-dimensional spaces is inherently computationally expensive due to the curse of dimensionality. As a well-known solution to approximate NN search, locality-sensitive hashing (LSH) is able to answer c-approximate NN ( c -ANN) queries in sublinear time with constant probability. Existing LSH methods focus mainly on building hash bucket based indexing such that the candidate points can be retrieved quickly. However, existing coarse-grained structures fail to offer accurate distance estimation for candidate points, which translates into additional computational overhead when having to examine unnecessary points. This in turn reduces the performance of query processing. In contrast, we propose a fast and accurate LSH framework, called PM-LSH, that aims to compute the c -ANN query on large- scale, high-dimensional datasets. First, we adopt a simple yet effective PM-tree to index the data points. Second, we develop a tunable confidence interval to achieve accurate distance estimation and guarantee high result quality. Third, we propose an efficient algorithm on top of the PM-tree to improve the performance of computing c -ANN queries. Extensive experiments with real-world data offer evidence that PM-LSH is capable of outperforming existing proposals with respect to both efficiency and accuracy.

Accumulative Errors Optimization for Visual Odometry of ORB-SLAM2 Based on RGB-D Cameras

Oriented feature from the accelerated segment test (oFAST) and rotated binary robust independent elementary features (rBRIEF) SLAM2 (ORB-SLAM2) represent a recognized complete visual simultaneous location and mapping (SLAM) framework with visual odometry as one of its core components. Given the accumulated error problem with RGB-Depth ORB-SLAM2 visual odometry, which causes a loss of camera tracking and trajectory drift, we created and implemented an improved visual odometry method to optimize the cumulative error. First, this paper proposes an adaptive threshold oFAST algorithm to extract feature points from images and rBRIEF is used to describe the feature points. Then, the fast library for approximate nearest neighbors strategy is used for image rough matching, the results of which are optimized by progressive sample consensus. The image matching precision is further improved by using an epipolar line constraint based on the essential matrix. Finally, the efficient Perspective-n-Point method is used to estimate the camera pose and a least-squares optimization problem is constructed to adjust the estimated value to obtain the final camera pose. The experimental results show that the proposed method has better robustness, higher image matching accuracy and more accurate determination of the camera motion trajectory.

CoverBLIP: accelerated and scalable iterative matched-filtering for magnetic resonance fingerprint reconstruction**Preliminary results related to this work were appeared as an IEEE MLSP conference paper [] and an ISMRM abstract [

Current popular methods for magnetic resonance fingerprint (MRF) recovery are bottlenecked by the heavy computations of a matched-filtering step due to the growing size and complexity of the fingerprint dictionaries in multi-parametric quantitative MRI applications. We address this shortcoming by arranging dictionary atoms in the form of cover tree structures and adopt the corresponding fast approximate nearest neighbour searches to accelerate matched-filtering. For datasets belonging to smooth low-dimensional manifolds cover trees offer search complexities logarithmic in terms of data population. With this motivation we propose an iterative reconstruction algorithm, named CoverBLIP, to address large-size MRF problems where the fingerprint dictionary i.e. discrete manifold of Bloch responses, encodes several intrinsic NMR parameters. We study different forms of convergence for this algorithm and we show that provided with a notion of embedding, the inexact and non-convex iterations of CoverBLIP linearly convergence toward a near-global solution with the same order of accuracy as using exact brute-force searches. Our further examinations on both synthetic and real-world datasets and using different sampling strategies, indicates between 2–3 orders of magnitude reduction in total search computations. Cover trees are robust against the curse-of-dimensionality and therefore CoverBLIP provides a notion of scalability—a consistent gain in time-accuracy performance—for searching high-dimensional atoms which may not be easily preprocessed (i.e. for dimensionality reduction) due to the increasing degrees of non-linearities appearing in the emerging multi-parametric MRF dictionaries.

KDFC-ART: a KD-tree approach to enhancing Fixed-size-Candidate-set Adaptive Random Testing

Adaptive random testing (ART) was developed as an enhanced version of random testing to increase the effectiveness of detecting failures in programs by spreading the test cases evenly over the input space. However, heavy computation may be incurred. In this paper, three enhanced algorithms for fixed-size-candidate-set ART (FSCS-ART) are proposed based on the $k$ -dimensional tree (KD-tree) structure. The first algorithm Naive-KDFC constructs a KD-tree by splitting the input space with respect to every dimension successively in a round-robin fashion. The second algorithm SemiBal-KDFC improves the balance of the KD-tree by prioritizing the splitting according to the spread in each dimension. In order to control the number of traversed nodes in backtracking, the third algorithm LimBal-KDFC introduces an upper bound for the nodes involved. Simulation and empirical studies have been conducted to investigate the efficiency and effectiveness of the three algorithms. The experimental results show that these algorithms significantly reduce the computation time of the original FSCS-ART for low dimensions and for the case of high dimensions with low failure rates. The efficiency of SemiBal-KDFC is better than that of Naive-KDFC when the dimension is no more than 8, but LimBal-KDFC is the most efficient of all three. Although the limited backtracking leads only to an approximate nearest neighbor in LimBal-KDFC, its failure-detection effectiveness is, in fact, better than FSCS-ART in high-dimensional input spaces and has no significant deterioration in low-dimensional spaces.

Deep Progressive Hashing for Image Retrieval

Hashing is a widely adopted method based on an approximate nearest neighbor search and is used in large-scale image retrieval tasks. Conventional learning-based hashing algorithms employ end-to-end representation learning, which is a one-off technique. Because of the tradeoff between efficiency and performance, conventional learning-based hashing methods must sacrifice code length to improve performance, which increases their computational complexity. To improve the efficiency of binary codes, motivated by the “nonsalient-to-salient” attention scheme of humans, we propose a recursive hashing mechanism that maps progressively expanded salient regions to a series of binary codes. These salient regions are generated by a conventional saliency model based on bottom-up saliency-driven attention and a semantic-guided saliency model based on top-down task-driven attention. After obtaining a series of salient regions, we perform long-range temporal modeling of salient regions using a graph-based recurrent deep network to obtain more refined representative features. The later output nodes inherit aggregated information from all previous nodes and extract discriminative features from more salient regions. Therefore, this network possesses more significant information and satisfactory scalability. The proposed recursive hashing neural network, optimized by a triplet ranking loss, is end-to-end trainable. Extensive experimental results from several image retrieval benchmarks show the scalability of our method and demonstrate its strong performance compared with state-of-the-art methods.