High-dimensional Search Space Research Articles

Distance Encoded Product Quantization for Approximate K-Nearest Neighbor Search in High-Dimensional Space.

Approximate K-nearest neighbor search is a fundamental problem in computer science. The problem is especially important for high-dimensional and large-scale data. Recently, many techniques encoding high-dimensional data to compact codes have been proposed. The product quantization and its variations that encode the cluster index in each subspace have been shown to provide impressive accuracy. In this paper, we explore a simple question: is it best to use all the bit-budget for encoding a cluster index? We have found that as data points are located farther away from the cluster centers, the error of estimated distance becomes larger. To address this issue, we propose a novel compact code representation that encodes both the cluster index and quantized distance between a point and its cluster center in each subspace by distributing the bit-budget. We also propose two distance estimators tailored to our representation. We further extend our method to encode global residual distances in the original space. We have evaluated our proposed methods on benchmarks consisting of GIST, VLAD, and CNN features. Our extensive experiments show that the proposed methods significantly and consistently improve the search accuracy over other tested techniques. This result is achieved mainly because our methods accurately estimate distances.

Application of an Efficient Gradient-Based Optimization Strategy for Aircraft Wing Structures

In this paper, a practical optimization framework and enhanced strategy within an industrial setting are proposed for solving large-scale structural optimization problems in aerospace. The goal is to eliminate the difficulties associated with optimization problems, which are mostly nonlinear with numerous mixed continuous-discrete design variables. Particular emphasis is placed on generating good initial starting points for the search process and in finding a feasible optimum solution or improving the chances of finding a better optimum solution when traditional techniques and methods have failed. The efficiency and reliability of the proposed strategy were demonstrated through the weight optimization of different metallic and composite laminated wingbox structures. The results show the effectiveness of the proposed procedures in finding an optimized solution for high-dimensional search space cases with a given level of accuracy and reasonable computational resources and user efforts. Conclusions are also inferred with regards to the sensitivity of the optimization results obtained with respect to the choice of different starting values for the design variables, as well as different optimization algorithms in the optimization process.

A highly accurate differential evolution–particle swarm optimization algorithm for the construction of initial value problem solvers

ABSTRACTIn this work a new evolutionary computation technique is introduced for the construction of initial value solvers based on Runge–Kutta (RK) pairs. The derivation of RK pairs corresponds to solving a nonlinear optimization problem with a multimodal objective function in a high dimensional search space; additional difficulty stems from the fact that only solutions with accuracy at least equal to machine epsilon are acceptable. The proposed approach involves hybridizing a Differential Evolution (DE) strategy with elements from Particle Swarm Optimization (PSO) in order to produce a method for solving optimization problems with high accuracy. The resulting methodology is applied to two different problems of RK pair derivation of orders 5 and 4 and compared with standard DE techniques. Numerical experiments show that the proposed hybrid DE-PSO satisfies the strict accuracy requirements imposed by the particular problem, while outperforming its rivals.

PyDREAM: high-dimensional parameter inference for biological models in python.

SummaryBiological models contain many parameters whose values are difficult to measure directly via experimentation and therefore require calibration against experimental data. Markov chain Monte Carlo (MCMC) methods are suitable to estimate multivariate posterior model parameter distributions, but these methods may exhibit slow or premature convergence in high-dimensional search spaces. Here, we present PyDREAM, a Python implementation of the (Multiple-Try) Differential Evolution Adaptive Metropolis [DREAM(ZS)] algorithm developed by Vrugt and ter Braak (2008) and Laloy and Vrugt (2012). PyDREAM achieves excellent performance for complex, parameter-rich models and takes full advantage of distributed computing resources, facilitating parameter inference and uncertainty estimation of CPU-intensive biological models.Availability and implementationPyDREAM is freely available under the GNU GPLv3 license from the Lopez lab GitHub repository at http://github.com/LoLab-VU/PyDREAM.Supplementary information Supplementary data are available at Bioinformatics online.

A two stage grading approach for feature selection and classification of microarray data using Pareto based feature ranking techniques: A case study

High dimensional search space in microarray data with large number of genes and few dozen of samples increases the complexity of analysis of such databases. All the genes are not significant and hence informative genes are required to be extracted. So dimension reduction is necessary for this process. It is often found in literature that the ranking approaches are used for feature selection. Different ranking techniques may assign different rank to the same gene and the selection made based on these ranks may not be suitable for different problems. So use of one ranking technique may lead to rejection of some important genes and possibly selection of some insignificant genes. Such selection may degrade the performance of the classifier. To overcome this problem, here abi-objective ranked based Pareto front technique is proposed. In this technique using two ranked based technique the Pareto optimal solution is generated with a set of features. For the experimental work, 21 models based on 7 feature ranking strategies are considered. Eight different microarray data are taken to find the suitable ranking combination for the work. A grading method is used to rank the models and statistical test is performed to validate the findings.

Query-aware locality-sensitive hashing scheme for $$l_p$$ norm

The problem of c-Approximate Nearest Neighbor (c-ANN) search in high-dimensional space is fundamentally important in many applications, such as image database and data mining. Locality-Sensitive Hashing (LSH) and its variants are the well-known indexing schemes to tackle the c-ANN search problem. Traditionally, LSH functions are constructed in a query-oblivious manner, in the sense that buckets are partitioned before any query arrives. However, objects closer to a query may be partitioned into different buckets, which is undesirable. Due to the use of query-oblivious bucket partition, the state-of-the-art LSH schemes for external memory, namely C2LSH and LSB-Forest, only work with approximation ratio of integer $$c \ge 2$$cź2. In this paper, we introduce a novel concept of query-aware bucket partition which uses a given query as the "anchor" for bucket partition. Accordingly, a query-aware LSH function under a specific $$l_p$$lp norm with $$p \in (0,2]$$pź(0,2] is a random projection coupled with query-aware bucket partition, which removes random shift required by traditional query-oblivious LSH functions. The query-aware bucket partitioning strategy can be easily implemented so that query performance is guaranteed. For each $$l_p$$lp norm $$(p \in (0,2])$$(pź(0,2]), based on the corresponding p-stable distribution, we propose a novel LSH scheme named query-aware LSH (QALSH) for c-ANN search over external memory. Our theoretical studies show that QALSH enjoys a guarantee on query quality. The use of query-aware LSH function enables QALSH to work with any approximation ratio $$c > 1$$c>1. In addition, we propose a heuristic variant named QALSH$$^+$$+ to improve the scalability of QALSH. Extensive experiments show that QALSH and QALSH$$^+$$+ outperform the state-of-the-art schemes, especially in high-dimensional space. Specifically, by using a ratio $$c < 2$$c<2, QALSH can achieve much better query quality.

A Generic Method for Accelerating LSH-Based Similarity Join Processing

Locality sensitive hashing (LSH) is an efficient method for solving the problem of approximate similarity search in high-dimensional spaces. Through LSH, a high-dimensional similarity join can be processed in the same way as hash join, making the cost of joining two large datasets linear. By judicially analyzing the properties of multiple LSH algorithms, we propose a generic method to speed up the process of joining two large datasets using LSH. The crux of our method lies in the way which we identify a set of representative points to reduce the number of LSH lookups. Theoretical analyzes show that our proposed method can greatly reduce the number of lookup operations and retain the same result accuracy compared to executing LSH lookups for every query point. Furthermore, we demonstrate the generality of our method by showing that the same principle can be applied to LSH algorithms for three different metrics: the Euclidean distance (QALSH), Jaccard similarity measure (MinHash), and Hamming distance (sequence hashing). Results from experimental studies using real datasets confirm our error analyzes and show significant improvements of our method over the state-of-the-art LSH method: to achieve over 0.95 recall, we only need to operate LSH lookups for at most 15 percent of the query points.

Memory Vectors for Similarity Search in High-Dimensional Spaces

We study an indexing architecture to store and search in a database of high-dimensional vectors from the perspective of statistical signal processing and decision theory. This architecture is composed of several memory units, each of which summarizes a fraction of the database by a single representative vector. The potential similarity of the query to one of the vectors stored in the memory unit is gauged by a simple correlation with the memory unit's representative vector. This representative optimizes the test of the following hypothesis: the query is independent from any vector in the memory unit versus the query is a simple perturbation of one of the stored vectors. Compared to exhaustive search, our approach finds the most similar database vectors significantly faster without a noticeable reduction in search quality. Interestingly, the reduction of complexity is provably better in high-dimensional spaces. We empirically demonstrate its practical interest in a large-scale image search scenario with off-the-shelf state-of-the-art descriptors.

Distributed high-dimensional similarity search approach for large-scale wireless sensor networks

Similarity search in high-dimensional space has become increasingly important in many wireless sensor network applications. However, existing approaches to similarity search is based on the premise that sensed data are centralized to deal with, or sensed data are simple enough to be stored in a relational database. Different from the previous work, we propose a distributed approximate similarity search algorithm to retrieve similar high-dimensional sensed data for query in wireless sensor networks. First, the sensors are divided into several clusters using the distributed clustering method. Furthermore, the sink transmits the compressed hash code set to the cluster heads. Finally, the estimated similarity score is compared with a specified threshold to filter out irrelevant sensed data. Therefore, the higher search precision and energy efficiency can be achieved. Extensive simulation results show that the proposed algorithms provide significant performance gains in terms of precision and energy efficiency compared with the existing algorithms.

Tree-based indexing for real-time ConvNet landmark-based visual place recognition

Recent impressive studies on using ConvNet landmarks for visual place recognition take an approach that involves three steps: (a) detection of landmarks, (b) description of the landmarks by ConvNet features using a convolutional neural network, and (c) matching of the landmarks in the current view with those in the database views. Such an approach has been shown to achieve the state-of-the-art accuracy even under significant viewpoint and environmental changes. However, the computational burden in step (c) significantly prevents this approach from being applied in practice, due to the complexity of linear search in high-dimensional space of the ConvNet features. In this article, we propose two simple and efficient search methods to tackle this issue. Both methods are built upon tree-based indexing. Given a set of ConvNet features of a query image, the first method directly searches the features’ approximate nearest neighbors in a tree structure that is constructed from ConvNet features of database images. The database images are voted on by features in the query image, according to a lookup table which maps each ConvNet feature to its corresponding database image. The database image with the highest vote is considered the solution. Our second method uses a coarse-to-fine procedure: the coarse step uses the first method to coarsely find the top- N database images, and the fine step performs a linear search in Hamming space of the hash codes of the ConvNet features to determine the best match. Experimental results demonstrate that our methods achieve real-time search performance on five data sets with different sizes and various conditions. Most notably, by achieving an average search time of 0.035 seconds/query, our second method improves the matching efficiency by the three orders of magnitude over a linear search baseline on a database with 20,688 images, with negligible loss in place recognition accuracy.

A genetic algorithm with multi-parent crossover using quaternion representation for numerical function optimization

Finding optimal solutions of a numerical function of more than one independent variable is an important problem with many practical applications including process control systems, data fitting, and engineering designs. Over the last few decades, techniques for solving unconstrained optimization problems have been proposed. Evolutionary Algorithms have emerged as one of the most popular selections for tackling these problems, among which Genetic Algorithms (GAs) are widely used in practice. In recent literature on GAs, a Genetic Algorithm with multi-parent crossover (GA-MPC) was found to be superior over other algorithms. Nevertheless, the GA-MPC still has some difficulties when dealing with separable test issues and convergence to global optima in the high-dimensional search space. Meanwhile, quaternions, which are an extension of complex numbers, can allow algorithms to expand the search space to avoid getting stuck in the local optima. Therefore, this study aims to employ quaternions for representing individuals in the GA-MPC to enhance the effectiveness of the GA-MPC. Experimental results for ten benchmark functions indicated that the GA-MPC using the quaternion representation of individuals improved the quality of solutions compared with the original GA-MPC.

BLBTree: an Efficient Index Structure for Fast Search

Due of its interest in different areas, similarity search in high-dimensional spaces is one of the principal research axes today for CBIR Systems. The properties for high-dimensional data demand some adequate search methods to have a research in optimal time. Due to the curse of dimensionality, search time in the index structure exponentially increases, according the dimension of the descriptor. However, the performances of classical index structures become less than the sequential scan of data in search time when answering exact nearest neighbor queries. To overcome this problematic, two major approaches for high-dimensional data in CBIR systems have been proposed: The first approach permits to speed up the search by using multiple levels of lower bound distances, the second approach exploits the BTree index structure to speed up the search. We propose to combine both techniques to search for large nearest neighbors in a high-dimensional space. We develop a new multidimensional index structure, called BLBTree. In BLBTree, instead of computing the distances in the high-dimensional original space to find the nearest neighbor, lot of candidates are to be rejected based on the lower bound distances: the research in BLBTree does not calculate the distances in the high-dimensional original space to find the nearest neighbor if the lower bounding distance exceeds the threshold, because the distances in the high-dimensional cannot be lower than the distances in the low-dimensional. We note that each object in BLBTree is described by a pyramid structure of histograms, with each histogram representing the lower resolution of the other histogram; due to this property, our index structure is well-suited for high-dimensional and large-scale problems. We have shown that our approach provides interesting and powerful experimental results.

Iterative optimization of tool path planning in 5-axis flank milling of ruled surfaces by integrating sampling techniques

Simultaneous adjustment of all cutter locations in a tool path using meta-heuristic algorithms provides a systematic approach to reducing machining errors in 5-axis flank machining of ruled surfaces. However, these algorithms experienced unsatisfactory quality of optimal solutions and lengthy search time in high-dimensional search space. To solve these problems, we propose an iterative optimization scheme that progressively simplifies solution space by sampling techniques. Akaike information criterion (AIC) is used to screen significant factors from sampling data. An electromagnetism-like mechanism (EM) algorithm searches through a simplified solution space constructed only using these factors. An iteration process consisting of such sampling, screening, and searching steps repeats several times until final optimal solutions are obtained. Test results of representative surfaces validate the effectiveness of the proposed scheme. Both solution quality and search efficiency are improved comparing to those produced by previous studies. This work enhances the practicality of optimization-driven tool path planning in 5-axis flank machining.

User-customized brain computer interfaces using Bayesian optimization

Objective. The brain characteristics of different people are not the same. Brain computer interfaces (BCIs) should thus be customized for each individual person. In motor-imagery based synchronous BCIs, a number of parameters (referred to as hyper-parameters) including the EEG frequency bands, the channels and the time intervals from which the features are extracted should be pre-determined based on each subject’s brain characteristics. Approach. To determine the hyper-parameter values, previous work has relied on manual or semi-automatic methods that are not applicable to high-dimensional search spaces. In this paper, we propose a fully automatic, scalable and computationally inexpensive algorithm that uses Bayesian optimization to tune these hyper-parameters. We then build different classifiers trained on the sets of hyper-parameter values proposed by the Bayesian optimization. A final classifier aggregates the results of the different classifiers. Main Results. We have applied our method to 21 subjects from three BCI competition datasets. We have conducted rigorous statistical tests, and have shown the positive impact of hyper-parameter optimization in improving the accuracy of BCIs. Furthermore, We have compared our results to those reported in the literature. Significance. Unlike the best reported results in the literature, which are based on more sophisticated feature extraction and classification methods, and rely on prestudies to determine the hyper-parameter values, our method has the advantage of being fully automated, uses less sophisticated feature extraction and classification methods, and yields similar or superior results compared to the best performing designs in the literature.

Materialized Cube Selection Using Particle Swarm Optimization Algorithm

Storage of pre-computed views in data warehouse can essentially reduce query processing cost for decision support queries. The problem is to choose an optimal set of materialized views. Various frameworks such as lattice, MVPP and AND-OR graphs and algorithms like heuristic based, greedy, stochastic algorithm have been proposed in the literature for materialized view selection. Heuristic and greedy algorithms become slower in high dimensional search space while stochastic algorithms do not guarantee global optimal solution but reach to the optimum most solution in a fast and efficient way. In this paper we have implemented Particle Swarm Optimization (PSO) algorithm, one of the stochastic algorithm, on lattice framework to select an optimal set of views for materialization in data warehouse by minimizing query processing cost. We have compared our results with Genetic algorithm to prove the effectiveness of PSO algorithm over genetic algorithm.

Effective and Efficient Algorithms for Flexible Aggregate Similarity Search in High Dimensional Spaces

Numerous applications in different fields, such as spatial databases, multimedia databases, data mining, and recommender systems, may benefit from efficient and effective aggregate similarity search, also known as aggregate nearest neighbor (AggNN) search. Given a group of query objects $Q$ , the goal of AggNN is to retrieve the $k$ most similar objects from the database, where the underlying similarity measure is defined as an aggregation (usually sum or max ) of the distances between the retrieved objects and every query object in $Q$ . Recently, the problem was generalized so as to retrieve the $k$ objects which are most similar to a fixed proportion of the elements of $Q$ . This variant of aggregate similarity search is referred to as “flexible AggNN”, or FANN. In this work, we propose two approximation algorithms, one for the sum variant of FANN, and the other for the max variant. Extensive experiments are provided showing that, relative to state-of-the-art approaches (both exact and approximate), our algorithms produce query results with good accuracy, while at the same time being very efficient.

Dynamical analysis of Grover’s search algorithm in arbitrarily high-dimensional search spaces

We discuss at length the dynamical behavior of Grover's search algorithm for which all the Walsh---Hadamard transformations contained in this algorithm are exposed to their respective random perturbations inducing the augmentation of the dimension of the search space. We give the concise and general mathematical formulations for approximately characterizing the maximum success probabilities of finding a unique desired state in a large unsorted database and their corresponding numbers of Grover iterations, which are applicable to the search spaces of arbitrary dimension and are used to answer a salient open problem posed by Grover (Phys Rev Lett 80:4329---4332, 1998).

Enabling efficient approximate nearest neighbor search for outsourced database in cloud computing

Approximate nearest neighbor (ANN) search in high-dimensional space has been studied extensively in recent years. However, it supports only ANN search over plaintext in traditional locality-sensitive hashing (LSH). How to perform ANN search over encrypted data becomes a new challenging task. In this paper, we make an attempt to formally address the problem. We propose a new secure and efficient ANN search scheme over encrypted data based on SortingKeys-LSH (SK-LSH) and mutable order-preserving encryption (mOPE). In our construction, a secure index is generated by incorporating SK-LSH with mOPE, which can simultaneously achieve efficient ANN search and ensure data confidentiality. Furthermore, the proposed solution can achieve efficient range query on encrypted data. Security analysis demonstrates that our construction can achieve the desired security properties.

Markerless human motion tracking using hierarchical multi-swarm cooperative particle swarm optimization.

The high-dimensional search space involved in markerless full-body articulated human motion tracking from multiple-views video sequences has led to a number of solutions based on metaheuristics, the most recent form of which is Particle Swarm Optimization (PSO). However, the classical PSO suffers from premature convergence and it is trapped easily into local optima, significantly affecting the tracking accuracy. To overcome these drawbacks, we have developed a method for the problem based on Hierarchical Multi-Swarm Cooperative Particle Swarm Optimization (H-MCPSO). The tracking problem is formulated as a non-linear 34-dimensional function optimization problem where the fitness function quantifies the difference between the observed image and a projection of the model configuration. Both the silhouette and edge likelihoods are used in the fitness function. Experiments using Brown and HumanEva-II dataset demonstrated that H-MCPSO performance is better than two leading alternative approaches—Annealed Particle Filter (APF) and Hierarchical Particle Swarm Optimization (HPSO). Further, the proposed tracking method is capable of automatic initialization and self-recovery from temporary tracking failures. Comprehensive experimental results are presented to support the claims.

Evolving agent-based models using self-adaptive complexification

This paper focuses on parameter search for multi-agent based models using evolutionary algorithms. Large numbers and variable dimensions of parameters require an optimization method which can efficiently handle a high dimensional search space. We are proposing the use of complexification as it emulates the natural way of evolution by starting with a small constrained search space and expanding it as the evolution progresses. To further improve performance we suggest and experiment with methods of self-adaptation to enable the algorithm to adjust its parameters individually to the problem at hand. We examined the effects of these methods on an EA by evolving parameters for two multi-agent based models.