Unnecessary Distance Computations Research Articles

Efficient Algorithm for K-Multiple-Means

K-Multiple-Means is an extension of K-means for the clustering of multiple means used in many applications, such as image segmentation, load balancing, and blind-source separation. Since K-means uses only one mean to represent each cluster, it fails to capture non-spherical cluster structures of data points. However, since K-Multiple-Means represents the cluster by computing multiple means and grouping them into specified c clusters, it can effectively capture the non-spherical clusters of the data points. To obtain the clusters, K-Multiple-Means updates a similarity matrix of a bipartite graph between the data points and the multiple means by iteratively computing the leading c singular vectors of the matrix. K-Multiple-Means, however, incurs a high computation cost for large-scale data due to the iterative SVD computations. Our proposal, F-KMM, increases the efficiency of K-Multiple-Means by computing the singular vectors from a smaller similarity matrix between the multiple means obtained from the similarity matrix of the bipartite graph. To compute the similarity matrix of the bipartite graph efficiently, we skip unnecessary distance computations and estimate lower bounding distances between the data points and the multiple means. Theoretically, the proposed approach guarantees the same clustering results as K-Multiple-Means since it can exactly compute the singular vectors from the similarity matrix between the multiple means. Experiments show that our approach is several orders of magnitude faster than previous clustering approaches that use multiple means.

Speeding up k-means clustering in high dimensions by pruning unnecessary distance computations

Standard k-means clustering necessitates computing pairwise Euclidean distances between each instance x in a data set D and all cluster centers, resulting in inadequate efficiency when dealing with high-dimensional data sets. Given its widespread usage, it is imperative that k-means clustering should be performed quickly to ensure efficient solutions. This paper is dedicated to exploring ways to improve the efficiency of the k-means algorithm in high-dimensional space. Unlike approximated approaches, our proposed method LBKC can achieve acceleration while yielding clustering results that are the same as what standard k-means clustering generates. LBKC utilizes the lower bound of Euclidean distance to safely avoid a large number of unnecessary distance calculations, thus achieving the goal of accelerating k-means process. Three carefully designed lower bounds based on the block vector, segment mean, and nonlinear embedding are presented in this paper, and they are employed in the proposed method. Furthermore, our approach LBKC is orthogonal to state-of-the-art methods, and we show how LBKC can be naturally combined with them to further improve their performance. Comprehensive experiments are conducted on a variety of data sets to evaluate the performance of the proposed approaches and related competitors, and the experimental results verify the effectiveness of our proposals.

On the approximation of Euclidean SL via geometric method

Single-linkage with distance-r stopping condition (SL) is a classical clustering technique, which can discover arbitrary shaped clusters. However, its O(n2) worst-case time complexity is still a challenge, where n is the number of points in the dataset. In this paper, we present a relaxed version of the problem in Euclidean space, called the ρ-approximate SL problem. We propose a grid-based algorithm, called ρ-SL, to solve this problem in O(n+ν) expected time, where ν is an integer associated with the maximum and minimum coordinates of each dimension. ρ-SL first partitions the dataset into non-empty grids and constructs a grid tree to organize them for efficient non-empty neighboring grid queries. Then, an initial cluster structure is established to keep track of all pairs of grids that may be mergeable. ρ-SL proceeds to identify the pairs of grids that are mergeable by a fast merging algorithm which takes into account the spatial relationships among points to reduce unnecessary distance calculations. Experiments were conducted on synthetic and real-world datasets to evaluate the performance of ρ-SL. The results show that the proposed algorithm is more efficient than SL and its fastest variants.

Fast Density-Based Clustering: Geometric Approach

DBSCAN is a fundamental density-based clustering algorithm with extensive applications. However, a bottleneck of DBSCAN is its O(n2) worst-case time complexity. In this paper, we propose an algorithm called GAP-DBC, which exploits the geometric relationships between points to solve this problem. GAP-DBC introduces an efficient partitioning algorithm to partition the data set with a limited number of range queries and then establishes an initial cluster structure based on the partition. GAP-DBC proceeds to iteratively refine the cluster structure by additional range queries. Finally, the cluster structure is accomplished using an iterative algorithm that utilizes the spatial relationships among points to reduce unnecessary distance calculations. We further demonstrate theoretically that GAP-DBC has an excellent guarantee in terms of computational efficiency. We conducted experiments on both synthetic and real-world data sets to evaluate the performance of GAP-DBC. The results show that our algorithm is competitive with other state-of-the-art algorithms.

GriT-DBSCAN: A spatial clustering algorithm for very large databases

DBSCAN is a fundamental spatial clustering algorithm with numerous practical applications. However, a bottleneck of DBSCAN is its O(n2) worst-case time complexity. To address this limitation, we propose a new grid-based algorithm for exact DBSCAN in Euclidean space called GriT-DBSCAN, which is based on the following two techniques. First, we introduce grid tree to organize the non-empty grids for the purpose of efficient non-empty neighboring grids queries. Second, by utilizing the spatial relationships among points, we propose a technique that iteratively prunes unnecessary distance calculations when determining whether the minimum distance between two sets is less than or equal to a certain threshold. We theoretically demonstrate that GriT-DBSCAN has excellent reliability in terms of time complexity. In addition, we obtain two variants of GriT-DBSCAN by incorporating heuristics, or by combining the second technique with an existing algorithm. Experiments are conducted on both synthetic and real-world data sets to evaluate the efficiency of GriT-DBSCAN and its variants. The results show that our algorithms outperform existing algorithms.

A Unified Approach to Spatial Proximity Query Processing in Dynamic Spatial Networks.

Nearest neighbor (NN) and range (RN) queries are basic query types in spatial databases. In this study, we refer to collections of NN and RN queries as spatial proximity (SP) queries. At peak times, location-based services (LBS) need to quickly process SP queries that arrive simultaneously. Timely processing can be achieved by increasing the number of LBS servers; however, this also increases service costs. Existing solutions evaluate SP queries sequentially; thus, such solutions involve unnecessary distance calculations. This study proposes a unified batch algorithm (UBA) that can effectively process SP queries in dynamic spatial networks. With the proposed UBA, the distance between two points is indicated by the travel time on the shortest path connecting them. The shortest travel time changes frequently depending on traffic conditions. The goal of the proposed UBA is to avoid unnecessary distance calculations for nearby SP queries. Thus, the UBA clusters nearby SP queries and exploits shared distance calculations for query clusters. Extensive evaluations using real-world roadmaps demonstrated the superiority and scalability of UBA compared with state-of-the-art sequential solutions.

A k-mismatch string matching for generalized edit distance using diagonal skipping method.

This paper proposes an approximate string matching with k-mismatches when calculating the generalized edit distance. When the edit distance is generalized, more sophisticated string matching can be provided. However, the execution time increases because of the bundle of complex computations for calculating complicated edit distances. The computational costs for finding which steps or edit distances are over k-mismatches cannot be significant in the generalized edit distance metric. Therefore, we can reduce the execution time by determining steps over k-mismatches and then skipping them. The diagonal step calculations using the pruning register skips unnecessary distance calculations over k-mismatches. The overhead of control statements and reordered memory accesses can be amortized by skipping multiple steps. Even though the proposed skipping method requires additional overhead, the proposed scheme's practical embodiments show that the execution time of string matching is reduced significantly when k is small.

Accelerated Single-Linkage algorithm using triangle inequality

Single-Linkage algorithm is a distance-based Hierarchical clustering method that can find arbitrary-shaped clusters but is most unsuitable for large datasets because of its high time complexity. The paper proposes an efficient accelerated technique for the algorithm with a merging threshold. It is a two-stage algorithm with the first one as an incremental pre-clustering step that uses the triangle inequality method to eliminate the unnecessary distance computations. The incremental approach makes it suitable for partial clustering of streaming data along with the collection. The second step using the property of the Single-Linkage algorithm itself takes a clustering decision without comparing all the patterns. This method shows how the neighbourhood between the input patterns can be used as a tool to accelerate the algorithm without hampering the cluster quality. Experiments are conducted with various standard and large real datasets and the result confirms its effectiveness for large datasets.

An Improved DBSCAN Algorithm Based on the Neighbor Similarity and Fast Nearest Neighbor Query

DBSCAN is the most famous density based clustering algorithm which is one of the main clustering paradigms. However, there are many redundant distance computations among the process of DBSCAN clustering, due to brute force Range-Query used to retrieve neighbors for each point in DBSCAN, which yields high complexity (O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> )) and low efficiency. Thus, it is unsuitable and not applicable for large scale data. In this paper, an improved DBSCAN based on neighbor similarity is proposed, which utilizes and Cover Tree to retrieve neighbors for each point in parallel, and the triangle inequality to filter many unnecessary distance computations. From the experiments conducted on large scale data sets, it is demonstrated that the proposed algorithm greatly speedup the original DBSCAN, and outperform the main improvements of DBSCAN. Comparing with ρ-approximate DBSCAN, which is the current fastest but approximate version of DBSCAN, the proposed algorithm has two advantages: one is faster and the other is that the result is accurate.

A Fast Algorithm for Identifying Density-Based Clustering Structures Using a Constraint Graph

OPTICS is a state-of-the-art algorithm for visualizing density-based clustering structures of multi-dimensional datasets. However, OPTICS requires iterative distance computations for all objects and is thus computed in O ( n 2 ) time, making it unsuitable for massive datasets. In this paper, we propose constrained OPTICS (C-OPTICS) to quickly create density-based clustering structures that are identical to those by OPTICS. C-OPTICS uses a bi-directional graph structure, which we refer to as the constraint graph, to reduce unnecessary distance computations of OPTICS. Thus, C-OPTICS achieves a good running time to create density-based clustering structures. Through experimental evaluations with synthetic and real datasets, C-OPTICS significantly improves the running time in comparison to existing algorithms, such as OPTICS, DeLi-Clu, and Speedy OPTICS (SOPTICS), and guarantees the quality of the density-based clustering structures.

A Hybrid MPI/OpenMP Parallelization of $K$ -Means Algorithms Accelerated Using the Triangle Inequality

The standard formulation of the K -means clustering (Lloyd's method) performs many unnecessary distance calculations. In this paper, we focus on four approaches that use the triangle inequality to avoid unnecessary distance calculations. These approaches are Drake's, Elkan's, Annulus, and Yinyang algorithms. We propose a hybrid MPI/OpenMP parallelization of these algorithms in which the dataset and the corresponding data structures storing bounds on distances are evenly divided among MPI processes. Then, in the assignment step of a K -means iteration, each MPI process computes the assignment of its portion of data using OpenMP threads. In the update step of the iteration, the cluster centroids are computed using a hierarchical all-reduce operation. In the computational experiments, we compared the strong scalability of these four algorithms with the scalability of Lloyd's algorithm, parallelized using the same approach. The results indicate that all four algorithms maintain an advantage in computing time over Lloyd's algorithm. A comparison with two software packages, whose sources are publicly available, in the same computing environment, shows that our implementations are more efficient.

Fast neighbor search by using revised k-d tree

We present two new neighbor query algorithms, including range query (RNN) and nearest neighbor (NN) query, based on revised k-d tree by using two techniques. The first technique is proposed for decreasing unnecessary distance computations by checking whether the cell of a node is inside or outside the specified neighborhood of query point, and the other is used to reduce redundant visiting nodes by saving the indices of descendant points. We also implement the proposed algorithms in Matlab and C. The Matlab version is to improve original RNN and NN which are based on k-d tree, C version is to improve k-Nearest neighbor query (kNN) which is based on buffer k-d tree. Theoretical and experimental analysis have shown that the proposed algorithms significantly improve the original RNN, NN and kNN in low dimension, respectively. The tradeoff is that the additional space cost of the revised k-d tree is approximately O(αnlog (n)).

Multi-Regional Delta-Tracking Method for Neutron Transport Tracking in Monte Carlo Criticality Calculation

The Monte Carlo method has been widely used as a standard method to perform neutron transport simulations in reactor physics. In conventional Monte Carlo codes corresponding to the neutron transport tracking with ray-tracing method, the distances to material boundaries must be computed frequently when the neutron changes its kinetic energy or moving into new material regions to determine the neutron transport length. However, if the neutron’s mean free path length, to some extent, is greater than the macro size of the model, a huge amount of distances need to be computed. As a result, the computational efficiency of the neutron transport tracking will be degraded. An improved multi-regional delta-tracking method based on domain decomposition was introduced to solve this problem, in which the original heterogeneous model would be decomposed into many sub-regions and each sub-region was tracked using a local delta-tracking method. Consequently, the computational efficiency of the neutron transport tracking can be improved theoretically without the unnecessary distance calculations. The improved multi-regional delta-tracking method was incorporated into the MOSRT system, which is a multi-objective modeling and simulation platform for radiation transport system. Finally, the method was validated using the criticality benchmarks and its accuracy and efficiency were demonstrated in Monte Carlo criticality calculation. The results indicated that the new method was consistent with the conventional methods, but with a more competitive run-time performance.

A fast clustering algorithm based on pruning unnecessary distance computations in DBSCAN for high-dimensional data

Clustering is an important technique to deal with large scale data which are explosively created in internet. Most data are high-dimensional with a lot of noise, which brings great challenges to retrieval, classification and understanding. No current existing approach is “optimal” for large scale data. For example, DBSCAN requires O(n2) time, Fast-DBSCAN only works well in 2 dimensions, and ρ-Approximate DBSCAN runs in O(n) expected time which needs dimension D to be a relative small constant for the linear running time to hold. However, we prove theoretically and experimentally that ρ-Approximate DBSCAN degenerates to an O(n2) algorithm in very high dimension such that 2D > > n. In this paper, we propose a novel local neighborhood searching technique, and apply it to improve DBSCAN, named as NQ-DBSCAN, such that a large number of unnecessary distance computations can be effectively reduced. Theoretical analysis and experimental results show that NQ-DBSCAN averagely runs in O(n*log(n)) with the help of indexing technique, and the best case is O(n) if proper parameters are used, which makes it suitable for many realtime data.

Doubly Approximate Nearest Neighbor Classification

Nonparametric classification models, such as K-Nearest Neighbor (KNN), have become particularly powerful tools in machine learning and data mining, due to their simplicity and flexibility. However, the testing time of the KNN classifier becomes unacceptable and the KNN's performance deteriorates significantly when applied to data sets with millions of dimensions. We observe that state-of-the-art approximate nearest neighbor (ANN) methods aim to either reduce the number of distance comparisons based on tree structure or decrease the cost of distance computation by dimension reduction methods. In this paper, we propose a doubly approximate nearest neighbor classification strategy, which marries the two branches which compress the dimensions for decreasing distance computation cost as well as reduce the number of distance comparison instead of full scan. Under this strategy, we build a compressed dimensional tree (CD-Tree) to avoid unnecessary distance calculations. In each decision node, we propose a novel feature selection paradigm by optimizing the feature selection vector as well as the separator (indicator variables for splitting instances) with the maximum margin. An efficient algorithm is then developed to find the globally optimal solution with convergence guarantee. Furthermore, we also provide a data-dependent generalization error bound for our model, which reveals a new insight for the design of ANN classification algorithms. Our empirical studies show that our algorithm consistently obtains competitive or better classification results on all data sets, yet we can also achieve three orders of magnitude faster than state-of-the-art libraries on very high dimensions.

Efficient search for top-k discords in streaming time series

The problem of anomaly detection in streaming time series has received much attention recently. The problem addresses finding the most anomalous subsequence (discord) over a time-series stream, which might arrive at high speed. The fact that finding top-k discords is more useful than finding the most unusual subsequence since users might make a choice among the top-k discords instead of choosing only one. Hence, an efficient method of search for top-k discords in streaming time series is proposed in the paper. The method uses a lower bound threshold, a lower bounding technique on a common dimensionality reduction transform, and a state-of-the-art technique of the distance computation between two time-series subsequences to prune off unnecessary distance calculations. The three techniques are arranged in a cascading fashion to speed up the performance of the method. Furthermore, the proposed method can return a set of top-k discords on the fly. The experimental results show that the proposed method can acquire quality discords nearly identical to those obtained by HOT SAX, a well-known method of anomaly detection. Remarkably, our proposed method demonstrates a fast response in handling time-series streams at high speed.

A fast DBSCAN clustering algorithm by accelerating neighbor searching using Groups method

Density based clustering methods are proposed for clustering spatial databases with noise. Density Based Spatial Clustering of Applications with Noise (DBSCAN) can discover clusters of arbitrary shape and also handles outliers effectively. DBSCAN obtains clusters by finding the number of points within the specified distance from a given point. It involves computing distances from given point to all other points in the dataset. The conventional index based methods construct a hierarchical structure over the dataset to speed-up the neighbor search operations. The hierarchical index-structures fail to scale for datasets of dimensionality above 20. In this paper, we propose a novel graph-based index structure method Groups that accelerates the neighbor search operations and also scalable for high dimensional datasets. Experimental results show that the proposed method improves the speed of DBSCAN by a factor of about 1.5–2.2 on benchmark datasets. The performance of DBSCAN degrades considerably with noise due to unnecessary distance computations introduced by noise points while the proposed method is robust to noise by pruning out noise points early and eliminating the unnecessary distance computations. The cluster results produced by our method are exactly similar to that of DBSCAN but executed at a much faster pace.

Triangular inequality-based rotation-invariant boundary image matching for smart devices

Nowadays there are many efforts to develop image matching applications exploiting a large number of images stored in smart devices such as smartphones, smart pads, and smart cameras. Boundary image matching converts boundary images to time-series and identifies similar boundary images using time-series matching on those time-series. In boundary image matching, computing the rotation-invariant distance between image time-series is a very time-consuming process since it requires a lot of Euclidean distance computations for all possible rotations. To support the boundary image matching in smart devices, we need to devise a simple but fast computation mechanism for rotation-invariant distances. For this purpose, in this paper we propose a novel rotation-invariant matching solution that significantly reduces the number of distance computations using the triangular inequality. To this end, we first present the notion of self-rotation distance and formally show that the self-rotation distance with the triangular inequality produces a tight lower bound and prunes many unnecessary distance computations. Using the self-rotation distance, we then propose a triangular inequality-based solution to rotation-invariant image matching. We next present the concept of k-self rotation distance as a generalized version of the self-rotation distance and formally show that this $$k$$ k -self rotation distance produces a tighter lower bound and prunes more unnecessary distance computations. Using the $$k$$ k -self rotation distance we also propose an advanced triangular inequality-based solution to rotation-invariant image matching. Experimental results show that our self-rotation distance-based algorithms significantly outperform the existing algorithms by up to one or two orders of magnitude, and we believe that this performance improvement makes our algorithms very suitable for smart devices.

Improving DBSCAN’s execution time by using a pruning technique on bit vectors

Clustering is the process of assigning a set of physical or abstract objects into previously unknown groups. The goal of clustering is to group similar objects into the same clusters and dissimilar objects into different clusters. Similarities between objects are evaluated by using the attribute values of objects. There are many clustering algorithms in the literature; among them, DBSCAN is a well known density-based clustering algorithm. We improve DBSCAN’s execution time performance for binary data sets and Hamming distances. We achieve considerable speed gains by using a novel pruning technique, as well as bit vectors, and binary operations. Our novel method effectively discards distant neighbors of an object and computes only the distances between an object and its possible neighbors. By discarding distant neighbors, we avoid unnecessary distance computations and use less CPU time when compared with the conventional DBSCAN algorithm. However, the accuracy of our method is identical to that of the original DBSCAN. Experimental test results on real and synthetic data sets demonstrate that, by using our pruning technique, we obtain considerably faster execution time results compared to DBSCAN.

New Efficient Strategy to Accelerate k-Means Clustering Algorithm

One of the most popular clustering techniques is the k-means clustering algorithm. However, the utilization of the k-means is severely limited by its high computational complexity. In this study, we propose a new strategy to accelerate the k-means clustering algorithm through the Partial Distance (PD) logic. The proposed strategy avoids many unnecessary distance calculations by applying efficient PD strategy. Experiments show the efficiency of the proposed strategy when applied to different data sets.