Kd-tree Data Structure Research Articles

TransientX: A high-performance single-pulse search package

Context. Radio interferometers composed of a large array of small antennas possess large fields of view, coupled with high sensitivities. For example, the Karoo Array Telescope (MeerKAT) achieves a gain of up to 2.8 KJy−1 across its >1 deg2 field of view. This capability significantly enhances the survey speed for pulsars and fast transients. It also introduces challenges related to the high data rate, which reaches a few Tb s−1 for MeerKAT, and it requires substantial computing power. Aims. To handle the high data rate of surveys, we have developed a high-performance single-pulse search software called “TransientX”. This software integrates multiple processes into one pipeline, which includes radio-frequency interference mitigation, dedispersion, matched filtering, clustering, and candidate plotting. Methods. In TRANSIENTX, we developed an efficient CPU-based dedispersion implementation using the sub-band dedispersion algorithm. Additionally, TRANSIENTX employs the density-based spatial clustering of applications with noise (DBSCAN) algorithm to eliminate duplicate candidates, using an efficient implementation based on the kd-tree data structure. We also calculate the decrease of signal-to-noise ratio (s/N) resulting from dispersion measure, boxcar width, spectral index, and pulse-shape mismatches. Remarkably, we find that the decrease of S/N resulting from the mismatch between a boxcar-shaped template and a Gaussian-shaped pulse with scattering remains relatively small, at approximately 9%, even when the scattering timescale is ten times that of the pulse width. Additionally, the decrease in the S/N resulting from the spectral index mismatch becomes significant with multi-octave receivers. Results. We have benchmarked the individual processes, including dedispersion, matched filtering, and clustering. Our dedispersion implementation can be executed in real time using a single CPU core on data with 4096 dispersion measure trials, which consist of 4096 channels and have a time resolution of 153 µs. Overall, TRANSIENTX offers the capability for efficient CPU-only real-time single-pulse searching.

MCMSTStream: applying minimum spanning tree to KD-tree-based micro-clusters to define arbitrary-shaped clusters in streaming data

Stream clustering has emerged as a vital area for processing streaming data in real-time, facilitating the extraction of meaningful information. While efficient approaches for defining and updating clusters based on similarity criteria have been proposed, outliers and noisy data within stream clustering areas pose a significant threat to the overall performance of clustering algorithms. Moreover, the limitation of existing methods in generating non-spherical clusters underscores the need for improved clustering quality. As a new methodology, we propose a new stream clustering approach, MCMSTStream, to overcome the abovementioned challenges. The algorithm applies MST to micro-clusters defined by using the KD-Tree data structure to define macro-clusters. MCMSTStream is robust against outliers and noisy data and has the ability to define clusters with arbitrary shapes. Furthermore, the proposed algorithm exhibits notable speed and can handling high-dimensional data. ARI and Purity indices are used to prove the clustering success of the MCMSTStream. The evaluation results reveal the superior performance of MCMSTStream compared to state-of-the-art stream clustering algorithms such as DenStream, DBSTREAM, and KD-AR Stream. The proposed method obtained a Purity value of 0.9780 and an ARI value of 0.7509, the highest scores for the KDD dataset. In the other 11 datasets, it obtained much higher results than its competitors. As a result, the proposed method is an effective stream clustering algorithm on datasets with outliers, high-dimensional, and arbitrary-shaped clusters. In addition, its runtime performance is also quite reasonable.

Robust LiDAR-Based Vehicle Detection for On-Road Autonomous Driving

The stable detection and tracking of high-speed vehicles on the road by using LiDAR can input accurate information for the decision-making module and improve the driving safety of smart cars. This paper proposed a novel LiDAR-based robust vehicle detection method including three parts: point cloud clustering, bounding box fitting and point cloud recognition. Firstly, aiming at the problem of clustering quality degradation caused by the uneven distribution of LiDAR point clouds and the difference in clustering radius between point cloud clusters in traditional DBSCAN (TDBSCAN) obstacle clustering algorithms, an improved DBSCAN algorithm based on distance-adaptive clustering radius (ADBSCAN) is designed, and a point cloud KD-Tree data structure is constructed to speed up the traversal of the algorithm; meanwhile, the OPTICS algorithm is introduced to enhance the performance of the proposed algorithm. Then, by adopting different fitting strategies for vehicle contour points in various states, the adaptability of the bounding box fitting algorithm is improved; Moreover, in view of the shortcomings of the poor robustness of the L-shape algorithm, the principal component analysis method (PCA) is introduced to obtain stable bounding box fitting results. Finally, considering the time-consuming and low-accuracy training of traditional machine learning algorithms, advanced PointNet in deep learning technique is built to send the point cloud within the bounding box of a high-confidence vehicle into PointNet to complete vehicle recognition. Experiments based on our autonomous driving perception platform and the KITTI dataset prove that the proposed method can stably complete vehicle target recognition and achieve a good balance between time-consuming and accuracy.

Subgraph Query Matching in Multi-Graphs Based on Node Embedding

This paper presents an efficient algorithm for matching subgraph queries in a multi-graph based on features-based indexing techniques. The KD-tree data structure represents these nodes’ features, while the set-trie index data structure represents the multi-edges to make queries effectively. The vertex core number, triangle number, and vertex degree are the eight features’ main features. The densest vertex in the query graph is extracted based on these main features. The proposed model consists of two phases. The first phase’s main idea is that, for the densest extracted vertex in the query graph, find the density similar neighborhood structure in the data graph. Then find the k-nearest neighborhood query to obtain the densest subgraph. The second phase for each layer graph, mapping the vertex to feature vector (Vertex Embedding), improves the proposed model. To reduce the node-embedding size to be efficient with the KD-tree, indexing a dimension reduction, the principal component analysis (PCA) method is used. Furthermore, symmetry-breaking conditions will remove the redundancy in the generated pattern matching with the query graph. In both phases, the filtering process is applied to minimize the number of candidate data nodes of the initiate query vertex. The filtering process is applied to minimize the number of candidate data nodes of the initiate query vertex. Finally, testing the effect of the concatenation of the structural features (orbits features) with the meta-features (summary of general, statistical, information-theoretic, etc.) for signatures of nodes on the model performance. The proposed model is tested over three real benchmarks, multi-graph datasets, and two randomly generated multi-graph datasets. The results agree with the theoretical study in both random cliques and Erdos random graph. The experiments showed that the time efficiency and the scalability results of the proposed model are acceptable.

A Continuous Region-Based Skyline Computation for a Group of Mobile Users

Skyline queries, which are based on the concept of Pareto dominance, filter the objects from a potentially large multi-dimensional collection of objects by keeping the best, most favoured objects in satisfying the user′s preferences. With today′s advancement of technology, ad hoc meetings or impromptu gatherings involving a group of people are becoming more and more common. Intuitively, deciding on an optimal meeting point is not a straightforward task especially when conflicting criteria are involved and the number of criteria to be considered is vast. Moreover, a point that is near to a user might not meet all the various users′ preferences, while a point that meets most of the users′ preferences might be located far away from these users. The task becomes more complicated when these users are on the move. In this paper, we present the Region-based Skyline for a Group of Mobile Users (RSGMU) method, which aims to resolve the problem of continuously finding the optimal meeting points, herein called skyline objects, for a group of users while they are on the move. RSGMU assumes a centroid-based movement where users are assumed to be moving towards a centroid that is identified based on the current locations of each user in the group. Meanwhile, to limit the searching space in identifying the objects of interest, a search region is constructed. However, the changes in the users′ locations caused the search region of the group to be reconstructed. Unlike the existing methods that require users to frequently report their latest locations, RSGMU utilises a dynamic motion formula, which abides to the laws of classical physics that are fundamentally symmetrical with respect to time, in order to predict the locations of the users at a specified time interval. As a result, the skyline objects are continuously updated, and the ideal meeting points can be decided upon ahead of time. Hence, the users′ locations as well as the spatial and non-spatial attributes of the objects are used as the skyline evaluation criteria. Meanwhile, to avoid re-computation of skylines at each time interval, the objects of interest within a Single Minimum Bounding Rectangle that is formed based on the current search region are organized in a Kd-tree data structure. Several experiments have been conducted and the results show that our proposed method outperforms the previous work with respect to CPU time.

Efficient Local Refinement near Parametric Boundaries Using kd-Tree Data Structure and Algebraic Level Sets

In analysis of problems with parametric spline boundaries that are immersed or inserted into an underlying domain, the discretization on the underlying domain usually does not conform to the inserted boundaries. While the fixed underlying discretization is of great convenience as the immersed boundaries evolve, the field approximations near the inserted boundaries require refinement in the underlying domain, as do the quadrature cells. In this paper, a kd-tree data structure together with a sign-based and/or distance-based refinement strategy is proposed for local refinement near the inserted boundaries as well as for adaptive quadrature near the boundaries. The developed algorithms construct and utilize implicit forms of parametric Non-Uniform Rational B-Spline (NURBS) surfaces to algebraically (and non-iteratively) estimate distance as well as sign relative to the inserted boundary. The kd-tree local refinement is demonstrated to produce fewer sub-cells for the same accuracy of solution as compared to the classical quad/oct tree-based subdivision. Consistent with the kd-tree data structure, we describe a new a priori refinement algorithm based on the signed and unsigned distance from the inserted boundary. We first demonstrate the local refinement strategy coupled with the the kd-tree data structure by constructing Truncated Hierarchical B-spline (THB-spline) “meshes”. We next demonstrate the accuracy and efficiency of the developed local refinement strategy through adaptive quadrature near NURBS boundaries inserted within volumetric three-dimensional NURBS discretizations.

Secure and Practical Group Nearest Neighbor Query for Location-Based Services in Cloud Computing

Group nearest neighbor (GNN) query enables a group of location-based service (LBS) users to retrieve a point from point of interests (POIs) with the minimum aggregate distance to them. For resource constraints and privacy concerns, LBS provider outsources the encrypted POIs to a powerful cloud server. The encryption-and-outsourcing mechanism brings a challenge for the data utilization. However, as previous work from k − anonymity technique leaks all contents of POIs and returns an answer set with redundant communication cost, the LBS system cannot work properly with those privacy-preserving schemes. In this paper, we illustrate a secure group nearest neighbor query scheme, which is referred to as SecGNN. It supports the GNN query with n n ≥ 3 LBS users and assures the data privacy and query privacy. Since SecGNN only achieves linear search complexity, an efficiency enhanced scheme (named Sec GNN + ) is introduced by taking advantage of the KD-tree data structure. Specifically, we convert the GNN problem to the nearest neighbor problem for their centroid, which can be computed by anonymous veto network and Burmester–Desmedt conference key agreement protocols. Furthermore, the Sec GNN + scheme is introduced from the KD-tree data structure and a designed tool, which supports the computation of inner products over ciphertexts. Finally, we run experiments on a real-database and a random database to evaluate the performance of our SecGNN and Sec GNN + schemes. The experimental results show the high efficiency of our proposed schemes.

An Improved Ray-Tracing Algorithm for SBR-Based EM Scattering Computation of Electrically Large Targets

We put forward an improved ray-tracing algorithm to obtain a fast prediction of the radar cross section (RCS) for electrically large complex targets using the shooting and bouncing ray (SBR) method. Our algorithm employs a rope-based kd-tree data structure together with the neighbor-facet-tree based backward ray-tracing technique. The proposed algorithm shows excellent performance in the evaluation of scattering properties of the electrically large targets at high frequency. To assess and validate the proposed method, we compute the RCS of electrically small targets and of a realistic aircraft model and compare results with those of the traditional SBR method and multilevel fast multipole method. The agreement of the results together with the runtime performance demonstrate the accuracy and efficiency of our method.

An Effective Algorithm of Uneven Road Surface Modeling and Calculating Reaction Forces for a Vehicle Dynamics Simulation

Computer simulations of vehicle dynamics become more complex when a vehicle movement takes place on the uneven road surface. In such a case two problems must be solved. The first one concerns a way of road surface modeling, and the second one a way of precisely determining a place of interaction of reaction forces of the road on the vehicle wheels. In this paper triangular irregular networks (TIN) surface was used for modeling surface unevenness, and the author’s algorithm based on the efficient kd-tree data structure was developed for determining a place of an application road surface reaction forces. For calculating the reaction forces including rolling resistance force the Pacejka Magic Formula tire model was used. The solution presented in the paper is computing efficient and for this reason it can be used in the in real-time simulations not only for a vehicle dynamics but for any objects moving over an uneven surface road.

Improved Mean Shift Algorithm for Maximizing Clustering Accuracy

Clustering is a machine learning method that can group similar data points. Mean Shift (MS) is a fixed window-based clustering algorithm, which calculates the number of clusters automatically but cannot guarantee the convergence of the algorithm. The main drawback of the Mean Shift Algorithm is that the algorithm requires to set a stopping criterion (threshold point) otherwise all clusters move towards one cluster and fixed bandwidth is used here. It cannot define the upper bound of iteration numbers and need to set the iteration numbers. This paper proposed a new Mean Shift Algorithm, called Improved Mean Shift (IMS) algorithm, which overcomes the all defined pitfalls of Mean Shift Algorithm. The IMS process KD-tree data structure was used to sort the dataset and all data points as initial cluster centroids without a random selection of initial centroids. In each iteration, it shifts the variable bandwidth sliding window to the actual data point nearest to the mean using k-nearest neighbours (kNN) algorithm and finds the number of clusters automatically. Also, this paper handles the missing values using Mean Imputation (MI). The IMS algorithm produces better results than the Mean Shift Algorithm on both synthetic and real datasets.

Tree-Based Algorithm for Stable and Efficient Data Clustering

The K-means algorithm is a well-known and widely used clustering algorithm due to its simplicity and convergence properties. However, one of the drawbacks of the algorithm is its instability. This paper presents improvements to the K-means algorithm using a K-dimensional tree (Kd-tree) data structure. The proposed Kd-tree is utilized as a data structure to enhance the choice of initial centers of the clusters and to reduce the number of the nearest neighbor searches required by the algorithm. The developed framework also includes an efficient center insertion technique leading to an incremental operation that overcomes the instability problem of the K-means algorithm. The results of the proposed algorithm were compared with those obtained from the K-means algorithm, K-medoids, and K-means++ in an experiment using six different datasets. The results demonstrated that the proposed algorithm provides superior and more stable clustering solutions.

A comprehensive approach to determining minimum cutter lengths for five-axis milling

Interference detection and avoidance by the shortest cutter for the five-axis milling machining is a critical task. Short tool length increases the rigidity and chatter stability of the cutter. In this research, a new and efficient method of interference detection and avoidance by the shortest cutter is developed. For the specific five-axis machine configuration, first possible candidate parts for the collision are found, which are complete cutting system (spindle, tool holder, and cutter), the work in process model, and the fixture. Spindle, tool holder, and tool are represented by the solid geometry identity of the cylinder, truncated cone, and cylinder, respectively, with the length and diameter as parameters. The surfaces of the work in process model and the fixture model are represented as the point cloud data of the suitable density. The Kd-tree data structure is employed on point cloud data which gives an efficient searching of the potential candidate points for the interference detection with the complete cutting system. All existing methods are able to detect the collision, but they are not capable to remove it with the optimum cutter length. The proposed algorithm has not only the capability of collision detection; it can also remove the collision with the optimum tool length. Other scope of the proposed algorithm is the selection of the tool holder to minimize the overhang tool length.

Analisis Kinerja Struktur Data Kd-Tree Pada Metode K-Nearest Neighbors

K-Nearest Neighbors is a commonly used classification technique that can be categorized into instance-based classification method. The performance of KNN is mostly determined by the size of the training data. This research compared and analyzed KD-Tree and Array data structures on KNN implementation. Dataset used in this research has large multidimensional features. From the experiment conducted we can conclude that KD-Tree data structure has better and relatively stable performance compared to Array data structure.

Multi-Class Simultaneous Adaptive Segmentation and Quality Control of Point Cloud Data

3D modeling of a given site is an important activity for a wide range of applications including urban planning, as-built mapping of industrial sites, heritage documentation, military simulation, and outdoor/indoor analysis of airflow. Point clouds, which could be either derived from passive or active imaging systems, are an important source for 3D modeling. Such point clouds need to undergo a sequence of data processing steps to derive the necessary information for the 3D modeling process. Segmentation is usually the first step in the data processing chain. This paper presents a region-growing multi-class simultaneous segmentation procedure, where planar, pole-like, and rough regions are identified while considering the internal characteristics (i.e., local point density/spacing and noise level) of the point cloud in question. The segmentation starts with point cloud organization into a kd-tree data structure and characterization process to estimate the local point density/spacing. Then, proceeding from randomly-distributed seed points, a set of seed regions is derived through distance-based region growing, which is followed by modeling of such seed regions into planar and pole-like features. Starting from optimally-selected seed regions, planar and pole-like features are then segmented. The paper also introduces a list of hypothesized artifacts/problems that might take place during the region-growing process. Finally, a quality control process is devised to detect, quantify, and mitigate instances of partially/fully misclassified planar and pole-like features. Experimental results from airborne and terrestrial laser scanning as well as image-based point clouds are presented to illustrate the performance of the proposed segmentation and quality control framework.

An efficient interpolation technique for jump proposals in reversible-jump Markov chain Monte Carlo calculations.

Selection among alternative theoretical models given an observed dataset is an important challenge in many areas of physics and astronomy. Reversible-jump Markov chain Monte Carlo (RJMCMC) is an extremely powerful technique for performing Bayesian model selection, but it suffers from a fundamental difficulty and it requires jumps between model parameter spaces, but cannot efficiently explore both parameter spaces at once. Thus, a naive jump between parameter spaces is unlikely to be accepted in the Markov chain Monte Carlo (MCMC) algorithm and convergence is correspondingly slow. Here, we demonstrate an interpolation technique that uses samples from single-model MCMCs to propose intermodel jumps from an approximation to the single-model posterior of the target parameter space. The interpolation technique, based on a kD-tree data structure, is adaptive and efficient in modest dimensionality. We show that our technique leads to improved convergence over naive jumps in an RJMCMC, and compare it to other proposals in the literature to improve the convergence of RJMCMCs. We also demonstrate the use of the same interpolation technique as a way to construct efficient ‘global’ proposal distributions for single-model MCMCs without prior knowledge of the structure of the posterior distribution, and discuss improvements that permit the method to be used in higher dimensional spaces efficiently.

Robust L∞ convex pose-graph optimisation for monocular localisation solution for unmanned aerial vehicles

In this study, a robust L∞ convex pose-graph optimisation solution for unmanned aerial vehicles (UAVs) monocular motion estimation with loop closing is presented. Most solutions proposed in literature formulate the pose-graph optimisation as a least-squares problem by minimising a cost function using iterative methods such as Gauss–Newton or Levenberg–Marquardt algorithms. However, with these algorithms, there is no guarantee to converge to a global minimum as they, with high probabilities, converge to a local minimum or even to an infeasible solution. The solution we propose in this work uses a new robust convex optimisation pose-graph technique, which efficiently corrects the UAV’s pose after loop-closure detections. Uncertainty estimation using derivative method and its propagation through multiview geometry algorithms are included in the developed solution. The detection of the visual loop closures, in appearance-based navigation, is achieved with our innovative, fast and efficient method based on Bayes decision theory with Gaussian mixture model in combination with the KD-tree data structure. Our navigation solution has been validated using real-world data in both indoor and outdoor environments acquired by a UAV equipped with a monocular system.

Nearest neighbor imputation using spatial–temporal correlations in wireless sensor networks

Missing data is common in Wireless Sensor Networks (WSNs), especially with multi-hop communications. There are many reasons for this phenomenon, such as unstable wireless communications, synchronization issues, and unreliable sensors. Unfortunately, missing data creates a number of problems for WSNs. First, since most sensor nodes in the network are battery-powered, it is too expensive to have the nodes re-transmit missing data across the network. Data re-transmission may also cause time delays when detecting abnormal changes in an environment. Furthermore, localized reasoning techniques on sensor nodes (such as machine learning algorithms to classify states of the environment) are generally not robust enough to handle missing data. Since sensor data collected by a WSN is generally correlated in time and space, we illustrate how replacing missing sensor values with spatially and temporally correlated sensor values can significantly improve the network’s performance. However, our studies show that it is important to determine which nodes are spatially and temporally correlated with each other. Simple techniques based on Euclidean distance are not sufficient for complex environmental deployments. Thus, we have developed a novel Nearest Neighbor (NN) imputation method that estimates missing data in WSNs by learning spatial and temporal correlations between sensor nodes. To improve the search time, we utilize a kd-tree data structure, which is a non-parametric, data-driven binary search tree. Instead of using traditional mean and variance of each dimension for kd-tree construction, and Euclidean distance for kd-tree search, we use weighted variances and weighted Euclidean distances based on measured percentages of missing data. We have evaluated this approach through experiments on sensor data from a volcano dataset collected by a network of Crossbow motes, as well as experiments using sensor data from a highway traffic monitoring application. Our experimental results show that our proposed K-NN imputation method has a competitive accuracy with state-of-the-art Expectation–Maximization (EM) techniques, while using much simpler computational techniques, thus making it suitable for use in resource-constrained WSNs.

A fast and robust bulk-loading algorithm for indexing very large digital elevation datasets II. Experimental results

The spatial indexing of eventually all the available topographic information of Earth is a highly valuable tool for different geoscientific application domains. The Shuttle Radar Topography Mission (SRTM) collected and made available to the public one of the world's largest digital elevation models (DEMs). With the aim of providing on easier and faster access to these data by improving their further analysis and processing, we have indexed the SRTM DEM by means of a spatial index based on the kd-tree data structure, called the Q-tree. This paper is the second in a two-part series that includes a thorough performance analysis to validate the bulk-load algorithm efficiency of the Q-tree. We investigate performance measuring elapsed time in different contexts, analyzing disk space usage, testing response time with typical queries, and validating the final index structure balance. In addition, the paper includes performance comparisons with Oracle 11g that helps to understand the real cost of our proposal. Our tests prove that the proposed algorithm outperforms Oracle 11g using around a 9% of the elapsed time, taking six times less storage with more than 96% of page utilization, and getting faster response times to spatial queries issued on 4.5 million points. In addition to this, the behavior of the spatial index has been successfully tested on both an open GIS (VT Builder) and a visualizer tool derived from the previous one.

A fast and robust bulk-loading algorithm for indexing very large digital elevation datasets: I. Algorithm

Digital elevation models (DEMs) constitute a valuable source of data for a number of geoscience-related applications. The Shuttle Radar Topography Mission (SRTM) collected and made available to the public the world's largest DEM (composed of billions of points) until that date. The SRTM DEM is stored on the NASA repository as a well-organized collection of flat files. The retrieval of this stored topographic information about a region of interest involves one selection of a proper list of files, their downloading, data filtering in the desired region, and their processing according to user needs. With the aim to provide an easier and faster access to this data by improving its further analysis and processing, we have indexed the SRTM DEM by means of a spatial indexing based on the kd-tree data structure, called the Q-tree. This paper is the first in a two-part series that describes the method followed to build an index on such huge amounts of data, minimizing the number of insert operations. We demonstrate that our method can build a very efficient space-partitioning index, with good performance in both point and range queries on the spatial data. To the best of our knowledge, this is the only successful spatial indexing proposal in the literature that deals with such a huge volume of data.

KD-TREE BASED FAST RAY TRACING FOR RCS PREDICTION

Ray tracing is of great use for computational electromagnetics, such as the well-known shooting and bouncing ray (SBR) method. In this paper, the kd-tree data structure, coupled with the mailbox technique, is proposed to accelerate the ray tracing in the SBR. The kd-tree is highly effective in handling the irregularly distribution of patches of the target, while the repeatedly intersection tests between the ray and the patch when using space division acceleration structures can be eliminated through the mailbox technique. Numerical results show excellent agreement with the measured data and the exact solution, and demonstrate that the kd-tree as well as the mailbox technique can greatly reduce the computation time.