Search Data Structures Research Articles

Succinct data structure for path graphs

We consider the problem of designing a succinct data structure for path graphs, that generalizes interval graphs, on n vertices while efficiently supporting degree, adjacency, and neighbourhood queries. We provide the following two solutions for this problem:1.an nlog⁡n+o(nlog⁡n)-bit succinct data structure that supports adjacency query in O(log⁡n) time, neighbourhood query in O(dlog⁡n) time and finally, degree query in min⁡{O(log2⁡n),O(dlog⁡n)} time where d is the degree of the queried vertex.2.an O(nlog2⁡n)-bit space-efficient data structure that supports adjacency, neighborhood, and degree queries optimally. Central to our data structures is the usage of the heavy path decomposition, followed by careful bookkeeping using an orthogonal range search data structure using wavelet trees among others, which may be of independent interest for designing succinct data structures for other graph classes.

Bag of World Anchors for Instant Large-Scale Localization.

In this work, we present a novel scene description to perform large-scale localization using only geometric constraints. Our work extends compact world anchors with a search data structure to efficiently perform localization and pose estimation of mobile augmented reality devices across multiple platforms (eg., hololens, ipad). The algorithm uses a bag-of-words approach to characterize distinct scenes (eg., rooms). Since the individual scene representations rely on compact geometric (rather than appearance-based) features, the resulting search structure is very lightweight and fast, lending itself to deployment on mobile devices. We present a set of experiments demonstrating the accuracy, performance and scalability of our novel localization method. In addition, we describe several use cases demonstrating how efficient cross-platform localization facilitates sharing of augmented reality experiences.

Convexity Hierarchies in Grid Networks

Several algorithms for path planning in grid networks rely on graph decomposition to reduce the search space size; either by constructing a search data structure on the components, or by using component information for A* guidance. The focus is usually on obtaining components of roughly equal size with few boundary nodes each. In this paper, we consider the problem of splitting a graph into convex components. A convex component is characterized by the property that for all pairs of its members, the shortest path between them is also contained in it. Thus, given a source node, a target node, and a (small) convex component that contains both of them, path planning can be restricted to this component without compromising optimality. We prove that it is NP-hard to find a balanced node separator that splits a given graph into convex components. However, we also present and evaluate heuristics for (hierarchical) convex decomposition of grid networks that perform well across various benchmarks. Moreover, we describe how existing path planning methods can benefit from the computation of convex components. As one main outcome, we show that contraction hierarchies become up to an order of magnitude faster on large grids when the contraction order is derived from a convex graph decomposition.

Range Search over Encrypted Multi-Attribute Data

This work addresses expressive queries over encrypted data by presenting the first systematic study of multi-attribute range search on a symmetrically encrypted database outsourced to an honest-but-curious server. Prior work includes a thorough analysis of single-attribute range search schemes (e.g. Demertzis et al. 2016) and a proposed high-level approach for multi-attribute schemes (De Capitani di Vimercati et al. 2021). We first introduce a flexible framework for building secure range search schemes over multiple attributes (dimensions) by adapting a broad class of geometric search data structures to operate on encrypted data. Our framework encompasses widely used data structures such as multi-dimensional range trees and quadtrees, and has strong security properties that we formally prove. We then develop six concrete highly parallelizable range search schemes within our framework that offer a sliding scale of efficiency and security tradeoffs to suit the needs of the application. We evaluate our schemes with a formal complexity and security analysis, a prototype implementation, and an experimental evaluation on real-world datasets.

An Efficient Blockchain Transaction Retrieval System

In the era of the digital economy, blockchain has developed well in various fields, such as finance and digital copyright, due to its unique decentralization and traceability characteristics. However, blockchain gradually exposes the storage problem, and the current blockchain stores the block data in third-party storage systems to reduce the node storage pressure. The new blockchain storage method brings the blockchain transaction retrieval problem. The problem is that when unable to locate the block containing this transaction, the user must fetch the entire blockchain ledger data from the third-party storage system, resulting in huge communication overhead. For this problem, we exploit the semi-structured data in the blockchain and extract the universal blockchain transaction characteristics, such as account address and time. Then we establish a blockchain transaction retrieval system. Responding to the lacking efficient retrieval data structure, we propose a scalable secondary search data structure BB+ tree for account address and introduce the I2B+ tree for time. Finally, we analyze the proposed scheme’s performance through experiments. The experiment results prove that our system is superior to the existing methods in single-feature retrieval, concurrent retrieval, and multi-feature hybrid retrieval. The retrieval time under single feature retrieval is reduced by 40.54%, and the retrieval time is decreased by 43.16% under the multi-feature hybrid retrieval. It has better stability in different block sizes and concurrent retrieval scales.

Proving highly-concurrent traversals correct

Modern highly-concurrent search data structures, such as search trees, obtain multi-core scalability and performance by having operations traverse the data structure without any synchronization. As a result, however, these algorithms are notoriously difficult to prove linearizable, which requires identifying a point in time in which the traversal's result is correct. The problem is that traversing the data structure as it undergoes modifications leads to complex behaviors, necessitating intricate reasoning about all interleavings of reads by traversals and writes mutating the data structure. In this paper, we present a general proof technique for proving unsynchronized traversals correct in a significantly simpler manner, compared to typical concurrent reasoning and prior proof techniques. Our framework relies only on sequential properties of traversals and on a conceptually simple and widely-applicable condition about the ways an algorithm's writes mutate the data structure. Establishing that a target data structure satisfies our condition requires only simple concurrent reasoning, without considering interactions of writes and reads. This reasoning can be further simplified by using our framework. To demonstrate our technique, we apply it to prove several interesting and challenging concurrent binary search trees: the logical-ordering AVL tree, the Citrus tree, and the full contention-friendly tree. Both the logical-ordering tree and the full contention-friendly tree are beyond the reach of previous approaches targeted at simplifying linearizability proofs.

Efficient FPGA Implementation of K-Nearest-Neighbor Search Algorithm for 3D LIDAR Localization and Mapping in Smart Vehicles

K-Nearest-Neighbor search (KNN) has been extensively used in the localization and mapping based on 3D laser point clouds in smart vehicles. Considering the real-time requirement of localization and stringent battery constraint in smart vehicles, it is a great challenge to develop highly energy-efficient KNN implementations. Unfortunately, previous KNN implementations either cannot efficiently build search data structures or cannot search efficiently in massive and unevenly distributed point clouds. To solve the issue, we propose a new framework to optimize the implementation of KNN on FPGAs. First, we propose a novel data structure with a spatial subdivision method, which can be built efficiently even for massive point clouds. Second, based on our data structure, we propose a KNN search algorithm which is able to search in unevenly distributed point clouds efficiently. We have implemented the new framework on both FPGA and GPU. Energy efficiency results show that our proposed method is on average 2.1 times and 6.2 times higher than the state-of-the-art implementations of KNN on FPGA and GPU platform, respectively.

Localization in Underwater Acoustic Sensor Networks using Trapezoid

Underwater Acoustic Sensor Networks (UASN) has driven a lot of attention from researchers because of advancements in sensor technology and unexplored applications of the ocean. UASNs monitor the targeted area with heterogeneous underwater sensors and relay that information to the onshore sink node in mission-critical applications. It is very much essential to know the source of information whenever some critical events happened in the UASNs. Hence, to learn the source of information, i.e. finding the location of the sensor node is crucial. To address this issue, in this paper, initially geometrical object such as trapezoid is used to form the clusters in the targeted region. After that, the proposed localization algorithm is applied and it works in three phases. (i) In the first phase, the sink node initiates the trapezoid formation process through Trapezoid Formation Agent (TFA) and divides the whole network into trapezoids of different geometrical shapes by traveling across the linear trajectory and also creates a search data structure. (ii) In the second phase, the sink deploys AUV at a certain depth for patrolling along the linear trajectory and broadcasts real-time location contained beacon messages at specified points through that anchor nodes are localized by using RSSI. (iii) Sink node activates Localization Agent (LA) in the third phase to perform the location identification process at the trapezoids by using the trilateration method. This work addresses the inherent localization issue of UASNs algorithms and hence it applies to the applications which consider the localization issue. This proposed scheme is well supported by node agencies and knowledgebase. The proposed scheme is simulated in C and validated by different performance parameters.

Family Member Search Algorithms for Peridynamic Analysis

Peridynamic equation of motion is usually solved numerically by using meshless approaches. Family search process is one of the most time-consuming parts of a peridynamic analysis. Especially for problems which require continuous update of family members inside the horizon of a material point, the time spent to search for family members becomes crucial. Hence, efficient algorithms are required to reduce the computational time. In this study, various family member search algorithms suitable for peridynamic simulations are presented including brute-force search, region partitioning, and tree data structures. By considering problem cases for different number of material points, computational time between different algorithms is compared and the most efficient algorithm is determined.

Dynamic Interpolation Search revisited

A new dynamic Interpolation Search (IS) data structure is presented that achieves O(log⁡log⁡n) search time with high probability on unknown continuous or even discrete input distributions with measurable probability of element collisions, including power law and Binomial distributions. No such previous result holds for IS when the probability of element collisions is measurable. Moreover, our data structure exhibits O(1) search time with high probability (w.h.p.) for a wide class of input distributions that contains all those for which o(log⁡log⁡n) expected search time was previously known.

Querying relational event graphs using colored range searching data structures

We present a general approach for analyzing structural parameters of a relational event graph within arbitrary query time intervals using colored range query data structures. Relational event graphs generally represent social network datasets, where each graph edge carries a timestamp. We provide data structures based on colored range searching to efficiently compute several graph parameters (e.g., density, neighborhood overlap, h-index).

Efficient searching in meshfree methods

Meshfree methods such as the Reproducing Kernel Particle Method and the Element Free Galerkin method have proven to be excellent choices for problems involving complex geometry, evolving topology, and large deformation, owing to their ability to model the problem domain without the constraints imposed on the Finite Element Method (FEM) meshes. However, meshfree methods have an added computational cost over FEM that come from at least two sources: increased cost of shape function evaluation and the determination of adjacency or connectivity. The focus of this paper is to formally address the types of adjacency information that arises in various uses of meshfree methods; a discussion of available techniques for computing the various adjacency graphs; propose a new search algorithm and data structure; and finally compare the memory and run time performance of the methods.

Feature-Based and String-Based Models for Predicting RNA-Protein Interaction.

In this work, we study two approaches for the problem of RNA-Protein Interaction (RPI). In the first approach, we use a feature-based technique by combining extracted features from both sequences and secondary structures. The feature-based approach enhanced the prediction accuracy as it included much more available information about the RNA-protein pairs. In the second approach, we apply search algorithms and data structures to extract effective string patterns for prediction of RPI, using both sequence information (protein and RNA sequences), and structure information (protein and RNA secondary structures). This led to different string-based models for predicting interacting RNA-protein pairs. We show results that demonstrate the effectiveness of the proposed approaches, including comparative results against leading state-of-the-art methods.

Plagiarism detection in students’ programming assignments based on semantics: multimedia e-learning based smart assessment methodology

The multimedia-based e-Learning methodology provides virtual classrooms to students. The teacher uploads learning materials, programming assignments and quizzes on university’ Learning Management System (LMS). The students learn lessons from uploaded videos and then solve the given programming tasks and quizzes. The source code plagiarism is a serious threat to academia. However, identifying similar source code fragments between different programming languages is a challenging task. To solve the problem, this paper proposed a new plagiarism detection technique between C++ and Java source codes based on semantics in multimedia-based e-Learning and smart assessment methodology. First, it transforms source codes into tokens to calculate semantic similarity in token by token comparison. After that, it finds semantic similarity in scalar value for the complete source codes written in C++ and Java. To analyse the experiment, we have taken the dataset consists of four (4) case studies of Factorial, Bubble Sort, Binary Search and Stack data structure in both C++ and Java. The entire experiment is done in R Studio with R version 3.4.2. The experimental results show better semantic similarity results for plagiarism detection based on comparison.

OpenRBC: A Fast Simulator of Red Blood Cells at Protein Resolution

We present OpenRBC, a coarse-grained molecular dynamics code, which is capable of performing an unprecedented in silico experiment—simulating an entire mammal red blood cell lipid bilayer and cytoskeleton as modeled by multiple millions of mesoscopic particles—using a single shared memory commodity workstation. To achieve this, we invented an adaptive spatial-searching algorithm to accelerate the computation of short-range pairwise interactions in an extremely sparse three-dimensional space. The algorithm is based on a Voronoi partitioning of the point cloud of coarse-grained particles, and is continuously updated over the course of the simulation. The algorithm enables the construction of the key spatial searching data structure in our code, i.e., a lattice-free cell list, with a time and space cost linearly proportional to the number of particles in the system. The position and the shape of the cells also adapt automatically to the local density and curvature. The code implements OpenMP parallelization and scales to hundreds of hardware threads. It outperforms a legacy simulator by almost an order of magnitude in time-to-solution and >40 times in problem size, thus providing, to our knowledge, a new platform for probing the biomechanics of red blood cells.

A Highly-Efficient Implementation of the Doktorov Recurrence Equations for Franck-Condon Calculations.

In this paper, a new algorithm for computing Franck-Condon overlaps using the Doktorov recurrence equations is proposed. One of the major computational stresses of using the recurrence equations arises from searching data structures for overlaps that are stored in memory. The proposed algorithm alleviates this problem by tracking, throughout the algorithm, the locations in memory of overlaps that are required to use the recurrence relations. The tracking procedure helps to significantly reduce the run time of the algorithm compared to existing implementations.

Towards efficient security policy lookup on many-core network processing platforms

Modern network security devices employ packet classification and pattern matching algorithms to inspect packets. Due to the complexity and heterogeneity of different search data structures, it is difficult for existing algorithms to leverage modern hardware platforms to achieve high performance. This paper presents a Structural Compression (SC) method that optimizes the data structures of both algorithms. It reviews both algorithms under the model of search space decomposition, and homogenizes their search data structures. This approach not only guarantees deterministic lookup speed but also optimizes the data structure for efficient implementation on many-core platforms. The performance evaluation reveals that the homogeneous data structure achieves 10Gbps line-rate 64byte packet classification throughput and multi-Gbps deep inspection speed.

Iterative sampling based frequent itemset mining for big data

Frequent pattern mining attracts extensive research interests over the past two decades: including mining frequent item sets from transactions, extracting frequent sequences from bio-arrays and detecting common subgraph from molecular structures. In the era of big data, the explosive data volume brings new challenges to frequent pattern mining: (1) Space complexity: both input data, intermediate results and the outputted patterns could be too large to fit into memory which prevents many algorithms from executing; (2) Time complexity: many existing approaches rely on exhaustive search or complicated data structures to mine frequent patterns which prove to be inapplicable for big data. To deal with these two challenges. we propose ISbFIM, an Iterative Sampling based Frequent Itemset Mining method. Rather than process the entire data set at once, ISbFIM samples computationally-manageable subsets and extracts frequent itemsets from these subsets. By repeating this process for a sufficient number of times, we can guarantee both theoretically and empirically that the frequent itemsets can be enumerated without running into a combinatorial explosion. ISbFIM can be easily parallelized and applied to mine item sets, sequences or structures. We implement a Map-Reduce version of ISbFIM to demonstrate its scalability on big data.

Asynchronized Concurrency

We introduce "asynchronized concurrency (ASCY)," a paradigm consisting of four complementary programming patterns. ASCY calls for the design of concurrent search data structures (CSDSs) to resemble that of their sequential counterparts. We argue that ASCY leads to implementations which are portably scalable: they scale across different types of hardware platforms, including single and multi-socket ones, for various classes of workloads, such as read-only and read-write, and according to different performance metrics, including throughput, latency, and energy. We substantiate our thesis through the most exhaustive evaluation of CSDSs to date, involving 6 platforms, 22 state-of-the-art CSDS algorithms, 10 re-engineered state-of-the-art CSDS algorithms following the ASCY patterns, and 2 new CSDS algorithms designed with ASCY in mind. We observe up to 30% improvements in throughput in the re-engineered algorithms, while our new algorithms out-perform the state-of-the-art alternatives.

Asynchronized Concurrency

We introduce "asynchronized concurrency (ASCY)," a paradigm consisting of four complementary programming patterns. ASCY calls for the design of concurrent search data structures (CSDSs) to resemble that of their sequential counterparts. We argue that ASCY leads to implementations which are portably scalable: they scale across different types of hardware platforms, including single and multi-socket ones, for various classes of workloads, such as read-only and read-write, and according to different performance metrics, including throughput, latency, and energy. We substantiate our thesis through the most exhaustive evaluation of CSDSs to date, involving 6 platforms, 22 state-of-the-art CSDS algorithms, 10 re-engineered state-of-the-art CSDS algorithms following the ASCY patterns, and 2 new CSDS algorithms designed with ASCY in mind. We observe up to 30% improvements in throughput in the re-engineered algorithms, while our new algorithms out-perform the state-of-the-art alternatives.