Performance Of Range Queries Research Articles

Lock-free Contention Adapting Search Trees

Concurrent key-value stores with range query support are crucial for the scalability and performance of many applications. Existing lock-free data structures of this kind use a fixed synchronization granularity. Using a fixed synchronization granularity in a concurrent key-value store with range query support is problematic as the best performing synchronization granularity depends on a number of factors that are difficult to predict, such as the level of contention and the number of items that are accessed by range queries. We present the first linearizable lock-free key-value store with range query support that dynamically adapts its synchronization granularity. This data structure is called the lock-free contention adapting search tree (LFCA tree). An LFCA tree automatically performs local adaptations of its synchronization granularity based on heuristics that take contention and the performance of range queries into account. We show that the operations of LFCA trees are linearizable, that the lookup operation is wait-free, and that the remaining operations (insert, remove and range query) are lock-free. Our experimental evaluation shows that LFCA trees achieve more than twice the throughput of related lock-free data structures in many scenarios. Furthermore, LFCA trees are able to perform substantially better than data structures with a fixed synchronization granularity over a wide range of scenarios due to their ability to adapt to the scenario at hand.

A High-Performance Spatial Range Query-Based Data Discovery Method on Massive Remote Sensing Data via Adaptive Geographic Meshing and Coding

Many countries and organizations have been sharing a vast amount of remote sensing (RS) images across the Internet via their portals to promote the flourishing development of RS applications. These portals provide the data discovery ability to the public with necessary ways. Spatial range query is one of them to enable data discovery on RS images by their spatial extents. This article first analyzes the technical challenges in building a spatial index based on the key-value model. Then, this article presents how our proposed adaptive geographic meshing and coding method solves these challenges. The proposed method transforms the spatial extent of RS images to key-value-based indexing records to ensure spatial proximity as well as a low ratio of data duplication in indexing records. This article also shows how the proposed query selectivity strategy improves the performance of spatial range queries on indexing records. The strategy determines the order of two essential operations in a spatial range query, namely, the data deduplication and the spatial relationship computation. Finally, we implement a prototype system based on HBase and develop two MapReduce-based algorithms to speedup the process of the index construction and the spatial range query. Extensive experiments on a real dataset demonstrate that our proposed method owns a better query performance as well as a lower space overhead than competitors to handle spatial range queries on RS images.

B 3 -Tree

In this work, we propose B 3 -tree , a hybrid index for persistent memory that leverages the byte-addressability of the in-memory index and the page locality of B-trees. As in the byte-addressable in-memory index, B 3 -tree is updated by 8-byte store instructions. Also, as in disk-based index, B 3 -tree is failure-atomic since it makes every 8-byte store instruction transform a consistent index into another consistent index without the help of expensive logging. Since expensive logging becomes unnecessary, the number of cacheline flush instructions required for B 3 -tree is significantly reduced. Our performance study shows that B 3 -tree outperforms other state-of-the-art persistent indexes in terms of insert and delete performance. While B 3 -tree shows slightly worse performance for point query performance, the range query performance of B 3 -tree is 2x faster than FAST and FAIR B-tree because the leaf page size of B 3 -tree can be set to 8x larger than that of FAST and FAIR B-tree without degrading insertion performance. We also show that read transactions can access B 3 -tree without acquiring a shared lock because B 3 -tree remains always consistent while a sequence of 8-byte write operations are making changes to it. As a result, B 3 -tree provides high concurrency level comparable to FAST and FAIR B-tree.

Succinct Range Filters

We present the Succinct Range Filter (SuRF), a fast and compact data structure for approximate membership tests. Unlike traditional Bloom filters, SuRF supports both single-key lookups and common range queries: open-range queries, closed-range queries, and range counts. SuRF is based on a new data structure called the Fast Succinct Trie (FST) that matches the point and range query performance of state-of-the-art order-preserving indexes, while consuming only 10 bits per trie node. The false-positive rates in SuRF for both point and range queries are tunable to satisfy different application needs. We evaluate SuRF in RocksDB as a replacement for its Bloom filters to reduce I/O by filtering requests before they access on-disk data structures. Our experiments on a 100-GB dataset show that replacing RocksDB’s Bloom filters with SuRFs speeds up open-seek (without upper-bound) and closed-seek (with upper-bound) queries by up to 1.5× and 5× with a modest cost on the worst-case (all-missing) point query throughput due to slightly higher false-positive rate.

Partition pruning for range query on distributed log-structured merge-tree

Log-structured merge tree (LSM-tree) is adopted by many distributed storage systems. It contains a Memtable and a number of SSTables. The Memtable is an in-memory structure and the SSTable is a disk-based structure. Data records are horizontally partitioned over the primary key and stored in different SSTables. Data writes on records are first served by the Memtable and then compacted to SSTables periodically. Although this design optimizes data writes by avoiding random disk writes, it is unfriendly to read request since the results should be retrieved and merged from both Memtable and SSTables. In particular, when the Memtable and SSTables are distributed on different nodes, it incurs expensive costs to serve range queries. A range query on non-primary key columns has to scan all partitions, which generates many network and I/O expenses. In this paper, we propose a partition pruning strategy to save cost for range queries. A statistics cache is designed to determine whether a partition contains the desired data or not, which enables read requests to avoid scanning useless partitions. As records can be updated in Memtable freely, to prevent incorrect filtering, a version-based cache synchronization strategy is proposed to ensure the queries to obtain the latest data state. We implement the proposed method in an open source distributed database and conduct comprehensive experiments. Experimental results reveal that the performance of range queries increased 30% ~ 40% with our partition pruning technique.

Succinct Range Filters

We present the Succinct Range Filter (SuRF), a fast and compact data structure for approximate membership tests. Unlike traditional Bloom filters, SuRF supports both single-key lookups and common range queries. SuRF is based on a new data structure called the Fast Succinct Trie (FST) that matches the point and range query performance of state-of-the-art order-preserving indexes, while consuming only 10 bits per trie node. The false positive rates in SuRF for both point and range queries are tunable to satisfy different application needs. We evaluate SuRF in RocksDB as a replacement for its Bloom filters to reduce I/O by filtering requests before they access on-disk data structures. Our experiments on a 100 GB dataset show that replacing RocksDB's Bloom filters with SuRFs speeds up open-seek (without upper-bound) and closed-seek (with upper-bound) queries by up to 1.5× and 5× with a modest cost on the worst-case (all-missing) point query throughput due to slightly higher false positive rate.

A New Indexing Method for Uncertain Databases

This paper presents an indexing method called the uncertain data index (UD-index) for uncertain databases. The design objectives of the UD-index are improving the range query performance of the multidimensional indexing methods and providing a compromise between optimal index node clustering. Although more than ten years of database research has resulted in a great variety of multidimensional indexing methods, most efforts have focused on the data-level clustering and there has been no attempt to cluster index nodes themselves in dynamic environments. As a result, most related index nodes are widely scattered on the disk due to dynamic page allocation, and it requires many random disk accesses during the range search. The UD-index avoids that by storing the related nodes contiguously in a segment that contains a sequence of contiguous disk pages. The UD-index improves the range query performance by offering high-performance sequential disk access within a segment. A new cost model is introduced to estimate the range query performance. It takes into consideration the physical adjacency of pages read as well as the number of pages accessed. The analytic performance analysis indicates that the UD-index shows better performance than the traditional indexing methods in most cases.

Data-independent vantage point selection for range queries

Vantage point-based indexing is a popular technique for implementing range queries in main memory database. Vantage points are reference points that are used to improve the performance of range queries. In the past, vantage points have been derived from the data points in the database by using various heuristics. These approaches are, therefore, data dependent and not able to handle dynamic databases (allowing insertions and deletions) easily. Further, the amount of time needed for deriving vantage points for these approaches is very high for larger databases. We propose a data-independent technique for creating vantage points. Constraint of our approach is that values in each dimension of the feature vectors have to be bounded. Extensive experiments with real and synthetic data show that the proposed technique is superior to existing methods.

Optimizing Range Query Processing for Dual Bitmap Index

Bitmap-based indexes are known to be the most effective indexing method for retrieving and answering selective queries in a read-only environment. Various types of encoding bitmap indexes significantly improve query time efficiency by utilizing fast Boolean operations directly on the index before retrieving the raw data. In particular, the dual bitmap index improves the performance of equality queries in terms of the space vs. time trade-off. However, the performance of range queries is unsatisfactory. In this paper, an optimizing algorithm is proposed to improve the range query processing for the dual bitmap index. The results of the experiment conducted show that the proposed algorithm, called Dual-simRQ, reduces the number of bitmap vectors scanned and the Boolean operations performed, which impacts the overall performance for range query processing.

OLAPS: Online load-balancing in range-partitioned main memory database with approximate partition statistics

Modern database systems can achieve high throughput main-memory query execution by being aware of the dynamics of highly parallel hardware. In such systems, data is partitioned into smaller pieces to reach a better parallelism. Unfortunately, data skew is one of the main problems faced during parallel processing in a parallel main memory database. In some data-intensive applications, parallel range queries over a dynamic range partitioned system are important. Continuous insertions/deletions can lead to a very high degree of data skew and consequently a poor performance of parallel range queries. In this paper, we propose an approach for maintaining balanced loads over a set of nodes as in a system of communicating vessels, by migrating tuples between neighboring nodes. These frequent (or even continuous) data transfers inevitably involve dynamic changes in the partition statistics. To avoid the performance degradation typically associated with this dynamism, we provide a solution based on an approximate Partition Statistics Table. The basic idea behind this table is that both clients and nodes may have an imperfect knowledge about the effective load distribution. They can nevertheless locate any data with almost the same efficiency as using exact partition statistics. Furthermore, maintaining load distribution statistics do not require exchanging additional messages as opposed to the cost of efficient solutions from the state-of-art (which requires at least O(logn) messages). We show through intensive experiments that our proposal supports efficient range queries, while simultaneously guaranteeing storage balance even in the presence of numerous concurrent insertions/deletions generating a heavy skewed data distribution.

Spatial coding-based approach for partitioning big spatial data in Hadoop

Spatial data partitioning (SDP) plays a powerful role in distributed storage and parallel computing for spatial data. However, due to skew distribution of spatial data and varying volume of spatial vector objects, it leads to a significant challenge to ensure both optimal performance of spatial operation and data balance in the cluster. To tackle this problem, we proposed a spatial coding-based approach for partitioning big spatial data in Hadoop. This approach, firstly, compressed the whole big spatial data based on spatial coding matrix to create a sensing information set (SIS), including spatial code, size, count and other information. SIS was then employed to build spatial partitioning matrix, which was used to spilt all spatial objects into different partitions in the cluster finally. Based on our approach, the neighbouring spatial objects can be partitioned into the same block. At the same time, it also can minimize the data skew in Hadoop distributed file system (HDFS). The presented approach with a case study in this paper is compared against random sampling based partitioning, with three measurement standards, namely, the spatial index quality, data skew in HDFS, and range query performance. The experimental results show that our method based on spatial coding technique can improve the query performance of big spatial data, as well as the data balance in HDFS. We implemented and deployed this approach in Hadoop, and it is also able to support efficiently any other distributed big spatial data systems.

A Novel Air Index for Range Queries in Road Networks

Objective of the present work is to improve range query performance using Hybrid Spatial Air Index (HSAI). HSAI has been designed with combination of both cache management and network coding for processing range queries in road networks. HSAI has been utilized the advantage of both cache management and network coding and reduce client search space. The experiments have been conducted for evaluating performance, the experimental results show that HSAI outperform.

Fast Bulk Bitwise AND and OR in DRAM

Bitwise operations are an important component of modern day programming, and are used in a variety of applications such as databases. In this work, we propose a new and simple mechanism to implement bulk bitwise AND and OR operations in DRAM, which is faster and more efficient than existing mechanisms. Our mechanism exploits existing DRAM operation to perform a bitwise AND/OR of two DRAM rows completely within DRAM. The key idea is to simultaneously connect three cells to a bitline before the sense-amplification. By controlling the value of one of the cells, the sense amplifier forces the bitline to the bitwise AND or bitwise OR of the values of the other two cells. Our approach can improve the throughput of bulk bitwise AND/OR operations by $9.7X$ and reduce their energy consumption by $50.5X$ . Since our approach exploits existing DRAM operation as much as possible, it requires negligible changes to DRAM logic. We evaluate our approach using a real-world implementation of a bit-vector based index for databases. Our mechanism improves the performance of commonly-used range queries by 30 percent on average.

Supporting multidimensional range queries in Hierarchically Distributed Tree

SummaryAn examination of the multidimensional range query in existing peer‐to‐peer (P2P) overlay networks indicates that multidimensional range queries are sensitive to underlying topologies; this is because partitioning and mapping of multidimensional data space are two interconnected parts of a process that must be carried out cooperatively. The first section focuses on how to preserve data localities, whereas the second section concerns how to accommodate and maintain data localities at the P2P overlay layer. There are many studies that have been conducted on the first section since 1966, and those works that are well accepted are mostly based on recursive decomposition, which forms a tree structure in nature. However, less effort has been made to provide comparable support from the P2P overlay layer. In our previous work, we proposed the Hierarchically Distributed Tree (HD Tree) in order to better support multidimensional range queries in the P2P overlay network. This paper further explores error‐resilient routing and load balancing strategies that can be employed in the HD Tree. We also provide a complete set of experimental results for all routing operations: Join and Leave of nodes, range queries at different levels of selectivity, and the dynamic load balancing scheme. Comparisons are made by conducting simulations under both the ideal and the error‐prone routing environment and within various ary HD Trees. The experimental results show that load balancing in the HD Tree can be adjusted dynamically and globally, and it is actually a trade‐off between distributing the basic load and the involvement of nodes in range querying. The experimental results also indicate that a maximum of 10 percent of routing nodes’ failures do not have significant effects on the performance of range queries. However, a lower ary HD Tree appears to have better routing performance, whereas a higher ary HD Tree achieves a higher fault‐tolerant capacity. Nevertheless, the performance of range queries in a higher ary HD Tree can be further optimized if all possible routing options can be fully explored in the error‐prone routing environment. Copyright © 2013 John Wiley & Sons, Ltd.

Parallel indexing technique for spatio-temporal data

The requirements for efficient access and management of massive multi-dimensional spatio-temporal data in geographical information system and its applications are well recognized and researched. The most popular spatio-temporal access method is the R-Tree and its variants. However, it is difficult to use them for parallel access to multi-dimensional spatio-temporal data because R-Trees, and variants thereof, are in hierarchical structures which have severe overlapping problems in high dimensional space. We extended a two-dimensional interval space representation of intervals to a multi-dimensional parallel space, and present a set of formulae to transform spatio-temporal queries into parallel interval set operations. This transformation reduces problems of multi-dimensional object relationships to simpler two-dimensional spatial intersection problems. Experimental results show that the new parallel approach presented in this paper has superior range query performance than R*-trees for handling multi-dimensional spatio-temporal data and multi-dimensional interval data. When the number of CPU cores is larger than that of the space dimensions, the insertion performance of this new approach is also superior to R*-trees. The proposed approach provides a potential parallel indexing solution for fast data retrieval of massive four-dimensional or higher dimensional spatio-temporal data.

Analyzing design choices for distributed multidimensional indexing

Scientific datasets are often stored on distributed archival storage systems, because geographically distributed sensor devices store the datasets in their local machines and also because the size of scientific datasets demands large amount of disk space. Multidimensional indexing techniques have been shown to greatly improve range query performance into large scientific datasets. In this paper, we discuss several ways of distributing a multidimensional index in order to speed up access to large distributed scientific datasets. This paper compares the designs, challenges, and problems for distributed multidimensional indexing schemes, and provides a comprehensive performance study of distributed indexing to provide guidelines to choose a distributed multidimensional index for a specific data analysis application.

Efficient evaluation of partially-dimensional range queries in large OLAP datasets

In light of the increasing requirement for processing multidimensional queries on OLAP (relational) data, the database community has focused on the queries (especially range queries) on the large OLAP datasets from the view of multidimensional data. It is well-known that multidimensional indices are helpful to improve the performance of such queries. However, we found that much information irrelevant to queries also has to be read from disk if the existing multidimensional indices are used with OLAP data, which greatly degrade the search performance. This problem comes from particularity on the actual queries exerted on OLAP data. That is, in many OLAP applications, the query conditions probably are only with partial dimensions (not all) of the whole index space. Such range queries are called partially-dimensional (PD) range queries in this study. Based on R*-tree, we propose a new index structure, called AR*-tree, to counter the actual queries on OLAP data. The results of both mathematical analysis and many experiments with different datasets indicate that the AR*-tree can clearly improve the performance of PD range queries, esp. for large OLAP datasets.

Efficient range queries in spatial databases over Peer-to-Peer Networks

Peer-to-Peer (P2P) network, e.g., Chord, is designed mainly for one-dimensional data retrieval and point queries. Multimedia and spatial applications require efficient execution of range queries. This paper proposes an efficient scheme to support range query over Chord while balancing the storage load. The paper proposes a rotating token scheme that places the joining nodes in appropriate locations to share loads with overloaded nodes. To support range queries, we utilise an order-preserving mapping function to map keys to nodes in an order preserving manner. Simulation experiments show significant gain in storage load balancing and the performance of range queries over Chord.

Indexing Moving Objects for Trajectory Retrieval on Location-Based Services

Due to the continuous growth of wireless communication technology and mobile equipment, the history management of moving object is important in a wide range of location-based applications. To process queries for history data, trajectories, we generally use trajectory-preserving index schemes based on the trajectory preservation property. This property means that a leaf node only contains segments belonging to a particular trajectory, regardless of the spatiotemporal locality of segments. The sacrifice of spatiotemporal locality, however, causes the index to increase the dead space of MBBs of non-leaf nodes and the overlap between the MBBs of nodes. Therefore, an index scheme for trajectories shows good performance with trajectory-based queries, but not with coordinate-based queries, such as range queries. We propose new index schemes that improve the performance of range queries without reducing performance with trajectory based queries.

Multi-resolution bitmap indexes for scientific data

The unique characteristics of scientific data and queries cause traditional indexing techniques to perform poorly on scientific workloads, occupy excessive space, or both. Refinements of bitmap indexes have been proposed previously as a solution to this problem. In this article, we describe the difficulties we encountered in deploying bitmap indexes with scientific data and queries from two real-world domains. In particular, previously proposed methods of binning, encoding, and compressing bitmap vectors either were quite slow for processing the large-range query conditions our scientists used, or required excessive storage space. Nor could the indexes easily be built or used on parallel platforms. In this article, we show how to solve these problems through the use of multi-resolution, parallelizable bitmap indexes, which support a fine-grained trade-off between storage requirements and query performance. Our experiments with large data sets from two scientific domains show that multi-resolution, parallelizable bitmap indexes occupy an acceptable amount of storage while improving range query performance by roughly a factor of 10, compared to a single-resolution bitmap index of reasonable size.