Improve Query Performance Research Articles

Popularity-aware collective keyword queries in road networks

This paper addresses a popularity-aware collective keyword (PAC-K) query in road networks. Given a road network with POIs (Points of Interest), which is modeled as a road network graph, where each node locating in a two-dimensional space represents a road intersection or a POI, and each edge with weight represents a road segment, the PACK query aims to find a group of popular POIs (i.e., a popular region) that cover the query’s keywords and satisfy the distance requirements from each node to the query node and between each pair of nodes, such that the sum of rating scores over these nodes for the query keywords is maximized. We show the problem of answering the PACK query is NP-Hard. To solve this problem, we present exact and heuristic solutions on small and large road networks, respectively. In particular, to improve query performance, we propose a rating score scaling technique to reduce the search space and a redundant computation reducing technique to reduce the excessive redundant computations in query processing. Extensive performance studies using two real datasets confirm the efficiency, accuracy, and scalability of the proposed solutions.

Continuous k-Nearest Neighbor Queries in Wireless Environments

Objectives: Network Coding-based Spatial Air Index (NCSAI) has been proposed for improving query performance of continuous k-nearest neighbor queries in road networks. Methods: Due to unreliable nature of wireless links, network coding has been playing an important role in wireless environments and improves scalability and throughput. In this work,NCSAI has been adapted using XOR-based network coding because it is simple and efficient coding strategy. Findings: Experiments have been conducted for evaluating query performance, the experimental result show that performance of NCSAI better than state-of-the-art Network Partition Index (NPI). Improvements: We can improve the performance of NCSAI by adapting an efficient network coding strategies.

AdaptDB

Big data analytics often involves complex join queries over two or more tables. Such join processing is expensive in a distributed setting both because large amounts of data must be read from disk, and because of data shuffling across the network. Many techniques based on data partitioning have been proposed to reduce the amount of data that must be accessed, often focusing on finding the best partitioning scheme for a particular workload, rather than adapting to changes in the workload over time. In this paper, we present AdaptDB, an adaptive storage manager for analytical database workloads in a distributed setting. It works by partitioning datasets across a cluster and incrementally refining data partitioning as queries are run. AdaptDB introduces a novel hyper-join that avoids expensive data shuffling by identifying storage blocks of the joining tables that overlap on the join attribute, and only joining those blocks. Hyper-join performs well when each block in one table overlaps with few blocks in the other table, since that will minimize the number of blocks that have to be accessed. To minimize the number of overlapping blocks for common join queries, AdaptDB users smooth repartitioning to repartition small portions of the tables on join attributes as queries run. A prototype of AdaptDB running on top of Spark improves query performance by 2--3x on TPC-H as well as real-world dataset, versus a system that employs scans and shuffle-joins.

On proposing and evaluating a NoSQL document database logical approach

Purpose NoSQL databases do not require a default schema associated with the data. Even that, they are categorized by data models. A model associated with the data can promote better strategies for persistence and manipulation of data in the target database. Based on this motivation, the purpose of this paper is to present an approach for logical design of NoSQL document databases that consists a process that converts a conceptual modeling into efficient logical representations for a NoSQL document database. The authors also evaluate their approach and demonstrate that the generated NoSQL logical structures reduce the amount of data items accessed by queries. Design/methodology/approach This paper presents an approach for logical design of NoSQL document database schemas based on a conceptual schema. The authors generate compact and redundancy-free schemas and define appropriate representations in a NoSQL document logical model. The estimated volume of data and workload information can be considered to generate optimized NoSQL document structures. Findings This approach was evaluated through a case study with an experimental evaluation in the e-commerce application domain. The results demonstrate that the authors’ workload-based conversion process improves query performance on NoSQL documents by reducing the number of database accesses. Originality/value Unlike related work, the reported approach covers all typical conceptual constructs, details a conversion process between conceptual schemas and logical representations for NoSQL document database category and, additionally, considers the estimated database workload to perform optimizations in the logical structure. An experimental evaluation shows that the proposed approach is promising.

Active rule base development for dynamic vertical partitioning of multimedia databases

Currently, vertical partitioning has been used in multimedia databases in order to take advantage of its potential benefits in query optimization. Nevertheless, most vertical partitioning algorithms are static; this means that they optimize a vertical partitioning scheme (VPS) according to a workload, but if this workload suffers changes, the VPS may be degraded, which would result in long query response time. This paper presents a set of active rules to perform dynamic vertical partitioning in multimedia databases. First of all, these rules collect all the information that a vertical partitioning algorithm needs as input. Then, they evaluate this information in order to know if the database has experienced enough changes to trigger a performance evaluator. In this case, if the performance of the database falls below a threshold previously calculated by the rules, the vertical partitioning algorithm is triggered, which gets a new VPS. Finally, the rules materialize the new VPS. Our active rule base is implemented in the system DYMOND, which is an active rule-based system for dynamic vertical partitioning of multimedia databases. DYMOND's architecture and workflow are presented in this paper. Moreover, a case study is used to clarify and evaluate the functionality of our active rule base. Additionally, authors of this paper performed a qualitative evaluation with the aim of comparing and evaluating DYMOND's functionality. The results showed that DYMOND improved query performance in multimedia databases.

A general-purpose query-centric framework for querying big graphs

Pioneered by Google's Pregel, many distributed systems have been developed for large-scale graph analytics. These systems employ a user-friendly "think like a vertex" programming model, and exhibit good scalability for tasks where the majority of graph vertices participate in computation. However, the design of these systems can seriously under-utilize the resources in a cluster for processing light-workload graph queries, where only a small fraction of vertices need to be accessed. In this work, we develop a new open-source system, called Quegel , for querying big graphs. Quegel treats queries as first-class citizens in its design: users only need to specify the Pregel-like algorithm for a generic query, and Quegel processes light-workload graph queries on demand, using a novel superstep-sharing execution model to effectively utilize the cluster resources. Quegel further provides a convenient interface for constructing graph indexes, which significantly improve query performance but are not supported by existing graph-parallel systems. Our experiments verified that Quegel is highly efficient in answering various types of graph queries and is up to orders of magnitude faster than existing systems.

K-nearest neighbors on road networks

A k nearest neighbor ( k NN) query on road networks retrieves the k closest points of interest (POIs) by their network distances from a given location. Today, in the era of ubiquitous mobile computing, this is a highly pertinent query. While Euclidean distance has been used as a heuristic to search for the closest POIs by their road network distance, its efficacy has not been thoroughly investigated. The most recent methods have shown significant improvement in query performance. Earlier studies, which proposed disk-based indexes, were compared to the current state-of-the-art in main memory. However, recent studies have shown that main memory comparisons can be challenging and require careful adaptation. This paper presents an extensive experimental investigation in main memory to settle these and several other issues. We use efficient and fair memory-resident implementations of each method to reproduce past experiments and conduct additional comparisons for several overlooked evaluations. Notably we revisit a previously discarded technique (IER) showing that, through a simple improvement, it is often the best performing technique.

Hybrid query optimization for hard-to-compress bit-vectors

Bit-vectors are widely used for indexing and summarizing data due to their efficient processing in modern computers. Sparse bit-vectors can be further compressed to reduce their space requirement. Special compression schemes based on run-length encoders have been designed to avoid explicit decompression and minimize the decoding overhead during query execution. Moreover, highly compressed bit-vectors can exhibit a faster query time than the non-compressed ones. However, for hard-to-compress bit-vectors, compression does not speed up queries and can add considerable overhead. In these cases, bit-vectors are often stored verbatim (non-compressed). On the other hand, queries are answered by executing a cascade of bit-wise operations involving indexed bit-vectors and intermediate results. Often, even when the original bit-vectors are hard to compress, the intermediate results become sparse. It could be feasible to improve query performance by compressing these bit-vectors as the query is executed. In this scenario, it would be necessary to operate verbatim and compressed bit-vectors together. In this paper, we propose a hybrid framework where compressed and verbatim bitmaps can coexist and design algorithms to execute queries under this hybrid model. Our query optimizer is able to decide at run time when to compress or decompress a bit-vector. Our heuristics show that the applications using higher-density bitmaps can benefit from using this hybrid model, improving both their query time and memory utilization.

LS-AMS: An Adaptive Indexing Structure for Realtime Search on Microblogs

Indexing microblogs for realtime search is challenging, because new microblogs are created at tremendous speed, and user query requests keep constantly changing. To guarantee user obtain complete query results, micro-blogging site maintains huge indices which leads to index fragmentation or extra merging overhead during realtime search. This paper proposes an efficient LogStructured index structure with Adaptive Merging Strategy (LS-AMS) for realtime search on microblogs. LS-AMS structure consists of an inverted index buffer and a sequence of dynamically adjustable index packages with exponentially increasing sizes. These index packages manage their inverted indices using adaptive merging strategy, which can reduce the merging overhead to improve query performance and can adjust the index structure based on environmental factors, such as the arrival rate of query requests and new microblogs. Experimental results show that LS-AMS can greatly improve query performance without increasing the update cost and improve the self-adaptability in dynamic environment.

High-speed query processing over high-speed networks

Modern database clusters entail two levels of networks: connecting CPUs and NUMA regions inside a single server in the small and multiple servers in the large. The huge performance gap between these two types of networks used to slow down distributed query processing to such an extent that a cluster of machines actually performed worse than a single many-core server. The increased main-memory capacity of the cluster remained the sole benefit of such a scale-out. The economic viability of high-speed interconnects such as InfiniBand has narrowed this performance gap considerably. However, InfiniBand's higher network bandwidth alone does not improve query performance as expected when the distributed query engine is left unchanged. The scalability of distributed query processing is impaired by TCP overheads, switch contention due to uncoordinated communication, and load imbalances resulting from the inflexibility of the classic exchange operator model. This paper presents the blueprint for a distributed query engine that addresses these problems by considering both levels of networks holistically. It consists of two parts: First, hybrid parallelism that distinguishes local and distributed parallelism for better scalability in both the number of cores as well as servers. Second, a novel communication multiplexer tailored for analytical database workloads using remote direct memory access (RDMA) and low-latency network scheduling for high-speed communication with almost no CPU overhead. An extensive evaluation within the HyPer database system using the TPC-H benchmark shows that our holistic approach indeed enables high-speed query processing over high-speed networks.

HyXAC: Hybrid XML Access Control Integrating View-Based and Query-Rewriting Approaches

With the increasing usage of XML on information sharing over the Internet, a mechanism for defining and enforcing XML access control is demanded, such that only authorized entities can access the sets of XML data that they are allowed to. The research interests in these areas have grown significantly in recent years. Various access control enforcement solutions have been proposed, each with its inherent advantages and disadvantages. Yet, there is still no solution that can provide superior performance in all situations. In this paper, we present HyXAC, a hybrid approach to enforce XML access control. HyXAC integrates the two most popular categories of XML access control enforcement mechanisms, and earns the benefits from both. In particular, HyXAC first preprocesses user queries by rewriting queries and removing parts violating access control rules, and evaluates the re-written queries using subviews, if they are available. In HyXAC, views are not defined on a per-role basis. Instead, a sub-view is defined for each access control rule, and roles sharing identical rules will share sub-views. Moreover, HyXAC dynamically allocates memory and secondary storage resources to materialize and cache sub-views to improve query performance. We have conducted extensive experiments, and the results show that HyXAC improves query processing efficiency while optimizes the use of system resources.

An Efficient Distributed Data Management Method based key Columns Partition Preprocessing

With the development of mobile internet and social network, the scale of structured data have been increasing to PB level and above rapidly, while the query performance is greatly reduce. The efficiency of query optimization on large-scale datasets is currently a research focus in both academia and industry. In this paper, we present a distributed data management method, designed to improve query performance, called KCSQ. KCSQ analyses historical SQL commands, deduces statistics using frequency and the coupling degree of tables and table columns, and confirms the key column based on statistical evidence. When importing new tables into the HDFS, the data are divided into different blocks according to their key column. Any query on these columns can reduce the amount of data to be queried and the number of working nodes and thus effectively improves the throughput rate of the system.

An Approach for Query Optimization by using Schema Object Base View

Mining of Data is the extraction of hidden prognosticative information from large databases or set of data, is a strong new technology with great prospective to help companies focus on the most important information in their data base. Query optimization is a purpose of many relational database management systems. The query optimizer experiments to dictate the most efficient way to implement a given query by examining the possible query plans. There are different techniques is given for optimizing query using schema based and materialized views in data base namelyQuery Graph, Tableaus, Optimization of Queries having Aggregates. In this paper we are using Different query optimization parameter and create an effective approach by using this approach we are reduce query execution cost, query space and more effectible for the query. The complexity of Queries severely increase the execution cost of the queries and have a critical effect on performance and productivity of decision support systems. It is required to perform expensive join and aggregation operations frequently on the databases. Now if they are not pre calculated in advanced then it leads to reduce query performance. Schema object improve query performance by pre calculating expensive join and aggregation operations on the database prior to execution and storing the results in the database. Schema object define not only relationships, but also allow you to recompute expensive joins and aggregations which lead to optimized query performance in possible ways. Schema object leads to the decrease Query processing cost and Query Maintenance cost in terms of Time factor. Schema object improve query performance by pre calculating expensive join and aggregation operations on the database prior to execution and storing the results in the database. The big advantage of a Schema object based views is extremely fast retrieval of aggregate data, since it is precomputed and stored, at the expense of insert/update/delete so that it increase query performance than the ordinary view and table. Schema object based view is also called Materialized view.

다중 GPU를 이용한 R-tree의 병렬 범위 질의 처리 기법

Ever since the R-tree was proposed to index multi-dimensional data, many efforts have been made to improve its query performances. One common trend to improve query performance is to parallelize query processing with the use of multi-core architectures. To this end, a GPU-base R-tree has been recently proposed. However, even though a GPU-based R-tree can exhibit an improvement in query performance, it is limited in its ability to handle large volumes of data because GPUs have limited physical memory. To address this problem, we propose MGR-tree (Multi-GPU R-tree), which can manage large volumes of data by dividing nodes into multiple GPUs. Our experiments show that MGR-tree is up to 9.1 times faster than a sequential search on a GPU and up to 1.6 times faster than a conventional GPU-based R-tree.

Efficient Reverse Top-<italic>k</italic> Boolean Spatial Keyword Queries on Road Networks

Reverse k nearest neighbor (R k NN) queries have a broad application base such as decision support, profile-based marketing, and resource allocation. Previous work on R k NN search does not take textual information into consideration or limits to the Euclidean space. In the real world, however, most spatial objects are associated with textual information and lie on road networks. In this paper, we introduce a new type of queries, namely, reverse top-k Boolean spatial keyword (R k BSK) retrieval , which assumes objects are on the road network and considers both spatial and textual information. Given a data set P on a road network and a query point q with a set of keywords, an R k BSK query retrieves the points in P that have q as one of answer points for their top- k Boolean spatial keyword queries. We formalize the R k BSK query and then propose filter-and-refinement framework based algorithms for answering R k BSK search with arbitrary k and no any pre-computation . To accelerate the query process, several novel pruning heuristics that utilize both spatial and textual information are employed to shrink the search space efficiently. In addition, a new data structure called count tree has been developed to further improve query performance. A comprehensive experimental evaluation using both real and synthetic data sets demonstrates the effectiveness of our presented pruning heuristics and the performance of our proposed algorithms.

Similarity Joins: Their implementation and interactions with other database operators

Similarity Joins are extensively used in multiple application domains and are recognized among the most useful data processing and analysis operations. They retrieve all data pairs whose distances are smaller than a predefined threshold ε. While several standalone implementations have been proposed, very little work has addressed the implementation of Similarity Joins as physical database operators. In this paper, we focus on the study, design, implementation, and optimization of a Similarity Join database operator for metric spaces. We present DBSimJoin, a physical database operator that integrates techniques to: enable a non-blocking behavior, prioritize the early generation of results, and fully support the database iterator interface. The proposed operator can be used with multiple distance functions and data types. We describe the changes in each query engine module to implement DBSimJoin and provide details of our implementation in PostgreSQL. We also study ways in which DBSimJoin can be combined with other similarity and non-similarity operators to answer more complex queries, and how DBSimJoin can be used in query transformation rules to improve query performance. The extensive performance evaluation shows that DBSimJoin significantly outperforms alternative approaches and scales very well when important parameters like ε, data size, and number of dimensions increase.

Privacy-Preserving Top-k Spatial Keyword Queries in Untrusted Cloud Environments

With the rapid development of location-based services in mobile Internet, spatial keyword queries have been widely employed in various real-life applications in recent years. To realize the great flexibility and cost savings, more and more data owners are motivated to outsource their spatio-textual data services to the cloud. However, directly outsourcing such services to the untrusted cloud may arise serious privacy concerns. In this paper, we study the privacy-preserving top- $k$ spatial keyword query problem in untrusted cloud environments. Existing studies primarily focus on the design of privacy-preserving schemes for either spatial or keyword queries, and they cannot be applied to solve the privacy-preserving spatial keyword query problem. To address this problem, we present a novel privacy-preserving top- $k$ spatial keyword query scheme. In particular, we build an encrypted tree index to facilitate privacy-preserving top- $k$ spatial keyword queries, where spatial and textual data are encrypted in a unified way. To search with the encrypted tree index, we propose two effective techniques for the similarity computations between queries and tree nodes under encryption. To improve query performance on large-scale spatio-textual data, we further propose a keyword-based secure pruning method. Thorough analysis shows the validity and security of our scheme. Extensive experimental results on real datasets demonstrate our scheme achieves high efficiency and good scalability.

Disk-Based Management of Interaction Graphs

In our increasingly connected and instrumented world, live data recording the interactions between people, systems, and the environment is available in various domains, such as telecommunications and social media. This data often takes the form of a temporally evolving graph, where entities are the vertices and the interactions between them are the edges. An important feature of this graph is that the number of edges it has grows continuously, as new interactions take place. We call such graphs interaction graphs. In this paper we study the problem of storing interaction graphs such that temporal queries on them can be answered efficiently. Since interaction graphs are append-only and edges are added continuously, traditional graph layout and storage algorithms that are batch based cannot be applied directly. We present the design and implementation of a system that caches recent interactions in memory, while quickly placing the expired interactions to disk blocks such that those edges that are likely to be accessed together are placed together. We develop live block formation algorithms that are fast, yet can take advantage of temporal and spatial locality among the edges to optimize the storage layout with the goal of improving query performance. We evaluate the system on synthetic as well as real-world interaction graphs, and show that our block formation algorithms are effective for answering temporal neighborhood queries on the graph. Such queries form a foundation for building more complex online and offline temporal analytics on interaction graphs.

A Map-Reduce based parallel approach for improving query performance in a geospatial semantic web for disaster response

Rapid retrieval of spatial information is critical to ensure that emergency supplies and resources can reach the impacted areas in the most efficient manner. However, it remains challenging to find out the needed spatial information efficiently because of the intensive geocomputation processes involved and the heterogeneity of spatial data. It is quite cost prohibitive to query the spatial information from geographical knowledge bases containing complex topological relationships. This research introduces a Map-Reduce based parallel approach for improving the query performance of a geospatial ontology for disaster response. The approach focuses on parallelizing the spatial join computations of GeoSPARQL queries. The proposed parallel approach makes full use of data/task parallelism for spatial queries. The results of some initial experiments show that the proposed approach can reduce individual spatial query execution time by taking advantage of parallel processes. The proposed approach, therefore, may afford a large number of concurrent spatial queries in disaster response applications.

Protecting Encrypted Data against Inference Attacks in Outsourced Databases

Ensuring data privacy and improving query performance are two closely linked challenges for outsourced databases. Using mixed encryption methods to data attributes can reach an explicit trade-off between these two challenges. However, encryption cannot always conceal relations between attributes values. When the data tuples are accessed selectively, inferences based on comparing encrypted values could be launched and sensitive values may be disclosed. In this paper, we explore the attribute based inferences in mixed encrypted databases. We develop a method to construct private indexes on encrypted values to defend against inference while supporting efficient selective access to encrypted data. We have conducted some experiments to validate our proposed method.