Efficient Query Processing Research Articles

An Efficient Indexing Method for Skyline Computations with Partially Ordered Domains

Efficient processing of skyline queries with partially ordered domains has been intensively addressed in recent years. To further reduce the query processing time to support high-responsive applications, the skyline queries that were previously processed with user preferences similar to those of the new query contribute useful candidate result points. Hence, the answered queries can be cached with both their results and the user preferences such that the query processor can rapidly retrieve the result for a new query only from the result sets of cached queries with compatible user preferences. When caching a significant number of queries accumulated over time, it is essential to adopt effective access methods to index the cached queries to retrieve a set of relevant cached queries for facilitating the cache-based skyline query computations. In this paper, we propose an extended depth-first search indexing method (e-DFS for short) for accessing user preference profiles represented by directed acyclic graphs (DAGs), and emphasize the design of the e-DFS encoding that effectively encodes a user preference profile into a low-dimensional feature point which is eventually indexed by an R-tree. We obtain one or more traversal orders for each node in a DAG by traversing it through a modified version of the depth-first search which is utilized to examine the topology structure and dominance relations to measure closeness or similarity. As a result, e-DFS which combines the criteria of similarity evaluation is able to greatly reduce the search space by filtering out most of the irrelevant cached queries such that the query processor can avoid accessing the entire data set to compute the query results. Extensive experiments are presented to demonstrate the performance and utility of our indexing method, which outperforms the baseline planning techniques by reducing 37 percent of the computational time on average.

Attribute-driven community search

Recently, community search over graphs has gained significant interest. In applications such as analysis of protein-protein interaction (PPI) networks, citation graphs, and collaboration networks, nodes tend to have attributes. Unfortunately, most previous community search algorithms ignore attributes and result in communities with poor cohesion w.r.t. their node attributes. In this paper, we study the problem of attribute-driven community search, that is, given an undirected graph G where nodes are associated with attributes, and an input query Q consisting of nodes V q and attributes W q , find the communities containing V q , in which most community members are densely inter-connected and have similar attributes. We formulate this problem as finding attributed truss communities (ATC), i.e., finding connected and close k-truss subgraphs containing V q , with the largest attribute relevance score. We design a framework of desirable properties that good score function should satisfy. We show that the problem is NP-hard. However, we develop an efficient greedy algorithmic framework to iteratively remove nodes with the least popular attributes, and shrink the graph into an ATC. In addition, we also build an elegant index to maintain k -truss structure and attribute information, and propose efficient query processing algorithms. Extensive experiments on large real-world networks with ground-truth communities show that our algorithms significantly outperform the state of the art and demonstrates their efficiency and effectiveness.

Issues in complex event processing: Status and prospects in the Big Data era

Research issues in complex event processing (CEP) emphasizing on query optimization.Cover deterministic probabilistic models, centralized distributed settings.Issues for CEP optimization over Big Data enabling cloud computing platforms.Predictive Analytics and CEP in cloud platforms even with dispersed resource pools. Many Big Data technologies were built to enable the processing of human generated data, setting aside the enormous amount of data generated from Machine-to-Machine (M2M) interactions and Internet-of-Things (IoT) platforms. Such interactions create real-time data streams that are much more structured, often in the form of series of event occurrences. In this paper, we provide an overview on the main research issues confronted by existing Complex Event Processing (CEP) techniques, with an emphasis on query optimization aspects. Our study expands on both deterministic and probabilistic event models and spans from centralized to distributed network settings. In that, we cover a wide range of approaches in the CEP domain and review the current status of techniques that tackle efficient query processing. These techniques serve as a starting point for developing Big Data oriented CEP applications. Therefore, we further study the issues that arise upon trying to apply those techniques over Big Data enabling technologies, as is the case with cloud platforms. Furthermore, we expand on the synergies among Predictive Analytics and CEP with an emphasis on scalability and elasticity considerations in cloud platforms with potentially dispersed resource pools.

On Efficient Processing of Continuous Reverse Skyline Queries in Wireless Sensor Networks

The reverse skyline query plays an important role in information searching applications. This paper deals with continuous reverse skyline queries in sensor networks, which retrieves reverse skylines as well as the set of nodes that reported them for continuous sampling epochs. Designing an energy-efficient approach to answer continuous reverse skyline queries is non-trivial because the reverse skyline query is not decomposable and a huge number of unqualified nodes need to report their sensor readings. In this paper, we develop a new algorithm that avoids transmission of updates from nodes that cannot influence the reverse skyline. We propose a data mapping scheme to estimate sensor readings and determine their dominance relationships without having to know the true values. We also theoretically analyze the properties for reverse skyline computation, and propose efficient pruning techniques while guaranteeing the correctness of the answer. An extensive experimental evaluation demonstrates the efficiency of our approach.

Efficient XML query and update processing using a novel prime-based middle fraction labeling scheme

XML data can be represented by a tree or graph and the query processing for XML data requires the structural information among nodes. Designing an efficient labeling scheme for the nodes of Order-Sensitive XML trees is one of the important methods to obtain the excellent management of XML data. Previous labeling schemes such as region and prefix often sacrifice updating performance and suffer increasing labeling space when inserting new nodes. To overcome these limitations, in this paper we propose a new labeling idea of separating structure from order. According to the proposed idea, a novel Prime-based Middle Fraction Labeling Scheme (PMFLS) is designed accordingly, in which a series of algorithms are proposed to obtain the structural relationships among nodes and to support updates. PMFLS combines the advantages of both prefix and region schemes in which the structural information and sequential information are separately expressed. PMFLS also supports Order-Sensitive updates without relabeling or recalculation, and its labeling space is stable. Experiments and analysis on several benchmarks are conducted and the results show that PMFLS is efficient in handling updates and also significantly improves the performance of the query processing with good scalability.

QTwig: A Structural Join Algorithm for Efficient Query Retrieval Based on Region-Based Labeling

Dependability on XML has increased tremendously over the years. As such the need for efficient query processing technique is certainly important. Despite the fact that these techniques are able to process queries with various edge combinations, they still suffer from processing overheads by buffering large amount of intermediate results particularly for parent–child (P–C) edges. Therefore, in this paper, we propose architecture named ReLaQ, which comprises of two components, ReLab[Formula: see text] (node annotator) and QTwig (query processor) for efficient XML query processing. QTwig improves retrieval time by incorporating a pruning technique that avoids accessing irrelevant data during query processing. Experimental results indicated that ReLaQ superseded TwigStack for both path and twig queries using both regular- and skewed-structured datasets. In addition, this is also proven by means of correctness analysis of ReLaQ.

Dynamic spatial index for efficient query processing on the cloud

Data owners with large volumes of data can outsource spatial databases by taking advantage of the cost-effective cloud computing model with attractive on-demand features such as scalability and high computing power. Data confidentiality in outsourced databases is a key requirement and therefore, untrusted third-party service providers in the cloud should not be able to view or manipulate the data. This paper proposes DISC (Dynamic Index for Spatial data on the Cloud), a secure retrieval scheme to answer range queries over encrypted databases at the Cloud Service Provider. The dynamic spatial index is also able to support dynamic updates on the outsourced data at the cloud server. To be able to support secure query processing and updates on the Cloud, spatial transformation is applied to the data and the spatial index is encrypted using Order-Preserving Encryption. With transformation and cryptography techniques, DISC achieves a balance between efficient query execution and data confidentiality in a cloud environment. Additionally, a more secure scheme, DISC ∗, is proposed to balance the trade-off between query results returned and security provided. The security analysis section studies the various attacks handled by DISC. The experimental study demonstrates that the proposed scheme achieves a lower communication cost in comparison to existing cloud retrieval schemes.

An efficient continuous range query processing scheme in mobile P2P networks

With the development of mobile devices, interest in the mobile P2P-based services has increased. However, the existing schemes spend too much cost to process a continuous range query in mobile P2P network environments. In this paper, we propose a new efficient range query processing scheme in mobile P2P network environments. The proposed scheme consists of two phases. In the query distribution phase, it prunes the peers that are impossible to be included in the query result. In the monitoring phase, the proposed scheme updates the query result incrementally. It is shown through performance evaluation that the proposed scheme outperforms the existing scheme in terms of the number of messages.

Adaptive indexing approach for main memory column store

Owing to efficient query processing for random workload, the hybrid crack sort (HCS) has become an important adaptive indexing approach in main-memory column store. However, under sequential workload scenarios, the HCS does not obtain a good query execution performance, because of great reorganisation overhead imposed on the initial queries. The authors propose a hybrid radix crack sort (HRCS) approach to solve this problem. By the adoption of radix-based partition strategy, it divides the unsorted column into disjoint key ranges and then conducts data reorganisation in at most two key ranges for each query. For HRCS, only a small portion of the whole column needs to be touched for the processing of each query, thus reducing the reorganisation cost and improving the query execution performance. The final experiments show that the novel HRCS approach can obtain a higher query execution performance for not only random workload but also sequential workload, as compared with HCS.

Location-Based Aggregate Queries for Heterogeneous Neighboring Objects

Currently, most of the processing techniques for the conventional location-based queries focus only on a single type of objects. However, in real-life applications, the user may be interested in obtaining information about different types of objects, in terms of their neighboring relationship. We term the different types of stationary objects closer to each other the heterogeneous neighboring objects ( HNOs ). Efficient processing of the location-based queries on the HNOs is more complicated than that on a single data source, because the neighboring relationship between the HNOs inevitably affects the query result. In this paper, we present useful and important location-based aggregate queries on the HNOs , which can provide useful object information by considering both the spatial closeness of objects to the query object and the neighboring relationship between objects. The location-based aggregate queries consist of four queries: the shortest average-distance ( SAvgD ) query, the shortest minimal-distance ( SMinD ) query, the shortest maximal-distance ( SMaxD ) query, and the shortest sum-distance ( SSumD ) query. To process the location-based aggregate queries, we devise two heuristics, the HNOs-qualifying heuristic and the HNOs-pruning heuristic , to efficiently determine the HNOs sets. According to different query types, we further propose four heuristics, the SAvgD-pruning heuristic , the SMinD-pruning heuristic , the SMaxD-pruning heuristic , and the SSumD-pruning heuristic , to effectively reduce the number of distance computations required for query processing. Comprehensive experiments are conducted to demonstrate the effectiveness of the heuristics and the efficiency of the proposed approaches.

Efficient Graph Similarity Search in External Memory

Many real-world applications, such as bioinformatics, data mining, pattern recognition, and social network analysis, benefit from efficient solutions for the graph similarity search problem. Existing methods have limited scalability when they handle the large graph databases, for example, those with millions or billions of graphs that cannot fit in main memory. In this paper, we study the problem of graph similarity search under the graph edit distance constraint in external memory. We present an efficient framework for arbitrary $q$ - ${gram}$ -based representations of a graph. Specifically, we propose a $q$ - gram matrix index stored in hybrid layout in external memory to achieve efficient query processing, by converting the $q$ - gram counting filter into a sparse matrix-vector multiplication problem. Furthermore, we also boost the query performance by transforming the global filter to a 2-D query rectangle, which allows us to perform a query in a reduced region, significantly reducing the number of query I/Os in practice. Extensive experiments on real data sets confirm that 1) our method can compete with the state-of-the-art in-memory methods in index size and filtering ability, and outperform them on scalability of coping with the PubChem data set including 25 million chemical structure graphs and 2) compared with the popular $q$ - gram -based external inverted index, our external index structure needs much fewer number of query I/Os on the PubChem data set.

Distributed Query Plan Generation using Cuckoo Search Algorithm

Query processing in distributed databases involves data transmission amongst sites capable of providing answers to a distributed query. For this, a distributed query processing strategy, which generates efficient query processing plans for a given distributed query, needs to be devised. Since in distributed databases, the data is fragmented and replicated at multiple sites, the number of query plans increases exponentially with increase in the number of sites capable of providing answers to a distributed query. As a result, generating efficient query processing plans, from amongst all possible query plans, becomes a complex problem. This distributed query plan generation (DQPG) problem has been addressed using the Cuckoo Search Algorithm (CSA) in this paper. Accordingly, a CSA based DQPG algorithm (DQPGCSA) that aims to generate Top-K query plans having minimum cost of processing a distributed query has been proposed. Experimental based comparison of DQPGCSA with the existing GA based DQPG algorithm shows that the former is able to generate Top-K query plans that have a comparatively lower query processing cost. This, in turn, reduces the query response time resulting in efficient decision making.

An Indexing Method to Construct Unbalanced Layers for High-Dimensional Data in Mobile Environments

A top-k query processing is widely used in many applications and mobile environments. An index is used for efficient query processing and layer-based indexing methods are representative to perform the top-k query processing efficiently. However, the existing methods have a problem of high index building time for multidimensional and large data; thus, it is difficult to use them. In this paper, we proposed a new concept of constructing layer-based index, which is called unbalanced layer (UB-Layer). The existing methods construct a layer as a balanced layer with outermost data and wrap the rest of the input data. However, UB-Layer constructs a layer as an unbalanced layer that does not wrap the rest of the data. To construct UB-Layer, we fist divide the dimension of the input data into divided-dimensional data and compute the convex hull in each divided-dimensional data. And then, we combine divided-convex hull to build UB-Layer. We also propose UB-SelectAttribute algorithm for dividing the dimension with major attributes. We demonstrate the superiority of the proposed methods by the performance experiments.

Querying of Disparate Association and Interaction Data in Biomedical Applications.

In biomedical applications, network models are commonly used to represent interactions and higher-level associations among biological entities. Integrated analyses of these interaction and association data has proven useful in extracting knowledge, and generating novel hypotheses for biomedical research. However, since most datasets provide their own schema and query interface, opportunities for exploratory and integrative querying of disparate data are currently limited. In this study, we utilize RDF-based representations of biomedical interaction and association data to develop a querying framework that enables flexible specification and efficient processing of graph template matching queries. The proposed framework enables integrative querying of biomedical databases to discover complex patterns of associations among a diverse range of biological entities, including biomolecules, biological processes, organisms, and phenotypes. Our experimental results on the UniProt dataset show that the proposed framework can be used to efficiently process complex queries, and identify biologically relevant patterns of associations that cannot be readily obtained by querying each dataset independently.

PANDA: toward partial topology-based search on large networks in a single machine

A large body of research has focused on efficient and scalable processing of subgraph search queries on large networks. In these efforts, a query is posed in the form of a connected query graph. Unfortunately, in practice end users may not always have precise knowledge about the topological relationships between nodes in a query graph to formulate a connected query. In this paper, we present a novel graph querying paradigm called partial topology-based network search and propose a query processing framework called panda to efficiently process partial topology query (ptq) in a single machine. A ptq is a disconnected query graph containing multiple connected query components. ptqs allow an end user to formulate queries without demanding precise information about the complete topology of a query graph. To this end, we propose an exact and an approximate algorithm called sen-panda and po-panda, respectively, to generate top-kmatches of a ptq. We also present a subgraph simulation-based optimization technique to further speedup the processing of ptqs. Using real-life networks with millions of nodes, we experimentally verify that our proposed algorithms are superior to several baseline techniques.

RQL: A Query Language for Rule Discovery in Databases

Promoting declarative approaches in data mining is a long standing theme, the main idea being to simplify as much as possible the way data analysts interact with their data. This paper goes into this direction by proposing a well-founded logical query language, SafeRL, allowing the expression of a wide variety of rules to be discovered against a database. By rules, we mean statements of the form “if …then …”, as defined in logics for “implications” between boolean variables. As a consequence, SafeRL extends and generalizes functional dependencies to new and unexpected rules. We provide a query rewriting technique and a constructive proof of the main query equivalence theorem, leading to an efficient query processing technique. From SafeRL, we have devised RQL, a user-friendly SQL-like query language. We have shown how a tight integration can be performed on top of any relational database management system. Every RQL query turns out to be seen as a query processing problem, instead of a particular rule mining problem. This approach has been implemented and experimented on sensor network data. A web prototype has been released and is freely available (http://rql.insa-lyon.fr). Data analysts can upload a sample of their data, write their own RQL queries and get answers to know whether or not a rule holds (if not, a counterexample from the database is displayed) and much more.

Private and Efficient Query Processing on Outsourced Genomic Databases.

Applications of genomic studies are spreading rapidly in many domains of science and technology such as healthcare, biomedical research, direct-to-consumer services, and legal and forensic. However, there are a number of obstacles that make it hard to access and process a big genomic database for these applications. First, sequencing genomic sequence is a time consuming and expensive process. Second, it requires large-scale computation and storage systems to process genomic sequences. Third, genomic databases are often owned by different organizations, and thus, not available for public usage. Cloud computing paradigm can be leveraged to facilitate the creation and sharing of big genomic databases for these applications. Genomic data owners can outsource their databases in a centralized cloud server to ease the access of their databases. However, data owners are reluctant to adopt this model, as it requires outsourcing the data to an untrusted cloud service provider that may cause data breaches. In this paper, we propose a privacy-preserving model for outsourcing genomic data to a cloud. The proposed model enables query processing while providing privacy protection of genomic databases. Privacy of the individuals is guaranteed by permuting and adding fake genomic records in the database. These techniques allow cloud to evaluate count and top-k queries securely and efficiently. Experimental results demonstrate that a count and a top-k query over 40 Single Nucleotide Polymorphisms (SNPs) in a database of 20000 records takes around 100 and 150s, respectively.

VP2RQ: Efficient verifiable privacy-preserving range query processing in two-tiered wireless sensor networks

In the field of wireless sensor networks, the secure range query technique is a challenging issue. In two-tiered wireless sensor networks, a verifiable privacy-preserving range query processing method is proposed that is based on bucket partition, information identity authentication, and check-code fusion. During the data collection process, each sensor node puts its collected data into buckets according to the bucket partition strategy, encrypts the non-empty buckets, generates the check-codes for the empty buckets, and fuses them. Then, the check-codes and the encrypted buckets are submitted to the parent node until they reach the storage node. During query processing, the base station converts the queried range into the interested bucket tag set and sends it to the storage node. The storage node determines the candidate-encrypted buckets, generates the check-code through code fusion, and sends them to the base station. The base station obtains query results and verifies the completeness of the result with the check-code. Both the theoretical analysis and experimental results show that verifiable privacy-preserving range query is capable of protecting the privacy sensor data, query result, and query range, which also supports the completeness verification of the query result. Compared to existing methods, verifiable privacy-preserving range query performs better on communication cost.

Scalable 3D Spatial Queries for Analytical Pathology Imaging with MapReduce.

3D analytical pathology imaging examines high resolution 3D image volumes of human tissues to facilitate biomedical research and provide potential effective diagnostic assistance. Such approach - quantitative analysis of large-scale 3D pathology image volumes - generates tremendous amounts of spatially derived 3D micro-anatomic objects, such as 3D blood vessels and nuclei. Spatial exploration of such massive 3D spatial data requires effective and efficient querying methods. In this paper, we present a scalable and efficient 3D spatial query system for querying massive 3D spatial data based on MapReduce. The system provides an on-demand spatial querying engine which can be executed with as many instances as needed on MapReduce at runtime. Our system supports multiple types of spatial queries on MapReduce through 3D spatial data partitioning, customizable 3D spatial query engine, and implicit parallel spatial query execution. We utilize multi-level spatial indexing to achieve efficient query processing, including global partition indexing for data retrieval and on-demand local spatial indexing for spatial query processing. We evaluate our system with two representative queries: 3D spatial joins and 3D k-nearest neighbor query. Our experiments demonstrate that our system scales to large number of computing nodes, and efficiently handles data-intensive 3D spatial queries that are challenging in analytical pathology imaging.

Efficient Processing of Relevant Nearest-Neighbor Queries

Novel Web technologies and resulting applications have led to a participatory data ecosystem that, when utilized properly, will lead to more rewarding services. In this work, we investigate the case of Location-Based Services, specifically how to improve the typical location-based Point-of-Interest (POI) request processed as a k -Nearest-Neighbor query. This work introduces Links-of-Interest (LOI) between POIs as a means to increase the relevance and overall result quality of such queries. By analyzing user-contributed content in the form of travel blogs, we establish the overall popularity of an LOI, that is, how frequently the respective POI pair was visited and is mentioned in the same context. Our contribution is a query-processing method for so-called k -Relevant Nearest Neighbor ( k -RNN) queries that considers spatial proximity in combination with LOI information to retrieve close-by and relevant (as judged by the crowd) POIs. Our method is based on intelligently combining indices for spatial data (a spatial grid) and for relevance data (a graph) during query processing. Using landmarks as a means to prune the search space in the Relevance Graph, we improve the proposed methods. Using in addition A*-directed search, the query performance can be further improved. An experimental evaluation using real and synthetic data establishes that our approach efficiently solves the k -RNN problem.