Query Processing Framework Research Articles

An Efficient Framework for Secure Dynamic Skyline Query Processing in the Cloud

AbstractThis study introduces an innovative framework named scale for processing dynamic skyline queries securely in cloud environments. Unlike previous approaches that require complex operations on encrypted data, scale simplifies dynamic skyline domination to mere comparisons, significantly improving query efficiency. Through empirical evaluations over four datasets, we show that scale accelerates query processing nearly 1000-fold compared to existing state-of-the-art methods. Specifically, scale shows significant efficiency improvements by simplifying query interactions to a single round between the user and the cloud, which is validated through empirical studies on multiple datasets. Moreover, we introduce two distributed versions of scale, dist-scale-s and dist-scale-e, which further optimize performance by facilitating parallel processing. This adaptation showcases a substantial reduction in response times and computational overhead, underpinning the scalability and effectiveness of our framework in handling large-scale, secure cloud-based queries.

Contact Tracing Over Uncertain Indoor Positioning Data

Pandemics often cause dramatic losses of human lives and impact our societies in many aspects such as public health, tourism, and economy. To contain the spread of an epidemic like COVID-19, efficient and effective contact tracing is important, especially in indoor venues where the risk of infection is higher. In this work, we formulate and study a novel query called Indoor Contact Query ( <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ICQ</small> ) over raw, uncertain indoor positioning data that digitalizes people's movements indoors. Given a query object <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$o$</tex-math></inline-formula> , e.g., a person confirmed to be a virus carrier, an <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ICQ</small> analyzes uncertain indoor positioning data to find objects that most likely had close contact with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$o$</tex-math></inline-formula> for a long period of time. To process <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ICQ</small> , we propose a set of techniques. First, we design an enhanced indoor graph model to organize different types of data necessary for <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ICQ</small> . Second, for indoor moving objects, we devise methods to determine uncertain regions and to derive positioning samples missing in the raw data. Third, we propose a query processing framework with a close contact determination method, a search algorithm, and the acceleration strategies. We conduct extensive experiments on synthetic and real datasets to evaluate our proposals. The results demonstrate the efficiency and effectiveness of our proposals.

A Framework for Privacy Preserving Localized Graph Pattern Query Processing

This paper studies privacy preserving graph pattern query services in a cloud computing paradigm. In such a paradigm, data owner stores the large data graph to a powerful cloud hosted by a service provider (SP) and users send their queries to SP for query processing. However, as SP may not always be trusted, the sensitive information of users' queries, importantly, the query structures, should be protected. In this paper, we study how to outsource the localized graph pattern queries (LGPQs) on the SP side with privacy preservation. LGPQs include a rich set of semantics, such as subgraph homomorphism, subgraph isomorphism, and strong simulation, for which each matched graph pattern is located in a subgraph called ball that have a restriction on its size. To provide privacy preserving query service for LGPQs, this paper proposes the first framework, called Prilo, that enables users to privately obtain the query results. To further optimize Prilo, we propose Prilo* that comprises the first bloom filter for trees in the trust execution environment (TEE) on SP, a query-oblivious twiglet-based technique for pruning non-answers, and a secure retrieval scheme of balls that enables user to obtain query results early. We conduct detailed experiments on real world datasets to show that Prilo* is on average 4x faster than the baseline, and meanwhile, preserves query privacy.

Annolog: A Query Processing Framework for Modelling and Reasoning with Annotated Data

&lt;p&gt;Data annotation is the categorization and labelling of data for applications, such as machine learning, artificial intelligence, and data integration. The categorization and labelling are done to achieve a specific use case in relation to solving problems. Existing data annotation systems and modules face imperfections such as knowledge and annotation not being formally integrated, narrow application range, and difficulty to apply on existing database management applications. To analyze and process annotated data, obtain the relationship between different annotations, and capture metainformation in data provenance and probabilistic databases, in this paper, we design a back-end query processing framework as a supplementary interface for the database management system to extend operation to datasets and boost efficiency. The framework utilizes Java language and the MVC model for development to achieve lightweight, cross-platform, and high adaptability identities. The contribution of this paper is mainly reflected in two aspects. The first contribution is to implement query processing, provenance semiring, and semiring homomorphism over annotated data. The second contribution is to combine query processing and provenance with SQL statements in order to enable the database manager to invoke operations to annotation.&lt;/p&gt; &lt;p&gt;&amp;nbsp;&lt;/p&gt;

SGPAC: generalized scalable spatial GroupBy aggregations over complex polygons

This paper studies the spatial group-by query over complex polygons. Given a set of spatial points and a set of polygons, the spatial group-by query returns the number of points that lie within the boundaries of each polygon. Groups are selected from a set of non-overlapping complex polygons, typically in the order of thousands, while the input is a large-scale dataset that contains hundreds of millions or even billions of spatial points. This problem is challenging because real polygons (like counties, cities, postal codes, voting regions, etc.) are described by very complex boundaries. We propose a highly-parallelized query processing framework to efficiently compute the spatial group-by query on highly skewed spatial data. We also propose an effective query optimizer that adaptively assigns the appropriate processing scheme based on the query polygons. Our experimental evaluation with real data and queries has shown significant superiority over all existing techniques.

Efficient Oblivious Query Processing for Range and kNN Queries

Increasingly, individuals and companies adopt a cloud service provider as a primary data and IT infrastructure platform. The remote access of the data inevitably brings the issue of trust. Data encryption is necessary to keep sensitive information secure and private on the cloud. Yet adversaries can still learn valuable information regarding encrypted data by observing data access patterns. To solve such problem, Oblivious RAMs (ORAMs) are proposed to completely hide access patterns. However, most ORAM constructions are expensive and not suitable to deploy in a database for supporting query processing over large data. Furthermore, an ORAM processes queries <i>synchronously</i> , hence, does not provide high throughput for <i>concurrent query processing</i> . In this article, we design a practical <i>oblivious query processing framework</i> to enable efficient query processing over a cloud database. In particular, we focus on processing multiple range and <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> NN queries <i>asynchronously and concurrently with high throughput</i> . The key idea is to integrate indices into ORAM which leverages a suite of optimization techniques (e.g., oblivious batch processing and caching). The effectiveness and efficiency of our oblivious query processing framework is demonstrated through extensive evaluations over large datasets. Our construction shows an order of magnitude speedup in comparison with other baselines.

Spatial-Keyword Skyline Publish/Subscribe Query Processing Over Distributed Sliding Window Streaming Data

Current spatial-keyword publish/subscribe systems need to handle spatial-keyword skyline queries over geo-textual streams to continuously obtain good results. The skyline queries in such systems face two main problems: (1) query problems, because the powerful query capability is required for the strict limit of the response time and the large number of items concerned by the users, and (2) scalability issue, because millions of active users are maintained simultaneously with many network-connected machines. Unfortunately, the current approach is towards static data. Thus, this paper first proposes a distributed skyline query processing framework. Then, we optimize the skyline computing by introducing MF-R <inline-formula><tex-math notation="LaTeX">$^t$</tex-math></inline-formula> -tree, which is an update-efficient and space-saving indexing structure and a fast approach for processing a continuous spatial-keyword skyline query called <inline-formula><tex-math notation="LaTeX">$eager^*$</tex-math></inline-formula> . Finally, a spatial and textual signature-based communication optimization method is proposed to support scalability. The experimental results indicate that (1) MF-R <inline-formula><tex-math notation="LaTeX">$^t$</tex-math></inline-formula> -tree can significantly reduce update costs, while maintaining a low storage cost, and a query performance comparable to IL-Quadtree, (2) <inline-formula><tex-math notation="LaTeX">$eager^*$</tex-math></inline-formula> can averagely accelerate 79.72 × faster than the method based on BNL, (3) the communication optimization method significantly reduces the communication cost, and (4) the distributed framework can efficiently support large-scale skyline queries.

An NVM SSD-based High Performance Query Processing Framework for Search Engines

Commercial search engines generally maintain hundreds of thousands of machines equipped with large sized DRAM which incurs high hardware cost since DRAM is expensive. Recently, NVM Optane SSD has been considered as a promising underlying storage device due to its price advantage and speed advantage. However, to achieve a comparable efficiency performance with in-memory index, applying NVM to both latency and I/O bandwidth critical applications still face non-trivial challenges, because NVM has much lower I/O speed and bandwidth compared to DRAM. In this paper, we propose an NVM SSD-optimized query processing framework, aiming to address both the latency and bandwidth issues of using NVM in search engines. First, we propose a pipelined query processing methodology which significantly reduces the I/O waiting time. Second, we propose a cache-aware query reordering algorithm which enables queries sharing more data to be processed adjacently. Third, we propose a data prefetching mechanism which reduces the extra thread waiting time and improves bandwidth utilization. Moreover, we propose intra-query parallel mechanisms for long-tail queries, including query subtask scheduling, heap concurrent access strategy, query parallelism prediction and adaptive pipelining. Extensive experimental studies show that our framework significantly outperforms the state-of-the-art baselines, which obtains comparable processing latency and throughput with DRAM in both inter-query and intra-query parallel scenarios.

Amnis: Optimized stream processing for edge computing

The proliferation of Internet-of-Things (IoT) devices is rapidly increasing the demands for efficient processing of low latency stream data generated close to the edge of the network. Edge computing-based stream processing techniques that carefully consider the heterogeneity of the computational and network resources available in the infrastructure provide significant benefits in optimizing the throughput and end-to-end latency of the data streams. In this paper, we propose a novel stream query processing framework called Amnis that optimizes the performance of the stream processing applications through a careful allocation of computational and network resources available at the edge. The Amnis approach differentiates itself through its consideration of data locality and resource constraints during physical plan generation and operator placement for the stream queries. Additionally, Amnis considers the coflow dependencies to optimize the network resource allocation through an application-level rate control mechanism. We implement a prototype of Amnis in Apache Storm. Our performance evaluation carried out in a real testbed shows that the proposed techniques achieve as much as 200X improvement on the end-to-end latency and 10X improvement on the overall throughput compared to the default resource aware scheduler in Storm.

Reference-Based Framework for Spatio-Temporal Trajectory Compression and Query Processing

The pervasiveness of GPS-enabled devices and wireless communication technologies results in massive trajectory data, incurring expensive cost for storage, transmission, and query processing. To relieve this problem, in this paper we propose a novel framework for compressing trajectory data, REST ( Re ference-based S patio- t emporal trajectory compression), by which a raw trajectory is represented by concatenation of a series of historical (sub-)trajectories (called reference trajectories) that form the compressed trajectory within a given spatio-temporal deviation threshold. In order to construct a reference trajectory set that can most benefit the subsequent compression, we propose three kinds of techniques to select reference trajectories wisely from a large dataset such that the resulting reference set is more compact yet covering most footprints of trajectories in the area of interest. To address the computational issue caused by the large number of combinations of reference trajectories that may exist for resembling a given trajectory, we propose efficient greedy algorithms that run in the blink of an eye and dynamic programming algorithms that can achieve the optimal compression ratio. Compared to existing work on trajectory compression, our framework has few assumptions about data such as moving within a road network or moving with constant direction and speed, and better compression performance with fairly small spatio-temporal loss. In addition, by indexing the reference trajectories directly with an in-memory R-tree and building connections to the raw trajectories with inverted index, we develop an extremely efficient algorithm that can answer spatio-temporal range queries over trajectories in their compressed form. Extensive experiments on a real taxi trajectory dataset demonstrate the superiority of our framework over existing representative approaches in terms of both compression ratio and efficiency.

RapidMatch

A subgraph query searches for all embeddings in a data graph that are identical to a query graph. Two kinds of algorithms, either graph exploration based or join based, have been developed for processing subgraph queries. Due to algorithmic and implementational differences, join-based systems can handle query graphs of a few vertices efficiently whereas exploration-based approaches typically process up to several tens of vertices in the query graph. In this paper, we first compare these two kinds of methods and prove that the complexity of result enumeration in state-of-the-art exploration-based methods matches that of the worst-case optimal join. Furthermore, we propose RapidMatch, a holistic subgraph query processing framework integrating the two approaches. Specifically, RapidMatch not only runs relational operators such as selections and joins, but also utilizes graph structural information, as in graph exploration, for filtering and join plan generation. Consequently, it outperforms the state of the art in both approaches on a wide range of query workloads.

A Query Processing Framework for Efficient Network Resource Utilization in Shared Sensor Networks

Shared Sensor Network (SSN) refers to a scenario where the same sensing and communication resources are shared and queried by multiple Internet applications. Due to the burgeoning growth in Internet applications, multiple application queries can exhibit overlapping in their functional requirements, such as the region of interest, sensing attributes, and sensing time duration. This overlapping results in redundant sensing tasks generation leading to the increased overall network traffic and energy consumption. Existing approaches operate on data sharing among various tasks to minimize the upstream traffic. However, no existing work attempts to prevent the redundant task generation to reduce the downstream traffic. Moreover, the allocation of suitable sensor nodes to meet the Quality of Service (QoS) requirements of the queries is still an open issue. This article proposes an end-to-end query processing framework (named, QueryPM) that first, calculates the functional requirements similarity among queries to prevent the redundant task generation. Then, it takes the QoS and functional requirements into account while allocating the tasks on the sensor nodes. Extensive simulations on the proposed approach show that downstream traffic, upstream traffic, and energy consumption reduced to 60%, 20--40%, and 40%, respectively, as compared to state-of-the-art mechanisms.

POLARIS

In this paper, we describe the Polaris distributed SQL query engine in Azure Synapse. It is the result of a multi-year project to re-architect the query processing framework in the SQL DW parallel data warehouse service, and addresses two main goals: (i) converge data warehousing and big data workloads, and (ii) separate compute and state for cloud-native execution. From a customer perspective, these goals translate into many useful features, including the ability to resize live workloads, deliver predictable performance at scale, and to efficiently handle both relational and unstructured data. Achieving these goals required many innovations, including a novel "cell" data abstraction, and flexible, fine-grained, task monitoring and scheduling capable of handling partial query restarts and PB-scale execution. Most importantly, while we develop a completely new scale-out framework, it is fully compatible with T-SQL and leverages decades of investment in the SQL Server single-node runtime and query optimizer. The scalability of the system is highlighted by a 1PB scale run of all 22 TPC-H queries; to our knowledge, this is the first reported run with scale larger than 100TB.

G-PICS: A Framework for GPU-Based Spatial Indexing and Query Processing

Support for efficient spatial data storage and retrieval has become a vital component in almost all spatial database systems. While GPUs have become a mainstream platform for high-throughput data processing in recent years, exploiting the massively parallel processing power of GPUs is non-trivial. Current approaches that parallelize one query at a time have low work efficiency and cannot make good use of GPU resources. On the other hand, many spatial database systems could receive a large number of queries simultaneously. In this paper, we present a comprehensive framework named G-PICS for parallel processing of concurrent spatial queries on GPUs. G-PICS encapsulates efficient parallel algorithms for constructing a variety of spatial trees with different space partitioning methods. G-PICS also provides highly optimized programs for processing major spatial query types, and such programs can be accessed via an API that could be further extended to implement user-defined algorithms. While support for dynamic data inputs is missing in existing work, G-PICS implements efficient parallel algorithms for bulk updates of data. Furthermore, G-PICS is designed to work in a Multi-GPU environment to support datasets beyond the size of a single GPU’s global memory. Empirical evaluation of G-PICS shows significant performance improvement over the state-of-the-art GPU and parallel CPU-based spatial query processing systems. In particular, G-PICS achieves double-digit speedup over such systems in tree construction (up to 53X) and query processing (up to 80X).

Efficient Path Query Processing Over Massive Trajectories on the Cloud

A path query aims to find trajectories passing a given sequence of connected road segments within a time period. It is very useful in many urban applications: 1) traffic modeling, 2) frequent path mining, 3) intersection coordination, and 4) traffic anomaly detection. Existing solutions for path query processing are implemented based on single machines, which are not efficient for the following tasks: 1) indexing large-scale historical data; 2) handling real-time trajectory updates; and 3) processing concurrent path queries from urban data mining applications. In this paper, we design and implement a cloud-based path query processing framework based on Microsoft Azure. We modify existing suffix tree structure to index trajectories using Azure Table. The proposed system consists of two main parts: 1) back-end processing , which performs pre-processing (i.e., parsing and map-matching) and index building tasks with a distributed computing platform (i.e., Storm) used to efficiently handle massive real-time trajectory updates; and 2) query processing , which answers path queries using Azure Storm to improve efficiency and overcome I/O bottleneck. Extensive experiments are performed based on the real-time taxi trajectories from Guiyang City, the capital of Guizhou Province, China to confirm the system efficiency. We also demonstrate a real deployed traffic analysis system based on our query processing framework.

Bounded Approximate Query Processing

OLAP is a core functionality in database systems and the performance is crucial to enable on-time decisions. However, OLAP queries are rather time consuming, especially on large datasets, and traditional exact solutions usually cannot meet the high-performance requirement. Recently, approximate query processing (AQP) has been proposed to enable approximate OLAP. However, existing AQP methods have some limitations. First, they may involve unacceptable errors on skewed data (e.g., long-tail distribution). Second, they require to store large amount of data and have no significant performance improvement. Third, they only support a small subset of SQL aggregation queries. To overcome these limitations, we propose a bounded approximate query processing framework ${\mathtt {BAQ}}$ BAQ . Given a predefined error bound and a set of queries, ${\mathtt {BAQ}}$ BAQ judiciously selects high-quality samples from the data to generate a unified synopsis offline, and then uses the synopsis to answer online queries. Compared with existing methods, ${\mathtt {BAQ}}$ BAQ has the following salient features. (1) ${\mathtt {BAQ}}$ BAQ does not need to generate a synopsis for each query while it only generates a unified synopsis, and thus ${\mathtt {BAQ}}$ BAQ has much smaller synopsis. (2) ${\mathtt {BAQ}}$ BAQ achieves much smaller error than existing studies. Specifically, ${\mathtt {BAQ}}$ BAQ can provide deterministic approximate results (i.e., the estimated query results must be within the error bound with 100 percent confidence) for SQL aggregation queries that do not contain selection conditions on numerical columns. For queries with selection conditions on numerical columns, we propose effective grouping-based techniques and the estimated results are also within the error bound in practice. Experimental results on both real and synthetic datasets show that ${\mathtt {BAQ}}$ BAQ significantly outperforms state-of-the-art approaches. For example, on a Microsoft production dataset (a real dataset with synthetic queries), ${\mathtt {BAQ}}$ BAQ has 10-100× improvement on synopsis size and 10-100× improvement on the error compared with state-of-the-art algorithms.

Secure string pattern query for open data initiative

The open data initiative allows the organizations from different countries to share and integrate their data to produce value added information products and services. However, the trust relationship between any two organizations can hardly be established due to the data sensitivity and their specific responsibilities. In this paper, we propose a secure string pattern query processing framework for multiple data owners. With this framework, we design an efficient and secure indexing structure called Secure String PAttern Search tree (S2PAStree). S2PAStree allows the execution of the secure string pattern query with logarithmic time complexity. We further propose a set of secure index construction protocols under the scenario that multiple data owners share and integrate their sensitive data. Finally, we conduct comprehensive experiments on three public datasets to evaluate the efficiency and the scalability of our solutions.

A Robotic Context Query-Processing Framework Based on Spatio-Temporal Context Ontology.

Service robots operating in indoor environments should recognize dynamic changes from sensors, such as RGB-depth (RGB-D) cameras, and recall the past context. Therefore, we propose a context query-processing framework, comprising spatio-temporal robotic context query language (ST-RCQL) and a spatio-temporal robotic context query-processing system (ST-RCQP), for service robots. We designed them based on spatio-temporal context ontology. ST-RCQL can query not only the current context knowledge, but also the past. In addition, ST-RCQL includes a variety of time operators and time constants; thus, queries can be written very efficiently. The ST-RCQP is a query-processing system equipped with a perception handler, working memory, and backward reasoner for real-time query-processing. Moreover, ST-RCQP accelerates query-processing speed by building a spatio-temporal index in the working memory, where percepts are stored. Through various qualitative and quantitative experiments, we demonstrate the high efficiency and performance of the proposed context query-processing framework.

Distributed Join Query Processing for Big RDF Data

The expansion of the services of the Semantic Web and the evolution of cloud computing technologies have significantly enhanced the capability of preserving and publishing information in standard open web formats, such that data can be both human-readable and machine-processable. This situation meets the challenge in the current big data era to effectively store, retrieve, and analyze resource description framework (RDF) data in swarms. Moreover, efficient data storage and retrieval that can scale to large amounts of possibly schema-less data have proven to be quite difficult to achieve, specifically, RDF data storage with complex and large graph patterns for representing semantic data, and SPARQL query languages. In this paper, we provide comprehensive discussion about the proposed algorithms of Join.Query processing of RDF data by considering MapReduce Framework in a distributed environment. Moreover, we introduced a framework for RDF query processing and the benchmark that is used for the performance evaluation. Finally, we offer an evaluation discussion on distributed join query processing for big RDF data.

CPP: Towards comprehensive privacy preserving for query processing in information networks

This paper addresses the privacy issue for query processing in information networks, where users search and aggregate information from many data sources. To cater for the privacy requirements of all parties, we propose a comprehensive privacy preserving framework (CPP in short) for preserving data privacy, query privacy and storage privacy simultaneously, and implement this framework for key-value query processing. We first develop a baseline scheme that adopts commutative encryption with full indistinguishability guarantee. To speedup query processing, we then propose a tradeoff between security and efficiency, which leads to a scheme that significantly reduces the use of commutative encryption with a little and bounded security cost. Finally, we validate the proposed framework and the implementation schemes by both theoretical analysis and experimental results.