Query Decomposition Research Articles

DecoPa: Query Decomposition for Parallel Complex Event Processing

Systems for Complex Event Processing (CEP) enable the detection of predefined patterns in event streams. While the evaluation of CEP queries is computationally hard, scalability may be achieved by parallelization. Yet, existing approaches for parallel CEP are driven by static query properties, such as partitioning keys and states of the evaluation model. They largely neglect the rates with which processing units may ingest and compare events for query evaluation. In this paper, we present an approach for parallel CEP that is based on a flexible decomposition of CEP queries. Our idea is to guide the decomposition by the sustainable throughput of each processing unit, in order to maximize the overall performance. To this end, we introduce DecoPa plans for parallel CEP, provide a cost model for them, elaborate on their correctness and optimality, and present an algorithm for their construction. Experiments using a DecoPa implementation in Flink illustrate throughput gains of up to 12 orders of magnitude compared to state-of-the-art approaches.

Datamator: An Authoring Tool for Creating Datamations via Data Query Decomposition

Datamation is designed to animate an analysis pipeline step by step, serving as an intuitive and efficient method for interpreting data analysis outcomes and facilitating easy sharing with others. However, the creation of a datamation is a difficult task that demands expertise in diverse skills. To simplify this task, we introduce Datamator, a language-oriented authoring tool developed to support datamation generation. In this system, we develop a data query analyzer that enables users to generate an initial datamation effortlessly by inputting a data question in natural language. Then, the datamation is displayed in an interactive editor that affords users the ability to both edit the analysis progression and delve into the specifics of each step undertaken. Notably, the Datamator incorporates a novel calibration network that is able to optimize the outputs of the query decomposition network using a small amount of user feedback. To demonstrate the effectiveness of Datamator, we conduct a series of evaluations including performance validation, a controlled user study, and expert interviews.

A Cost-Driven Top-K Queries Optimization Approach on Federated RDF Systems

RDF (Resource Description Framework) is a model widely used to construct knowledge bases, while SPARQL (SPARQL Protocol and RDF Query Language) is the standardized structured query language to manipulate RDF data. Recently, many data providers have published their RDF datasets in their own autonomous sites and provided SPARQL query interfaces, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RDF sources</i> . In order to integrate multiple RDF sources, researchers put forward the federated RDF system to support the federated SPARQL queries. However, existing studies can only support efficient basic queries but not top-k queries. Toward this end, we propose a cost-driven top-k queries optimization approach in federated RDF systems, which can support both top-k queries for single variable ordering and expression ordering. Firstly, we propose an optimized query decomposition method to decompose the federated query into multiple subqueries. Secondly, while considering the top-k operator, we propose a cost model to evaluate the query cost and join cost of subqueries. The optimal query plan can be obtained by the costed-based query plan generation algorithm. Finally, combined with the characteristics of top-k queries, an incremental query plan execution strategy is developed to minimize the total query cost. Experimental results show that the proposed method is effective, efficient and scalable.

Efficient query evaluation techniques over large amount of distributed linked data

As RDF becomes more widely established and the amount of linked data is rapidly increasing, the efficient querying of large amount of data becomes a significant challenge. In this paper, we propose a family of algorithms for querying large amount of linked data in a distributed manner. These query evaluation algorithms are independent of the way the data is stored, as well as of the particular implementation of the query evaluation. We then use the MapReduce paradigm to present a distributed implementation of these algorithms and experimentally evaluate them, although the algorithms could be straightforwardly translated into other distributed processing frameworks. We also investigate and propose multiple query decomposition approaches of Basic Graph Patterns (subclass of SPARQL queries) that are used to improve the overall performance of the distributed query answering. A deep analysis of the effectiveness of these decomposition algorithms is also provided.

Durable Subgraph Matching on Temporal Graphs

Durable subgraph matching on a temporal graph finds all subgraphs in the temporal graph that not only match the given query graph but also have duration longer than a user-specified duration threshold. The state-of-the-art algorithm (Semertzidis and Pitoura, 2016, 2019) for solving this problem requires lots of memory when the input temporal graph is large. In this paper, a new algorithm is proposed to solve this problem. Many effective techniques are developed to improve the performance of this algorithm, including the DFS-based query decomposition method, the TD-tree index structure, the sort-based vertex matching order, and the time instance set compaction method. The experimental results show that the proposed algorithm is an order of magnitude faster and requires significantly less memory than the state-of-the-art algorithm.

Subgraph matching over graph federation

Many real-life applications require processing graph data across heterogeneous sources. In this paper, we define the graph federation that indicates that the graph data sources are temporarily federated and offer their data for users. Next, we propose a new framework FedGraph to efficiently and effectively perform subgraph matching, which is a crucial application in graph federation. FedGraph consists of three phases, including query decomposition, distributed matching, and distributed joining. We also develop new efficient approximation algorithms and apply them in each phase to attack the NP-hard problem. The evaluations are conducted in a real test bed using both real-life and synthetic graph datasets. FedGraph outperforms the state-of-the-art methods, reducing the execution time and communication cost by 37.3 × and 61.8 ×, respectively.

Conversational interfaces for unconventional access to business relational data structures

PurposeCurrent possibilities of accessing business data by regular users usually involve complicated user interfaces or require technical expertise. This results in situations when business owners are separated from their data. The aim of this research is to apply an innovative approach leveraging conversational interfaces to tackle this problem.Design/methodology/approachThe authors examine the current possibilities of accessing business data by business, users with an emphasis on conversational interfaces employing a chatbot as an alternative to traditional approaches. The authors propose a new concept relying on a guided conversation, and through experiments with a real chatbot and database, the authors demonstrate the benefits of the proposed approach.FindingsThe authors found out that the key to the success of our approach is a decomposition of complex database queries and their incremental construction in conversations. This also enables natural discovery of the domain model through constantly provided feedback. Based on the experiments with a real chatbot, the authors demonstrate that defining conversation flows and maintaining the conversation context is a crucial aspect contributing to the overall accuracy, together with keeping the conversation within the defined limits in its certain parts.Originality/valueThe authors present a novel approach using natural language interfaces for accessing data by business users. In contrast to existing approaches, the authors emphasize incremental construction of queries, predefined conversation flows and constraining the conversations, when necessary.

Retracted] Social Governance Structure Construction and Resource Allocation Methods under the Management Mode of the Internet of Things

Aiming at the problems in the social governance structure level under the management mode of the Internet of Things, this paper designs a network agent based on the idea of service grid construction that has the functions of service resource configuration discovery, load balancing, and protocol conversion. Using it as a service governance infrastructure reduces the coupling between business services and service governance, and the network agent will intercept the service call traffic within the platform to achieve cross‐protocol and cross‐frame service communication. Through the proposed global query decomposition strategy and algorithm, a global query mapping is realized, which solves the problem of global query decomposition based on the global data model. We designed the unified management mechanism of two commonly used IoT databases based on XML. This process involves the realization of heterogeneous Internet of Things database resource configuration, data mode conversion, and global query mapping. Among them, in the process of data schema conversion, the conflict of the data in the schema mapping is resolved, the XML Schema is generated, the schema mapping metadata is established, and the XML document of the data is generated. We build a framework for optimal allocation of cloud computing resources based on load forecasting and propose an adaptive elastic control system for resource allocation based on resource status monitoring and demand forecasting. In view of the low resource utilization problem caused by the resource management mode of single virtual machine serving single user adopted by Internet of Things service providers, this paper constructs a new public cloud architecture with a five‐layer structure. On the basis of this architecture, an adaptive configuration mode of virtualized resources based on a single virtual machine serving multiple users is proposed. This mode can automatically search for optimal virtualization resources for application resource requests made by different users and run different applications on the same virtual machine without affecting the quality of service, so that IoT providers can guarantee services. At the same time, it improves the utilization efficiency of IoT resources and reduces energy consumption.

Decomposing Federated Queries in Presence of Replicated Fragments

Federated query engines allow for linked data consumption using SPARQL endpoints. Replicating data fragments from different sources enables data re-organization and provides the basis for more effective and efficient federated query processing. However, existing federated query engines are not designed to support replication. In this paper, we propose a replication-aware framework named LILAC, sparqL query decomposItion against federations of repLicAted data sourCes, that relies on replicated fragment descriptions to accurately identify sources that provide replicated data. We dened the query decomposition problem with fragment replication (QDP-FR). QDP-FR corresponds to the problem of finding the sub-queries to be sent to the endpoints that allows the federated query engine to compute the query answer, while the number of tuples to be transferred from endpoints to the federated query engine is minimized. An approximation of QDP-FR is implemented by the LILAC replication-aware query decomposition algorithm. Further, LILAC techniques have been included in the state-of-the-art federated query engines FedX and ANAPSID to evaluate the benefits of the proposed source selection and query decomposition techniques in different engines. Experimental results suggest that LILAC efficiently.

Decomposing federated queries in presence of replicated fragments

Federated query engines allow for linked data consumption using SPARQL endpoints. Replicating data fragments from different sources enables data re-organization and provides the basis for more effective and efficient federated query processing. However, existing federated query engines are not designed to support replication. In this paper, we propose a replication-aware framework named LILAC, sparqL query decomposItion against federations of repLicAted data sourCes, that relies on replicated fragment descriptions to accurately identify sources that provide replicated data. We defined the query decomposition problem with fragment replication (QDP-FR). QDP-FR corresponds to the problem of finding the sub-queries to be sent to the endpoints that allows the federated query engine to compute the query answer, while the number of tuples to be transferred from endpoints to the federated query engine is minimized. An approximation of QDP-FR is implemented by the LILAC replication-aware query decomposition algorithm. Further, LILAC techniques have been included in the state-of-the-art federated query engines FedX and ANAPSID to evaluate the benefits of the proposed source selection and query decomposition techniques in different engines. Experimental results suggest that LILAC efficiently solves QDP-FR and is able to reduce the number of transferred tuples and the execution time of the studied engines.

Task requirement aware pre-processing and Scheduling for IoT sensory environments

In Internet of Things (IoT) sensory environment Wireless Sensor Networks (WSNs) are connected to the Internet through gateways and this gives birth to many real time sensor based applications. Applications from Internet querying for Spatio-temporal information within WSN may require query decomposition. Decomposition of queries may result in tasks having similar functional and Quality of Service (QoS) requirements. Thus pre-processing the tasks results in decreased number of task executions within WSN as compared to executing tasks individually. Moreover prior knowledge of sensor nodes’ residual energy can help in deciding if a task can be scheduled for its successful completion. Existing energy monitoring protocols in WSNs incur considerable amount of message traffic and energy. Thus an Energy Monitoring System (EMS) that works on reduced message traffic and yet provide the correct energy information will be of great value. This paper proposes Task Requirements Aware Pre-processing and Scheduling (TRAPS) mechanism comprising a task pre-processor, EMS and scheduler within the gateway. The scheduler allocates the best available sensor nodes to the incoming tasks while meeting their QoS requirement. Task pre-processor and EMS are tested individually to measure the key performance metrics that include number of tasks entering WSN and energy information prediction accuracy respectively. Further the performance evaluation of TRAPS is extended to study its effects on various network parameters. It is found that TRAPS outperforms the findings of recent research on task scheduling in IoT sensory environments in terms of the identified network performance metrics.

Versatile Query Scrambling for Private Web Search

We consider the problem of privacy leaks suffered by Internet users when they perform web searches, and propose a framework to mitigate them. In brief, given a `sensitive' search query, the objective of our work is to retrieve the target documents from a search engine without disclosing the actual query. Our approach, which builds upon and improves recent work on search privacy, approximates the target search results by replacing the private user query with a set of blurred or scrambled queries. The results of the scrambled queries are then used to cover the private user interest. We model the problem theoretically, define a set of privacy objectives with respect to web search and investigate the effectiveness of the proposed solution with a set of queries with privacy issues on a large web collection. Experiments show great improvements in retrieval effectiveness over a previously reported baseline in the literature. Furthermore, the methods are more versatile, predictably-behaved, applicable to a wider range of information needs, and the privacy they provide is more comprehensible to the end-user. Additionally, we investigate the perceived privacy via a user study, as well as, measure the system's usefulness taking into account the trade off between retrieval effectiveness and privacy. The practical feasibility of the methods is demonstrated in a field experiment, scrambling queries against a popular web search engine. The findings may have implications for other IR research areas, such as query expansion, query decomposition, and distributed retrieval.

Query decomposition method for multi‐keyword search in P2P systems

A problem of multi-keyword search in a structured peer-to-peer (P2P) distributed computing system is considered. Methods have been developed to employ term-set indexing in a P2P system. Such an approach is an attempt to avoid excessive communication cost incurred by intersection operations in a single-term indexing method. In addition to limiting the maximum term-set size, index pruning was proposed to avoid exponential growth of the term-set index size. However, to obtain a global search result, query decomposition and query result intersection operations are still needed when the number of query terms exceeds the maximum term-set size. With index pruning employed, how a multi-term query is decomposed and affects the quality of retrieval results in terms of recall and precision is shown. A near-optimal query decomposition method is proposed to alleviate such a problem. Results from the simulation demonstrate the improvement gained by the proposed method.

Reader level filtering for efficient query processing in RFID middleware

In RFID systems, middleware is used to filter a large amount of streaming data gathered continuously from readers to process application requests. The high volume of data makes the middleware often in a highly overloaded situation. The reader filtering capability can be used to reduce data volume and middleware work-load. However, if the middleware dispatches query conditions to readers without any adjustment, it may generate a huge amount of duplicate data and imposes considerable load on the middleware. As such, an appropriate schema for data volume reduction is required. In this paper, we propose a query decomposition technique to divide queries into sub-queries for middleware and reader level execution. This new approach of query execution resolves the problem of duplicate data generation. Our experiments show that the proposed approach considerably improves the performance of middleware by reducing query processing time and network traffic between reader and middleware.

Query Optimization in OODBMS: Identifying Subquery for Query Management

This paper is based on relatively newer approach for query optimization in object databases, which uses query decomposition and cached query results to improve execution a query. Issues that are focused here is fast retrieval and high reuse of cached queries, Decompose Query into Sub query, Decomposition of complex queries into smaller for fast retrieval of result. Here we try to address another open area of query caching like handling wider queries. By using some parts of cached results helpful for answering other queries (wider Queries) and combining many cached queries while producing the result. Multiple experiments were performed to prove the productivity of this newer way of optimizing a query. The limitation of this technique is that it’s useful especially in scenarios where data manipulation rate is very low as compared to data retrieval rate.

A Query Driven Method of Mapping from Global Ontology to Local Ontology in Ontology-based Data Integration

At present, the mediator/wrapper integration methods are widely used in ontology based data integration because they solve the data update problems of data warehouse method. The key of this method is building of mapping from the global ontology in mediator to the local ontology in wrapper. This article analyzes the general mapping methods and designs a SPARQL query driven Global Local as View (GLAV) mapping method, it is proved in the experiment that this method can effectively support query decomposition and makes the ontology-based integration system has very good extensibility.

Query Optimization in OODBMS using Query Decomposition and Query Caching

Query optimization is of great importance for the performance of databases, especially for the execution of complex query statements. This paper is based on relatively newer approach for query optimization in object databases, which uses query decomposition and cached query results to improve execution times for a query. Multiple experiments will be executed to prove the effectiveness of this approach. The two main areas which will be covered are caching of query results and maintenance of the consistency of result sets after database updates.

A formal framework for query decomposition and knowledge integration in data warehouse federations

A federation of data warehouses is understood as a set of data warehouses, which can be processed as a whole in the logic level. Physically, the federation does not gather data into one place. This paper presents a formal framework for data and knowledge processing in data warehouse federations. The management system for a data warehouse federation consists of an user interface enabling presentation of user queries, a program for query decomposition and a program for integrating knowledge coming from different data warehouses as the answers to a user query. We propose a model for query decomposition process and knowledge integration. It contains also the algorithm for knowledge inconstancy processing. This kind of inconsistency often occurs since very often the knowledge extracted from different data warehouses refers to the same subject, but is not consistent.

AN ASPECT QUERY LANGUAGE MODEL BASED ON QUERY DECOMPOSITION AND HIGH‐ORDER CONTEXTUAL TERM ASSOCIATIONS

In information retrieval (IR) research, more and more focus has been placed on optimizing a query language model by detecting and estimating the dependencies between the query and the observed terms occurring in the selected relevance feedback documents. In this paper, we propose a novel Aspect Language Modeling framework featuring term association acquisition, document segmentation, query decomposition, and an Aspect Model (AM) for parameter optimization. Through the proposed framework, we advance the theory and practice of applying high‐order and context‐sensitive term relationships to IR. We first decompose a query into subsets of query terms. Then we segment the relevance feedback documents into chunks using multiple sliding windows. Finally we discover the higher order term associations, that is, the terms in these chunks with high degree of association to the subsets of the query. In this process, we adopt an approach by combining the AM with the Association Rule (AR) mining. In our approach, the AM not only considers the subsets of a query as “hidden” states and estimates their prior distributions, but also evaluates the dependencies between the subsets of a query and the observed terms extracted from the chunks of feedback documents. The AR provides a reasonable initial estimation of the high‐order term associations by discovering the associated rules from the document chunks. Experimental results on various TREC collections verify the effectiveness of our approach, which significantly outperforms a baseline language model and two state‐of‐the‐art query language models namely the Relevance Model and the Information Flow model.

Semantic optimization of query transformation in a large-scale peer-to-peer network

Ontologies have an important role in supporting efficient interoperability among information systems in distributed environment. In this paper, I propose a query transformation method to efficiently collect as many relevant resources from the distributed information systems (e.g., peer-to-peer network) as possible. More importantly, I consider a composite query which contains multiple semantics. The composite query from the source peer can be decomposed and propagated as maintaining the original contexts. Through the experiments, I have shown that query-activated concept (QAC)-based schemes have fulfilled an efficient query decomposition process on semantic peer-to-peer networks.