Query Processing Architecture Research Articles

Scalable Multi-grained Cross-modal Similarity Query with Interpretability

Cross-modal similarity query has become a highlighted research topic for managing multimodal datasets such as images and texts. Existing researches generally focus on query accuracy by designing complex deep neural network models and hardly consider query efficiency and interpretability simultaneously, which are vital properties of cross-modal semantic query processing system on large-scale datasets. In this work, we investigate multi-grained common semantic embedding representations of images and texts and integrate interpretable query index into the deep neural network by developing a novel Multi-grained Cross-modal Query with Interpretability (MCQI) framework. The main contributions are as follows: (1) By integrating coarse-grained and fine-grained semantic learning models, a multi-grained cross-modal query processing architecture is proposed to ensure the adaptability and generality of query processing. (2) In order to capture the latent semantic relation between images and texts, the framework combines LSTM and attention mode, which enhances query accuracy for the cross-modal query and constructs the foundation for interpretable query processing. (3) Index structure and corresponding nearest neighbor query algorithm are proposed to boost the efficiency of interpretable queries. (4) A distributed query algorithm is proposed to improve the scalability of our framework. Comparing with state-of-the-art methods on widely used cross-modal datasets, the experimental results show the effectiveness of our MCQI approach.

A scalable ledger-assisted architecture for secure query processing over distributed IoT data

Massive amount of IoT data poses unique challenges in centralized data management systems. Specifically, IoT data can originate from heterogeneous and distributed sources, and commonly regulations forbid data from different IoT stakeholders to be managed via central governance. To serve for IoT applications, recent proposals leverage distributed ledgers (e.g., blockchains) to function on top of distributed data storage with improved data interoperability. Unfortunately, most of them do not consider data security and privacy in the first place. Along with this transformative paradigm, in this paper, we propose a ledger-assisted architecture for secure distributed IoT data management. This architecture adapts searchable encryption to decentralized storage network to enable encrypted query processing. Meanwhile, it is designed to leverage the distributed ledger to harden both data and query integrity. To deal with continuously generated IoT data, we further devise an efficient secure data insertion protocol, and employ a recent variant of blockchain for users to validate updated query results in a scalable manner. Evaluations on Azure blockchain service confirm the practicality of our proposed architecture.

Couchbase analytics

Couchbase Server is a highly scalable document-oriented database management system. With a shared-nothing architecture, it exposes a fast key-value store with a managed cache for sub-millisecond data operations, indexing for fast queries, and a powerful query engine for executing declarative SQL-like queries. Its Query Service debuted several years ago and supports high volumes of low-latency queries and updates for JSON documents. Its recently introduced Analytics Service complements the Query Service. Couchbase Analytics, the focus of this paper, supports complex analytical queries (e.g., ad hoc joins and aggregations) over large collections of JSON documents. This paper describes the Analytics Service from the outside in, including its user model, its SQL++ based query language, and its MPP-based storage and query processing architecture. It also briefly touches on the relationship of Couchbase Analytics to Apache AsterixDB, the open source Big Data management system at the core of Couchbase Analytics.

Querying in Internet of Things with Privacy Preserving: Challenges, Solutions and Opportunities

IoT is envisioned as the next stage of the information revolution, enabling various daily applications and providing better service by conducting a deep fusion with cloud and fog computing. As the key mission of most IoT applications, data management, especially the fundamental function-data query, has long been plagued by severe security and privacy problems. Most query service providers, including the big ones (e.g., Google, Facebook, Amazon, and so on) are suffering from intensive attacks launched by insiders or outsiders. As a consequence, processing various queries in IoT without compromising the data and query privacy is an urgent and challenging issue. In this article, we propose a thing-fog-cloud architecture for secure query processing based on well studied classical paradigms. Following with a description of crucial technical challenges in terms of functionality, privacy and efficiency assurance, we survey the latest milestone-like approaches, and provide an insight into the advantages and limitations of each scheme. Based on the recent advances, we also discuss future research opportunities to motivate efforts to develop practical private query protocols in IoT.

Mind the gap

Executing domain specific workloads from a relational data warehouse is an increasingly popular task. Unfortunately, classic relational database management systems (RDBMS) are suboptimal in many domains (e.g., graph and linear algebra queries), and it is challenging to transfer data from an RDBMS to a domain specific toolkit in an efficient manner. This demonstration showcases the EmptyHeaded engine: an interactive query processing engine that leverages a novel query architecture to support efficient execution in multiple domains. To enable a unified design, the EmptyHeaded architecture is built around recent theoretical advancements in join processing and automated in-query data transformations. This demonstration highlights the strengths and weaknesses of this novel type of query processing architecture while showcasing its flexibility in multiple domains. In particular, attendees will use EmptyHeaded's Jupyter notebook front-end to interactively learn the theoretical advantages of this new (and largely unknown) approach and directly observe its performance impact in multiple domains.

System Architecture for Approximate Query Processing

<p><span style="font-size: 10.5pt; font-family: 'Times New Roman','serif'; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: 宋体; mso-font-kerning: 1.0pt; mso-ansi-language: EN-US; mso-fareast-language: ZH-CN; mso-bidi-language: AR-SA;" lang="EN-US"><span style="font-size: 10.5pt; font-family: 'Times New Roman','serif'; mso-bidi-font-size: 12.0pt; mso-fareast-font-family: 宋体; mso-font-kerning: 1.0pt; mso-ansi-language: EN-US; mso-fareast-language: ZH-CN; mso-bidi-language: AR-SA;" lang="EN-US">Decision making is an activity that addresses the problem of extracting knowledge and information from data stored in data warehouses, in order to improve the business processes of information systems. Usually, decision making is based on On-Line Analytical Processing, data mining, or approximate query processing. In the last case, answers to analytical queries are provided in a fast manner, although affected with a small percentage of error. In the paper, we present the architecture of an approximate query answering system. Then, we illustrate our ADAP (Analytical Data Profile) system, which is based on an engine able to provide fast responses to the main statistical functions by using orthogonal polynomials series to approximate the data distribution of multi­dimensional relations. Moreover, several experimental results to measure the approximation error are shown and the response-time to analytical queries is reported.</span></span></p>

Efficient Query Processing in 3D Motion Capture Gesture Databases

One of the most fundamental challenges when accessing gestural patterns in 3D motion capture databases is the definition of spatiotemporal similarity. While distance-based similarity models such as the Gesture Matching Distance on gesture signatures are able to leverage the spatial and temporal characteristics of gestural patterns, their applicability to large 3D motion capture databases is limited due to their high computational complexity. To this end, we present a lower bound approximation of the Gesture Matching Distance that can be utilized in an optimal multi-step query processing architecture in order to support efficient query processing. We investigate the performance in terms of accuracy and efficiency based on 3D motion capture databases and show that our approach is able to achieve an increase in efficiency of more than one order of magnitude with a negligible loss in accuracy. In addition, we discuss different applications in the digital humanities in order to highlight the significance of similarity search approaches in the research field of gestural pattern analysis.

An Envisioned Approach for Modeling and Supporting User-Centric Query Activities on Data Warehouses

In this vision paper, the authors discuss models and techniques for integrating, processing and querying data, information and knowledge within data warehouses in a user-centric manner. The user-centric emphasis allows us to achieve a number of clear advantages with respect to classical data warehouse architectures, whose most relevant ones are the following: (i) a unified and meaningful representation of multidimensional data and knowledge patterns throughout the data warehouse layers (i.e., loading, storage, metadata, etc); (ii) advanced query mechanisms and guidance that are capable of extracting targeted information and knowledge by means of innovative information retrieval and data mining techniques. Following this main framework, the authors first outline the importance of knowledge representation and management in data warehouses, where knowledge is expressed by existing ontology or patterns discovered from data. Then, the authors propose a user-centric architecture for OLAP query processing, which is the typical applicative interface to data warehouse systems. Finally, the authors propose insights towards cooperative query answering that make use of knowledge management principles and exploit the peculiarities of data warehouses (e.g., multidimensionality, multi-resolution, and so forth).

Towards cost-effective storage provisioning for DBMSs

Data center operators face a bewildering set of choices when considering how to provision resources on machines with complex I/O subsystems. Modern I/O subsystems often have a rich mix of fast, high performing, but expensive SSDs sitting alongside with cheaper but relatively slower (for random accesses) traditional hard disk drives. The data center operators need to determine how to provision the I/O resources for specific workloads so as to abide by existing Service Level Agreements (SLAs), while minimizing the total operating cost (TOC) of running the workload, where the TOC includes the amortized hardware costs and the run time energy costs. The focus of this paper is on introducing this new problem of TOC-based storage allocation, cast in a framework that is compatible with traditional DBMS query optimization and query processing architecture. We also present a heuristic-based solution to this problem, called DOT. We have implemented DOT in PostgreSQL, and experiments using TPC-H and TPC-C demonstrate significant TOC reduction by DOT in various settings.

DocBase

This article introduces a complete storage and retrieval architecture for a database environment for XML documents. DocBase, a prototype system based on this architecture, uses a flexible storage and indexing technique to allow highly expressive queries without the necessity of mapping documents to other database formats. DocBase is an integration of several techniques that include (i) a formal model called Heterogeneous Nested Relations (HNR), (ii) a conceptual model XER (Extensible Entity Relationship), (ii) formal query languages (Document Algebra and Calculus), (iii) a practical query language (Document SQL or DSQL), (iv) a visual query formulation method with QBT (Query By Templates), and (v) the DocBase query processing architecture. This paper focuses on the overall architecture of DocBase including implementation details, describes the details of the query-processing framework, and presents results from various performance tests. The paper summarizes experimental and usability analyses to demonstrate its feasibility as a general architecture for native as well as embedded document manipulation methods.

Streaming multiple aggregations using phantoms

Data streams characterize the high speed and large volume input of a new class of applications such as network monitoring, web content analysis and sensor networks. Among these applications, network monitoring may be the most compelling one--the backbone of a large internet service provider can generate 1 petabyte of data per day. For many network monitoring tasks such as traffic analysis and statistics collection, aggregation is a primitive operation. Various analytical and statistical needs naturally lead to related aggregate queries. In this article, we address the problem of efficiently computing multiple aggregations over high-speed data streams based on the two-level query processing architecture of GS, a real data stream management system deployed in AT & T. We discern that additionally computing and maintaining fine-granularity aggregations (called phantoms) has the benefit of supporting shared computation. Based on a thorough analysis, we propose algorithms to identify the best set of phantoms to maintain and determine allocation of resources (particularly, space) to compute the aggregations. Experiments show that our algorithm achieves near-optimal computation costs, which outperforms the best adapted algorithm by more than an order of magnitude.

An efficient multi-keyword query processing strategy on P2P based Web search

The paper presents a novel benefit based query processing strategy for efficient query routing. Based on DHT as the overlay network, it first applies Nash equilibrium to construct the optimal peer group based on the correlations of keywords and coverage and overlap of the peers to decrease the time cost, and then presents a two-layered architecture for query processing that utilizes Bloom filter as compact representation to reduce the bandwidth consumption. Extensive experiments conducted on a real world dataset have demonstrated that our approach obviously decreases the processing time, while improves the precision and recall as well.

Attribute Allocation and Retrieval Scheme for Large-Scale Sensor Networks

Wireless sensor network is an emerging technology that enables remote monitoring of large geographical regions. In this paper, we address the problem of distributing attributes over such a large-scale sensor network so that the cost of data retrieval is minimized. The proposed scheme is a data-centric storage scheme where the attributes are distributed over the network depending on the correlations between them. The problem addressed here is similar to the Allocation Problem of distributed databases. In this paper, we have defined the Allocation Problem in the context of sensor networks and have proposed a scheme for finding a good distribution of attributes to the sensor network. We also propose an architecture for query processing given such a distribution of attributes. We analytically determine the conditions under which the proposed architecture is beneficial and present simulation results to demonstrate the same. To the best of our knowledge, this is the first attempt to determine an allocation of attributes over a sensor network based on the correlations between attributes.

Building Scalable Electronic Market Places Using HyperQuery-Based Distributed Query Processing

Flexible distributed query processing capabilities are an important prerequisite for building scalable Internet applications, such as electronic Business-to-Business (B2B) market places. Architecting an electronic market place in a conventional data warehouse-like approach by integrating all the data from all participating enterprises in one centralized repository incurs severe problems: stale data, data security threats, administration overhead, inflexibility during query processing, etc. In this paper we present a new framework for dynamic distributed query processing based on so-called HyperQueries which are essentially query evaluation sub-plans "sitting behind" hyperlinks. Our approach facilitates the pre-materialization of static data at the market place whereas the dynamic data remains at the data sources. In contrast to traditional data integration systems, our approach executes essential (dynamic) parts of the data-integrating views at the data sources. The other, more static parts of the data are integrated a priori at the central portal, e.g., the market place. The portal serves as an intermediary between clients and data providers which execute their sub-queries referenced via hyperlinks. The hyperlinks are embedded as attribute values within data objects of the intermediary's database. Retrieving such a virtual object will execute the referenced HyperQuery in order to materialize the missing data. We illustrate the flexibility of this distributed query processing architecture in the context of B2B electronic market places with an example derived from the car manufacturing industry. Based on these HyperQueries, we propose a reference architecture for building scalable and dynamic electronic market places. All administrative tasks in such a distributed B2B market place are modeled as Web services and are initiated decentrally by the participants. Thus, sensitive data remains under the full control of the data providers. We describe optimization and implementation issues to obtain an efficient and highly flexible data integration platform for electronic market places. All proposed techniques have been fully implemented in our QueryFlow prototype system which served as the platform for our performance evaluation.

An adaptable distributed query processing architecture

Traditionally, distributed query optimization techniques generate static query plans at compile time. However, the optimality of these plans depends on many parameters (such as the selectivities of operations, the transmission speeds and workloads of servers) that are not only difficult to estimate but are also often unpredictable and fluctuant at runtime. As the query processor cannot dynamically adjust the plans at runtime, the system performance is often less than satisfactory. In this paper, we introduce a new highly adaptive distributed query processing architecture. Our architecture can quickly detect fluctuations in selectivities of operations, as well as transmission speeds and workloads of servers, and accordingly change the operation order of a distributed query plan during execution. We have implemented a prototype based on the Telegraph system [Telegragraph project. Available from < http://telegraph.cs.berkeley.edu/>]. Our experimental study shows that our mechanism can adapt itself to the changes in the environment and hence approach to an optimal plan during execution.

Exploiting k -constraints to reduce memory overhead in continuous queries over data streams

Continuous queries often require significant run-time state over arbitrary data streams. However, streams may exhibit certain data or arrival patterns, or constraints , that can be detected and exploited to reduce state considerably without compromising correctness. Rather than requiring constraints to be satisfied precisely, which can be unrealistic in a data streams environment, we introduce k-constraints , where k is an adherence parameter specifying how closely a stream adheres to the constraint. (Smaller k 's are closer to strict adherence and offer better memory reduction.) We present a query processing architecture, called k-Mon , that detects useful k -constraints automatically and exploits the constraints to reduce run-time state for a wide range of continuous queries. Experimental results showed dramatic state reduction, while only modest computational overhead was incurred for our constraint monitoring and query execution algorithms.

Performance of a distributed architecture for query processing on workstation clusters

A network of workstations (NOWs) is an attractive alternative to parallel database systems due to the cost advantage. In a typical database, client workstations (nodes) submit queries/transactions and receive responses from the database server. With even recent PC-based client nodes providing traditional workstation-class performance, performance improvements can be obtained by offloading some of the processing typically done on the traditional server node to these powerful client nodes. Parallel query processing takes advantage of the idle cycles on the client nodes to process the query. In this paper we introduce a distributed architecture for parallel query processing and study the performance of this architecture in various scenarios. We have implemented the distributed architecture using parallel virtual machine (PVM) on a Pentium-based NOW system. Our results show that the distributed architecture successfully achieves good speedups and scaleups. Furthermore, our results show that the distributed architecture handles both light and heavy workloads in the presence of a background (non-query) load.

Combining Approximation Techniques and Vector Quantization for Adaptable Similarity Search

Adaptable similarity queries based on quadratic form distance functions are widely popular in data mining application domains including multimedia, CAD, molecular biology or medical image databases. Recently it has been recognized that quantization of feature vectors can substantially improve query processing for Euclidean distance functions, as demonstrated by the scan-based VA-file and the index structure IQ-tree. In this paper, we address the problem that determining quadratic form distances between quantized vectors is difficult and computationally expensive. Our solution provides a variety of new approximation techniques for quantized vectors which are combined by an extended multistep query processing architecture. In our analysis section, we show that the filter steps complement each other. Consequently, it is useful to apply our filters in combination. We show the superiority of our approach over other architectures and over competitive query processing methods. In our experimental evaluation, the sequential scan is outperformed by a factor of 2.3. Compared to the X-tree on 64 dimensional color histogram data, we measured an improvement factor of 5.7.

Continuous queries over data streams

In many recent applications, data may take the form of continuous data streams, rather than finite stored data sets. Several aspects of data management need to be reconsidered in the presence of data streams, offering a new research direction for the database community. In this paper we focus primarily on the problem of query processing, specifically on how to define and evaluate continuous queries over data streams. We address semantic issues as well as efficiency concerns. Our main contributions are threefold. First, we specify a general and flexible architecture for query processing in the presence of data streams. Second, we use our basic architecture as a tool to clarify alternative semantics and processing techniques for continuous queries. The architecture also captures most previous work on continuous queries and data streams, as well as related concepts such as triggers and materialized views. Finally, we map out research topics in the area of query processing over data streams, showing where previous work is relevant and describing problems yet to be addressed.

A multistep approach for shape similarity search in image databases

Shape similarity searching is a crucial task in image databases, particularly in the presence of errors induced by segmentation or scanning images. The resulting slight displacements or rotations have not been considered so far in the literature. We present a new similarity model that flexibly addresses this problem. By specifying neighborhood influence weights, the user may adapt the similarity distance functions to his or her own requirements or preferences. Technically, the new similarity model is based on quadratic forms for which we present a multi-step query processing architecture, particularly for high dimensions as they occur in image databases. Our algorithm to reduce the dimensionality of quadratic form-based similarity queries results in a lower-bounding distance function that is proven to provide an optimal filter selectivity. Experiments on our test database of 10,000 images demonstrate the applicability and the performance of our approach, even in dimensions as high as 1,024.