Query Operations Research Articles

HOLISTIC EVALUATION OF XML QUERIES WITH STRUCTURAL PREFERENCES ON AN ANNOTATED STRONG DATAGUIDE

With the emergence of XML as de facto format for storing and exchanging information over the Internet, the search for ever more innovative and effective techniques for their querying is a major and current concern of the XML database community. Several studies carried out to help solve this problem are mostly oriented towards the evaluation of so-called exact queries which, unfortunately, are likely (especially in the case of semi-structured documents) to yield abundant results (in the case of vague queries) or empty results (in the case of very precise queries). From the observation that users who make requests are not necessarily interested in all possible solutions, but rather in those that are closest to their needs, an important field of research has been opened on the evaluation of preferences queries. In this paper, we propose an approach for the evaluation of such queries, in case the preferences concern the structure of the document. The solution investigated revolves around the proposal of an evaluation plan in three phases: rewriting-evaluation-merge. The rewriting phase makes it possible to obtain, from a partitioning -transformation operation of the initial query, a hierarchical set of preferences path queries which are holistically evaluated in the second phase by an instrumented version of the algorithm TwigStack. The merge phase is the synthesis of the best results.

Computing possible and certain answers over order-incomplete data

This paper studies the complexity of query evaluation for databases whose relations are partially ordered; the problem commonly arises when combining or transforming ordered data from multiple sources. We focus on queries in a useful fragment of SQL, namely positive relational algebra with aggregates, whose bag semantics we extend to the partially ordered setting. Our semantics leads to the study of two main computational problems: the possibility and certainty of query answers. We show that these problems are respectively NP-complete and coNP-complete, but identify tractable cases depending on the query operators or input partial orders. We further introduce a duplicate elimination operator and study its effect on the complexity results.

FSSE: Forward secure searchable encryption with keyed-block chains

Nowadays, people widely use Searchable Symmetric Encryption (SSE) to design the equality query and range query operation for encrypted database systems. A recent attack points out forward privacy is the necessary security goal which, however, has not been achieved by most of SSE schemes. Motivated by this security goal, we propose a forward secure SSE scheme whose index structure consists of keyed-block chains. The proposed scheme is efficient since the keyed-block chain can be implemented by symmetric encryption primitive and the scheme also supports both add and delete operations in one instance. We further employ the one-way permutation function to reduce the storage on the client side. Experiments show that our scheme is 4 × , 300 × and 300 × faster than the state-of-the-art forward private SSE scheme (CCS 2016) in search, update and token generation, respectively.

SubspaceDB : In-database subspace clustering for analytical query processing

High dimensional data analysis within relational database management systems (RDBMS) is challenging because of inadequate support from SQL. Currently, subspace clustering of high dimensional data is implemented either outside DBMS using wrapper code or inside DBMS using SQL User Defined Functions/Aggregates(UDFs/UDAs). However, both these approaches have potential disadvantages from performance, resource usage, and security perspective for voluminous and frequently updated data. Hence, we propose an efficient querying system, named SubspaceDB, that implements subspace clustering directly within an RDBMS. SubspaceDB provides a novel set of query operators, each with an optimization objective, to facilitate interactive analysis for subspace clustering. The query operators focus on retrieving optimal answers to four key query types : (a) Medoid queries, (b) Neighbourhood queries, (c) Partial similarity queries, and (d) Prominence queries, that aid the formation of subspace clusters. Experimental studies on real and synthetic databases of size 15M tuples and 104 attributes show that our proposed approach SubspaceDB can be over 10 times faster as compared to a conventional wrapper-based or SQL UDF approach. The proposed approach is also efficient in retrieving at least 50% data with performance improvement of at least 25%.

Communication-Efficient Data Aggregation Tree Construction for Complex Queries in IoT Applications

Data aggregation is a fundamental operation in Internet of Things (IoT) applications, e.g., distributed Internet-based industrial control and computing systems. As IoT devices are increasingly connected to the system via resource-constrained wireless communication links, it is critical to perform communication-efficient data aggregation to answer complex queries (e.g., skyline queries and equality joins) from IoT applications. In this paper, we investigate the problem of constructing an aggregation tree (AT) for complex queries with the minimum communication cost. As complex queries have a dynamic size of intermediate results, existing Steiner tree-based approaches for traditional query operators, e.g., MIN and top- ${k}$ , cannot be directly applied. We first formalize the aggregation gain by jointly considering the data pruning power (the size of data points that can be pruned during the aggregation for complex queries) and aggregation cost (the size of data points transmitted for the aggregation). By maximizing the aggregation gain, the data set that has a higher pruning power and a smaller size is selected and transferred for data aggregation at succeeding nodes. We then propose to construct the AT by connecting a set of aggregation operations with maximum aggregation gain. Extensive evaluation shows that our proposed framework achieves the promising results.

DistriPlan: an optimized join execution framework for geo-distributed scientific data

Scientific data is frequently stored across geographically distributed data repositories. Although there have been recent efforts to query scientific datasets using structured query operators, they have not yet supported joins across distributed data repositories. This paper describes a framework that supports join-like operations over multi-dimensional array datasets that are spread across multiple sites. More specifically, we first formally define join operations over array datasets and establish how they arise in the context of scientific data analysis. We then describe a methodology for optimizing such operations—components of our approach include enumeration algorithms for candidate plans, methods for pruning plans before they are enumerated, and a detailed cost model for selecting the best (cheapest) plan. We evaluate our approach using candidate queries, and show that the optimization effort is practical and profitable—query performance was improved significantly using our approach.

PySpark and RDKit: Moving towards Big Data in Cheminformatics.

The authors present an implementation of the cheminformatics toolkit RDKit in a distributed computing environment, Apache Hadoop. Together with the Apache Spark analytics engine, wrapped by PySpark, resources from commodity scalable hardware can be employed for cheminformatic calculations and query operations with basic knowledge in Python programming and understanding of the resilient distributed datasets (RDD). Three use cases of cheminfomatical computing in Spark on the Hadoop cluster are presented; querying substructures, calculating fingerprint similarity and calculating molecular descriptors. The source code for the PySpark-RDKit implementation is provided. The use cases showed that Spark provides a reasonable scalability depending on the use case and can be a suitable choice for datasets too big to be processed with current low-end workstations.

Towards Using Cloud Elasticity on the Internet of Things Landscape

The digital universe is growing at significant rates in recent years. One of the main responsible for this sentence is the Internet of Things, or IoT, which requires a middleware that should be capable to handle this increase of data volume at real-time. Particularly, data can arrive in the middleware in parallel as in terms of input data from Radio-Frequency Identification (RFID) readers as request-reply query operations from the users side. Solutions modeled at software, hardware and/or architecture levels present limitations to handle such load, facing the problem of scalability in the IoT scope. In this context, this arti- cle presents a model denoted Eliot - Elasticity-driven Internet of Things - which combines both cloud and high performance computing to address the IoT scal- ability problem in a novel EPCglobal-compliant architecture. Particularly, we keep the same API but offer an elastic EPCIS component in the cloud, which is designed as a collection of virtual machines (VMs) that are allocated and deallocated on-the-fly in accordance with the system load. Based on the Eliot model, we developed a prototype that could run over any black-box EPCglobal- compliant middleware. We selected the Fosstrak for this role, which is currently one of the most used IoT middlewares. Thus, the prototype acts as an upper layer over the Fosstrak to offer a better throughput and latency performances in an effortless way. The results are encouraging, presenting significant performance gains in terms of response time and request throughput when comparing both elastic (Eliot) and non-elastic (standard Fosstrak) executions.

Automatic Detection of Shoreline Change by Geographical Information System (GIS) and Remote Sensing in the Göksu Delta, Turkey

The changes occurring in the coastal regions of the countries with coasts are a very important issue for many professional disciplines, especially management and planning. Although the studies conducted on the detection of shoreline changes, future predictions, planning and taking measure are the center of interest all over the world, studies conducted on the subject are inadequate. In this study, it is aimed to reveal the shoreline change occurring in the Goksu Delta by using satellite images and GIS-based analyses. Within the scope of the study, the shoreline changes that took place in five different dates (1984, 1998, 2003, 2006, 2011) between 1984 and 2011 (27 years) were examined by the shoreline change envelope (SCE), end point rate (EPR), and linear regression rate (LRR) methods and digital shoreline analysis system extension program running on ArcGIS software, and the results obtained were analyzed in the GIS environment. Furthermore, a user interface program performing a number of query operations was developed within the scope of the study. In conclusion, the effects causing shoreline changes in the region were investigated, and a number of solution proposals were offered so that necessary measures could be taken.

Modeling simulation and virtual platform construction of crossbow making technology

In this paper, a set of virtual simulation platform for crossbow making is established. First, build a virtual model of Lisu crossbow, optimize the model, accelerate the organization, model generation, modification and editing of the database; then complete the database establishment of the corresponding model, nest the comment entity class and filemeta entity class into the file entity class, store them in a nested data set, and complete the data storage, management and query operations; finally, combined with Kalman filtering, arithmetic average filtering and KNN algorithm are used to build the quest 3D virtual platform to improve the 3D interaction effect with data gloves and complete the construction of the 3D platform.

Shrinkwrap

A private data federation is a set of autonomous databases that share a unified query interface offering in-situ evaluation of SQL queries over the union of the sensitive data of its members. Owing to privacy concerns, these systems do not have a trusted data collector that can see all their data and their member databases cannot learn about individual records of other engines. Federations currently achieve this goal by evaluating queries obliviously using secure multiparty computation. This hides the intermediate result cardinality of each query operator by exhaustively padding it. With cascades of such operators, this padding accumulates to a blow-up in the output size of each operator and a proportional loss in query performance. Hence, existing private data federations do not scale well to complex SQL queries over large datasets. We introduce Shrinkwrap, a private data federation that offers data owners a differentially private view of the data held by others to improve their performance over oblivious query processing. Shrinkwrap uses computational differential privacy to minimize the padding of intermediate query results, achieving up to a 35X performance improvement over oblivious query processing. When the query needs differentially private output, Shrinkwrap provides a trade-off between result accuracy and query evaluation performance.

Optimizing computational high-order schemes in finite volume simulations using unstructured mesh and topological data structures

Many numerical methods are based in mesh files to represent the computational domain. Also, an efficient storage and retrieval of mesh information can be achieved by data structures. Moreover, the development of topological operators is one of the important goals of the geometric modeling research field. While it loads mesh files more efficiently, it also allocates less main memory, provides persistence and allows consistent query operations. This paper proposes an improving of the computational scheme of high-order WENO schemes by coupling a standard cell centered, unstructured mesh, finite volume method with an topological data structure. The solver module uses the finite volume technique with a formulation that sets the property values to the control volume centroids. The two dimensional Euler equations are considered to represent the flow of interest. Beyond experiments using the improved approach, the computational cost of the data structure was measured by comparing with the traditional representation, and the results showed that our approach provides scalable loading and managing of meshes, having less memory occupation rate when comparing meshes with an increasing number of elements.

TripleID-Q: RDF Query Processing Framework Using GPU

Resource Description Framework (RDF) data represents information linkage around the Internet. It uses Inter- nationalized Resources Identifier (IRI) which can be referred to external information. Typically, an RDF data is serialized as a large text file which contains millions of relationships. In this work, we propose a framework based on TripleID-Q, for query processing of large RDF data in a GPU. The key elements of the framework are 1) a compact representation suitable for a Graphics Processing Unit (GPU) and 2) its simple representation conversion method which optimizes the preprocessing overhead. Together with the framework, we propose parallel algorithms which utilize thousands of GPU threads to look for specific data for a given query as well as to perform basic query operations such as union, join, and filter. The TripleID representation is smaller than the original representation 3-4 times. Querying from TripleID using a GPU is up to 108 times faster than using the traditional RDF tool. The speedup can be more than 1,000 times over the traditional RDF store when processing a complex query with union and join of many subqueries.

Real‐time inverse distance weighting interpolation for streaming sensor data

AbstractWith advances in technology and an increasing variety of inexpensive geosensors, environmental monitoring has become increasingly sensor dense and real time. Using sensor data streams enables real‐time applications such as environmental hazard detection, or earthquake, wildfire, or radiation monitoring. In‐depth analysis of such spatial fields is often based on a continuous representation. With very large numbers of concurrent observation streams, novel algorithms are necessary that integrate streams into rasters, or other continuous representations, continuously in real time. In this article, we present an approach leveraging data stream engines (DSEs) to achieve scalable, high‐throughput inverse distance weighting (IDW). In detail, we designed and implemented a novel stream query operator framework that extends general‐purpose DSEs. The proposed framework includes a two‐panel, spatio‐temporal grid‐based index and several algorithms, namely the Shell and k‐Shell algorithms, to estimate individual grid cells efficiently and adaptively for different sampling scenarios. For our performance experiments, we generated several different spatio‐temporal stream data sets based on the radiation deposits in the Fukushima region after the nuclear accident of 2011 in Japan. Our results showed that the k‐Shell algorithm of the proposed framework produces a raster based on 250k observation streams in under 0.5 s using a state‐of‐the‐art workstation.

LandQv2: A MapReduce-Based System for Processing Arable Land Quality Big Data

Arable land quality (ALQ) data are a foundational resource for national food security. With the rapid development of spatial information technologies, the annual acquisition and update of ALQ data covering the country have become more accurate and faster. ALQ data are mainly vector-based spatial big data in the ESRI (Environmental Systems Research Institute) shapefile format. Although the shapefile is the most common GIS vector data format, unfortunately, the usage of ALQ data is very constrained due to its massive size and the limited capabilities of traditional applications. To tackle the above issues, this paper introduces LandQv2, which is a MapReduce-based parallel processing system for ALQ big data. The core content of LandQv2 is composed of four key technologies including data preprocessing, the distributed R-tree index, the spatial range query, and the map tile pyramid model-based visualization. According to the functions in LandQv2, firstly, ALQ big data are transformed by a MapReduce-based parallel algorithm from the ESRI Shapefile format to the GeoCSV file format in HDFS (Hadoop Distributed File System), and then, the spatial coding-based partition and R-tree index are executed for the spatial range query operation. In addition, the visualization of ALQ big data with a GIS (Geographic Information System) web API (Application Programming Interface) uses the MapReduce program to generate a single image or pyramid tiles for big data display. Finally, a set of experiments running on a live system deployed on a cluster of machines shows the efficiency and scalability of the proposed system. All of these functions supported by LandQv2 are integrated into SpatialHadoop, and it is also able to efficiently support any other distributed spatial big data systems.

IOT SİSTEMLERİ İÇİN VERİTABANI SİSTEM ÖNERİLERİ

Internet of Things (IoT) is an interconnection of different types of information assets in which data is continuously generated and transmitted over the Internet. Technologies of the sensor, RFID, GPS, mobile devices, and Internet-enabled actuators play a significant role in IoT systems. IoT brings out new challenges in terms of data and information management because it is not easy to collect and manage a large amount of heterogeneous data that is aggregated at very high velocity as well as to retrieve and manage the information that is hidden within this large volume of data. In this paper, I discuss the main factors affecting the efficiency of data management in IoT systems, specifically query processing and transaction management. There are many lessons learned from traditional database systems, distributed systems, and sensor networks, however, traditional solutions are often inadequate to meet the needs of applications in such a complex ecosystem, namely IoT. In traditional database systems, for instance, query operations are usually local, and execution costs depend on the current processor power and other resource constraints (i.e. memory). On the other hand, transaction management mechanisms guarantee the ACID properties in order to provide overall data integrity. It is apparent that different types of IOT applications that operate on heterogeneous, streaming, real-time, and geographically distributed large data will significantly change the well-known aspects of querying and transaction management. Context-aware querying, distributed querying, MapReduce computing model and flexible transaction models such as web-based transaction handling are some of the current issues discussed in this paper. With the succinct yet comprehensive information presented in this work, I intend to provide a guide for researchers in the IoT systems, especially in the context of database systems.

Enhancing performance of IoT applications with load prediction and cloud elasticity

Elasticity is one of the most important services of cloud computing, referring to the ability to add or remove resources according to the needs of the application or service. In particular to the Internet of Things (IoT) scope, the use of this facility becomes pertinent since IoT requires a middleware that should be capable to handle high volume of data at real-time. Data can arrive in the middleware in parallel as in terms of input data from Radio-Frequency Identification (RFID) readers as request–reply query operations from the users side. Solutions modeled at software, hardware and/or architecture levels present limitations to handle such load. In this context, this article presents Proliot - a proactive elasticity model that combines cloud and high performance computing to address the IoT scalability problem in a novel EPCglobal-compliant architecture. The model can be seen as a service that keeps the same API but offers an elastic EPCIS component in the cloud, which is designed as a collection of virtual machines (VMs) that are automatically allocated and deallocated on-the-fly in accordance with the system load. The Proliot contribution consists in a mathematical formalism that uses Autoregressive Integrated Moving Average (ARIMA) and Weighted Moving Average to predict the IoT load behavior, so anticipating scaling in and out operations and then delivering VMs as close the moment they will be required as possible. Based on the Proliot model, we developed a prototype that was evaluated with different workload patterns against two concurrents: a threshold-based reactive elasticity model and non-elastic solution. The results were encouraging in favor of Proliot, presenting significant performance gains in terms of response time and request throughput.

Differential privacy for publishing enterprise-scale WLAN traces

Wireless trace data play an important role in wireless network researches. However, publishing the raw WLAN traces poses potential privacy risks of network users. Therefore, it is necessary to sanitize users’ sensitive information before these traces are published, and provide high data utility for wireless network researches as well. Although some existing works based on various anonymization methods have started to address the problem of sanitizing WLAN traces, we find the anonymization techniques cannot provide strong and provable privacy guarantee by analyzing a real WLAN trace dataset. Differential Privacy is the only framework that can provide strong and provable privacy guarantee. However, our analysis shows that existing studies on differential privacy fail to provide effective data utility for query operations on multi-dimensional and large-scale datasets. Aiming at WLAN trace datasets that have unique characteristics of multi-dimensional and large-scale, this paper proposes a privacy-preserving data publishing algorithm which not only satisfies differential privacy but also realizes high data utility for query operations. We prove that the proposed sanitization algorithm satisfies $$\epsilon $$ -differential privacy. Furthermore, the theoretical analysis shows the noise variance in our sanitization algorithm is $$O(\log ^{o(1)}n/\epsilon ^2)$$ which indicates the algorithm can achieve high data utility on large-scale datasets. Moreover, from the results of extensive experiments on an enterprise-scale WLAN trace dataset, we also show that our sanitization algorithm can provide high data utility for query operations.

Smart scheme: an efficient query execution scheme for event-driven stream processing

With the increase in stream data, a demand for stream processing has become diverse and complicated. To meet this demand, several stream processing engines (SPEs) have been developed which execute continuous queries (CQs) to process continuous data streams. Event-driven stream processing, which is one of the important requirements, continuously gets the incoming stream data and, however, generates query results only on the occurrence of specified events. In the basic query execution scheme, even when no event is raised, input stream tuples are continuously processed by query operators, though they do not generate any query result. This results in increased system load and wastage of system resources. For this problem, we propose a smart event-driven stream processing scheme, which makes use of smart windows to buffer the stream tuples during the absence of an event. When the event is raised, the buffered tuples are flushed and processed by the downstream operators. If the buffered tuples in the smart window expire due to the window size before the occurrence of an event, they are deleted directly from the smart window. Since CQs once registered are executed for several weeks, months or even years, SPEs usually execute several CQs in parallel and merge their query plans whenever possible to save processing cost. Due to the presence of smart window, existing multi-query optimization techniques cannot work for smart event-driven stream processing. Hence, this work proposes a multi-query optimization for the proposed smart scheme to cover the cases where multiple continuous queries are registered. Extensive experiments are performed on real and synthetic data streams to show the effectiveness of the proposed smart scheme and its multi-query optimization.

Query Optimization on Distributed Database Dengue Fever by Minimizing Attribute Involvement

Query optimization is an important task in a client/server environment of a distributed database, whereas a health epidemiologist data distribution based on DBD data on Geographic Information Systems (GIS). A proper method for a particular query process function is needed to generate query optimization on a distributed database. The query process requires important attention especially in distributed databases because the result of a cost-based query process is accessed by involving a number of attributes and visited sites. A query operation typically will search for data from various attributes in a scattered database table, although the processes do not require all table attributes. Query optimization requires minimum query operating costs (communication costs and access fees). The query cost can be optimized by separating attributes that are not required by the query. This can reduce the amount of communication and access time. The attributes should not be divided indiscriminately to obtain the best result of the query process and a vertical fragmentation method can be used to perform such attribute separation. In this research, attributes separation using vertical fragmentation method for a database health table is studied by comparing Bond Energy Algorithm (BEA) and Graphic Based Vertical Partitioning (GBVP) algorithm. The initial result of vertical fragmentation in both algorithms is the determination of types of attributes separated from a number of specific query process. The result of the separation of attributes from each algorithm is compared and evaluated using Partitioned Evaluator (PE) in order to achieve the access cost of several attributes. The results show that GBVP algorithm is more optimal for use in vertical table fragmentation process applied as query operation on distributed DBD database in a health field. The GBVP algorithm has less computational complexity, results a higher partition evaluator value and has lower query execution time than BEA.