Query Processing Capabilities Research Articles

Space‐efficient data structures for the inference of subsumption and disjointness relations

AbstractConventional database systems function as static data repositories, storing vast amounts of facts and offering efficient query processing capabilities. The sheer volume of data these systems store has a direct impact on their scalability, both in terms of storage space and query processing time. Deductive database systems, on the other hand, require far less storage space since they derive new knowledge by applying inference rules. The challenge is how to efficiently obtain the required derivations, compared to having them in explicit form. In this study, we concentrate on a set of predefined inference rules for subsumption and disjointness relations, including their negations. We use compact data structures to store the facts and provide algorithms to support each type of relation, minimizing even further the storage space requirements. Our experimental findings demonstrate the feasibility of this approach, which not only saves space but is often faster than a baseline that uses well‐known graph traversal algorithms implemented on top of a traditional adjacency list representation to derive the relations.

Revisiting B-tree Compression: An Experimental Study

B-trees are widely recognized as one of the most important index structures in database systems, providing efficient query processing capabilities. Over the past few decades, many techniques have been developed to enhance the efficiency of B-trees from various perspectives. Among them, B-tree compression is an important technique introduced as early as the 1970s to improve both space efficiency and query performance. Since then, several B-tree compression techniques have been developed. However, to our surprise, we have found that these B-tree compression techniques were never compared against each other in prior works. Consequently, many important questions remain unanswered, such as whether B-tree compression is truly effective or not. If it is effective, under what scenarios and which B-tree compression methods should be employed? In this paper, we conduct the first experimental evaluation of seven widely used B-tree compression techniques using both synthetic and real datasets. Based on our evaluation, we present lessons and insights that can be leveraged to guide system design decisions in modern databases regarding the use of B-tree compression.

Research on Classroom Teaching Evaluation and Instruction System Based on GIS Mobile Terminal

With the development of mobile terminal technology, GIS has begun to be applied in all walks of life. With its powerful spatial indexing, positioning, query, analysis, and graphics processing capabilities, it has become the evaluation and guidance analysis of domestic and foreign scholars. As an essential tool to improve the quality of teaching in our country’s primary and secondary schools and universities and to understand the status quo of education and teaching levels, appropriate teaching evaluation and guidance are essential. In order to adopt a better method to carry out teaching work, this research uses GIS mobile terminal to analyze the classroom teaching evaluation and guidance system, and through the research on teaching evaluation, the teaching work is made more effective. This paper uses questionnaire survey method, index weighted evaluation method, and fuzzy analytic hierarchy process as research methods. In the experimental part, a classroom teaching evaluation and guidance system based on GIS mobile terminal is designed. In the analysis part, a comprehensive analysis of teaching evaluation and guidance is carried out, including the analysis of teacher teaching evaluation, the analysis of teaching evaluation system, and the analysis of teachers and students’ understanding of teaching evaluation. Among them, after adopting the teaching evaluation system of GIS mobile terminal, the number of students who thought the teaching evaluation system was useless dropped from 4.6% to 1.8%. Students tend to emphasize the teacher’s effectiveness in teaching (57.2%) and interesting teaching content in the classroom (56.9%). This analysis has a certain significance for teaching evaluation, and it occupies a relatively important proportion in the improvement of the evaluation system. In general, the teaching evaluation system using GIS mobile terminals can improve the communication and teaching diagnosis between teachers and students and help further improve teaching.

AxleDB: A novel programmable query processing platform on FPGA

With the rise of Big Data, providing high-performance query processing capabilities through the acceleration of the database analytic has gained significant attention. Leveraging Field Programmable Gate Array (FPGA) technology, this approach can lead to clear benefits. In this work, we present the design and implementation of AxleDB: An FPGA-based platform that enables fast query processing for database systems by melding novel database-specific accelerators with commercial-off-the-shelf (COTS) storage using modern interfaces, in a novel, unified, and a programmable environment. AxleDB can perform a large subset of SQL queries through its set of instructions that can map compute-intensive database operations, such as filter, arithmetic, aggregate, group by, table join, or sort, on to the specialized high-throughput accelerators. To minimize the amount of SSD I/O operations required, AxleDB also supports hardware MinMax indexing for databases. We evaluated AxleDB with five decision support queries from the TPC-H benchmark suite and achieved a speedup from 1.8X to 34.2X and energy efficiency from 2.8X to 62.1X, in comparison to the state-of-the-art DBMS, i.e., PostgreSQL and MonetDB.

Effective 3-D surface modeling for geographic information systems

Abstract. In this work, we propose a dynamic, flexible and interactive urban digital terrain platform with spatial data and query processing capabilities of geographic information systems, multimedia database functionality and graphical modeling infrastructure. A new data element, called Geo-Node, which stores image, spatial data and 3-D CAD objects is developed using an efficient data structure. The system effectively handles data transfer of Geo-Nodes between main memory and secondary storage with an optimized directional replacement policy (DRP) based buffer management scheme. Polyhedron structures are used in digital surface modeling and smoothing process is performed by interpolation. The experimental results show that our framework achieves high performance and works effectively with urban scenes independent from the amount of spatial data and image size. The proposed platform may contribute to the development of various applications such as Web GIS systems based on 3-D graphics standards (e.g., X3-D and VRML) and services which integrate multi-dimensional spatial information and satellite/aerial imagery.

Using a Time Granularity Table for Gradual Granular Data Aggregation

The majority of today's systems increasingly require sophisticated data management as they need to store and to query large amounts of data for analysis and reporting purposes. In order to keep more “detailed” data available for longer periods, “old” data has to be reduced gradually to save space and improve query performance, especially on resource-constrained systems with limited storage and query processing capabilities. A number of data reduction solutions have been developed, however an effective solution particularly based on gradual data reduction is missing. This paper presents an effective solution for data reduction based on gradual granular data aggregation. With the gradual granular data aggregation mechanism, older data can be made coarse-grained while keeping the newest data fine-grained. For instance, when data is 3 months old aggregate to 1 minute level from 1 second level, when data is 6 months old aggregate to 2 minutes level from 1 minute level and so on. The proposed solution introduces a time granularity based data structure, namely a relational time granularity table that enables long term storage of old data by maintaining it at different levels of granularity and effective query processing due to a reduction in data volume. In addition, the paper describes the implementation strategy derived from a farming case study using standard database technologies.

Performance Analysis Measurement of Storage according to INNODB Parameter Change

This study analyzed the ad-hoc query processing capability according to changes of parameter values by performing TPC-H benchmark with 1GB database targeting HDD to find out optimized parameters of INNODB storage engine type parameters of MYSQL in accordance with data storage device. As a result of performing test for three parameters(innodb_buffer_pool_size, inndb_additional_mem_pool_size, innodb_log_buffer_size) among INNODB storage engine type parameters that are known to affect performance, as innodb_buffer_pool_size got bigger, QphH(Query-per-Hour) got better but the rest two parameters couldn’t affect QphH. Therefore, more researches about the case that the types of data storage device of servers are different from one another and bigger database capacity.

Advanced Data Processing in the Business Network System

The discovery, representation and reconstruction of Business Networks (BN) from Network Mining (NM) raw data is a difficult problem for enterprises. This is due to huge amounts of e.g. complex business processes within and across enterprise boundaries, heterogeneous technology stacks, and fragmented data. To remain competitive, visibility into the enterprise and partner networks on different, interrelated abstraction levels is desirable. We show the query and data processing capabilities of a novel data discovery, mining and network inference system, called Business Network System (BNS) that reconstructs the BN - integration and business process networks - from raw data, hidden in the enterprises' landscapes. The paper covers both the foundation and the key data processing characteristics features of BNS, including its underlying technologies, its overall system architecture, and data provenance approach.

Discovery of Hidden Relationship in a Large Data Itemsets through Apriori Algorithm of Association Analysis with UML

An association rule is a method to find out the frequent hidden relationship from a large amount of datasets in a database. Association analysis into existing database technology is very useful for indexing and query processing capabilities of database system and developing efficient and scalable mining algorithms as well as handling user specified or domain specific constraints and post processing the extracted patterns. In the present work, a methodology known as association analysis is presented which is very useful for discovery of interesting relationship hidden in large dataset, and an algorithm for generation of frequent data item set known as Apriori algorithm is used and validated the relations through Unified Modeling Language (UML). Authors used the lattice structure and also discussed the various association rules for the frequent data itemset which is found by Apriori algorithm. The different strategies in generation and traversal are breadth first and depth first search traversal. These techniques provide different tradeoff in terms of the input and output memory and computational time requirements. The entire concept is implemented by considering a real case study of Vehicle Insurance Policy system (VIPS) in context of Indian scenario.

Online visualization of geospatial stream data using the worldwide telescope

This demo presents the ongoing effort to meld the stream query processing capabilities of Microsoft StreamInsight with the visualization capabilities of the WorldWide Telescope. This effort provides visualization opportunities to manage, analyze, and process real-time information that is of spatio-temporal nature. The demo scenario is based on detecting, tracking and predicting interesting patterns over historical logs and real-time feeds of earthquake data.

Geospatial stream query processing using Microsoft SQL Server StreamInsight

Microsoft SQL Server spatial libraries contain several components that handle geometrical and geographical data types. With advances in geo-sensing technologies, there has been an increasing demand for geospatial streaming applications. Microsoft SQL Server StreamInsight ( StreamInsight , for brevity) is a platform for developing and deploying streaming applications that run continuous queries over high-rate streaming events. With its extensibility infrastructure, StreamInsight enables developers to integrate their domain expertise within the query pipeline in the form of user defined modules. This demo utilizes the extensibility infrastructure in Microsoft StreamInsight to leverage its continuous query processing capabilities in two directions. The first direction integrates SQL spatial libraries into the continuous query pipeline of StreamInsight. StreamInsight provides a well-defined temporal model over incoming events while SQL spatial libraries cover the spatial properties of events to deliver a solution for spatiotemporal stream query processing. The second direction extends the system with an analytical refinement and prediction layer. This layer analyzes historical data that has been accumulated and summarized over the years to refine, smooth and adjust the current query output as well as predict the output in the near future. The demo scenario is based on transportation data in Los Angeles County.

Providing built-in keyword search capabilities in RDBMS

A common approach to performing keyword search over relational databases is to find the minimum Steiner trees in database graphs transformed from relational data. These methods, however, are rather expensive as the minimum Steiner tree problem is known to be NP-hard. Further, these methods are independent of the underlying relational database management system (RDBMS), thus cannot benefit from the capabilities of the RDBMS. As an alternative, in this paper we propose a new concept called Compact Steiner Tree (CSTree), which can be used to approximate the Steiner tree problem for answering top-k keyword queries efficiently. We propose a novel structure-aware index, together with an effective ranking mechanism for fast, progressive and accurate retrieval of top-k highest ranked CSTrees. The proposed techniques can be implemented using a standard relational RDBMS to benefit from its indexing and query-processing capability. We have implemented our techniques in MYSQL, which can provide built-in keyword-search capabilities using SQL. The experimental results show a significant improvement in both search efficiency and result quality comparing to existing state-of-the-art approaches.

Improving the QoS for information discovery in autonomic wireless sensor networks

Wireless sensor networks (WSN) are attractive for information gathering in large-scale data rich environments. In order to fully exploit the data gathering and dissemination capabilities of these networks, energy-efficient and scalable solutions for data storage and information discovery are essential. Traditionally, the communication pattern in WSNs has been assumed to be many-to-one; i.e., numerous sensors gather information which is routed to a central point commonly referred to as the sink. However, many emerging applications for WSNs require dissemination of information to interested clients within the network requiring support for differing traffic patterns. Further, in-network query processing capabilities are required for autonomic information discovery. In this paper, we formulate the information discovery problem as a load-balancing problem, with the combined aim being to maximize network lifetime and minimize query processing delay resulting in quality of service (QoS) improvements. We propose novel methods for data dissemination, information discovery and data aggregation that are designed to provide significant QoS benefits. We make use of affinity propagation to group “similar” sensors and have developed efficient mechanisms that can resolve both ALL-type and ANY-type queries in-network with improved energy-efficiency and query resolution time. Simulation and Analytical results prove the proposed method(s) of information discovery offer significant QoS benefits for ALL-type and ANY-type queries in comparison to previous approaches.

Probabilistic top- k and ranking-aggregate queries

Ranking and aggregation queries are widely used in data exploration, data analysis, and decision-making scenarios. While most of the currently proposed ranking and aggregation techniques focus on deterministic data, several emerging applications involve data that is unclean or uncertain. Ranking and aggregating uncertain (probabilistic) data raises new challenges in query semantics and processing, making conventional methods inapplicable. Furthermore, uncertainty imposes probability as a new ranking dimension that does not exist in the traditional settings. In this article we introduce new probabilistic formulations for top- k and ranking-aggregate queries in probabilistic databases. Our formulations are based on marriage of traditional top- k semantics with possible worlds semantics. In the light of these formulations, we construct a generic processing framework supporting both query types, and leveraging existing query processing and indexing capabilities in current RDBMSs. The framework encapsulates a state space model and efficient search algorithms to compute query answers. Our proposed techniques minimize the number of accessed tuples and the size of materialized search space to compute query answers. Our experimental study shows the efficiency of our techniques under different data distributions with orders of magnitude improvement over naïve methods.

A semantic portal for the tourism and hospitality industry: its design, use and acceptance

A 'Semantic Portal' is similar to a traditional cyberspace portal, except that web resources are indexed using a rich domain ontology (a specification of key domain concepts) as opposed to, for example, a list of keywords. 'AcOntoWeb' (Accommodation Onto the Web) was originally designed as a website (and web services) package, customised for accommodation enterprises within the T&H sector. It has now been implemented as a semantic portal and, in this paper, we provide an overview of the portal, with an emphasis on cross-portal ontology integration and its powerful query processing capabilities. A survey, assessing the willingness of Australian accommodation enterprise operators to use this new technology, is also presented.

Cost-based optimization in DB2 XML

DB2 XML is a hybrid database system that combines the relational capabilities of DB2 Universal DatabaseTM (UDB) with comprehensive native XML support. DB2 XML augments DB2® UDB with a native XML store, XML indexes, and query processing capabilities for both XQuery and SQL/XML that are integrated with those of SQL. This paper presents the extensions made to the DB2 UDB compiler, and especially its cost-based query optimizer, to support XQuery and SQL/XML queries, using much of the same infrastructure developed for relational data queried by SQL. It describes the challenses to the relational infrastructure that supporting XQuery and SQL/XML poses and provides the rationale for the extensions that were made to the three main parts of the optimizer: the plan operators, the cardinality and cost model, and statistics collection.

Building Scalable Electronic Market Places Using HyperQuery-Based Distributed Query Processing

Flexible distributed query processing capabilities are an important prerequisite for building scalable Internet applications, such as electronic Business-to-Business (B2B) market places. Architecting an electronic market place in a conventional data warehouse-like approach by integrating all the data from all participating enterprises in one centralized repository incurs severe problems: stale data, data security threats, administration overhead, inflexibility during query processing, etc. In this paper we present a new framework for dynamic distributed query processing based on so-called HyperQueries which are essentially query evaluation sub-plans "sitting behind" hyperlinks. Our approach facilitates the pre-materialization of static data at the market place whereas the dynamic data remains at the data sources. In contrast to traditional data integration systems, our approach executes essential (dynamic) parts of the data-integrating views at the data sources. The other, more static parts of the data are integrated a priori at the central portal, e.g., the market place. The portal serves as an intermediary between clients and data providers which execute their sub-queries referenced via hyperlinks. The hyperlinks are embedded as attribute values within data objects of the intermediary's database. Retrieving such a virtual object will execute the referenced HyperQuery in order to materialize the missing data. We illustrate the flexibility of this distributed query processing architecture in the context of B2B electronic market places with an example derived from the car manufacturing industry. Based on these HyperQueries, we propose a reference architecture for building scalable and dynamic electronic market places. All administrative tasks in such a distributed B2B market place are modeled as Web services and are initiated decentrally by the participants. Thus, sensitive data remains under the full control of the data providers. We describe optimization and implementation issues to obtain an efficient and highly flexible data integration platform for electronic market places. All proposed techniques have been fully implemented in our QueryFlow prototype system which served as the platform for our performance evaluation.

ObjectGlobe: Ubiquitous query processing on the Internet

We present the design of ObjectGlobe, a distributed and open query processor for Internet data sources. Today, data is published on the Internet via Web servers which have, if at all, very localized query processing capabilities. The goal of the ObjectGlobe project is to establish an open marketplace in which data and query processing capabilities can be distributed and used by any kind of Internet application. Furthermore, ObjectGlobe integrates cycle providers (i.e., machines) which carry out query processing operators. The overall picture is to make it possible to execute a query with – in principle – unrelated query operators, cycle providers, and data sources. Such an infrastructure can serve as enabling technology for scalable e-commerce applications, e.g., B2B and B2C market places, to be able to integrate data and data processing operations of a large number of participants. One of the main challenges in the design of such an open system is to ensure privacy and security. We discuss the ObjectGlobe security requirements, show how basic components such as the optimizer and runtime system need to be extended, and present the results of performance experiments that assess the additional cost for secure distributed query processing. Another challenge is quality of service management so that users can constrain the costs and running times of their queries.

DiscoveryLink: A system for integrated access to life sciences data sources

Vast amounts of life sciences data reside today in specialized data sources, with specialized query processing capabilities. Data from one source often must be combined with data from other sources to give users the information they desire. There are database middleware systems that extract data from multiple sources in response to a single query. IBM's DiscoveryLink is one such system, targeted to applications from the life sciences industry. DiscoveryLink provides users with a virtual database to which they can pose arbitrarily complex queries, even though the actual data needed to answer the query may originate from several different sources, and none of those sources, by itself, is capable of answering the query. We describe the DiscoveryLink offering, focusing on two key elements, the wrapper architecture and the query optimizer, and illustrate how it can be used to integrate the access to life sciences data from heterogeneous data sources.

Middleware object query processing with deferred updates and autonomous sources

This paper presents a query processing algorithm called DECAF for use in middleware object query systems that are based on the use of an object cache. The DECAF algorithm is designed to work correctly even in the presence of updates to the underlying databases that don't go through the object cache (i.e., even for autonomous data sources that can be updated through legacy applications that do not perform their updates through the middleware object layer). DECAF's query results are consistent with updates performed by such transactions; its results are also consistent with any deferred updates that are present in the object cache but not yet committed at the database server. The DECAF algorithm attempts to push down query predicates to the underlying DBMSs to take advantage of the query processing capabilities of these systems and to reduce the amount of data transferred from these systems to the object cache.