Data and knowledge base research at Hong Kong University of Science and Technology

Query evaluation and optimization in deductive and object-oriented spatial databases

Spatial Data Warehouses (SDWs) combine spatial databases (SDBs) and data warehouses (DWs) allowing analysis of historical data. This data can be queried using Spatial On-Line Analytical Processing (SOLAP). SDW and SOLAP systems are emerging areas that raise several research issues. In this paper, we refer to a different problem existing in SDWs that motivated us to propose a framework - a conceptual multidimensional model able to express users' requirements for SDW and SOLAP applications. We present a different research direction that is important to consider providing satisfactory solutions for SDW and for SOLAP systems. That important area is spatial query optimization. For the past several years, the research on spatial database systems has actively progressed because the applications using the spatial information such as geographic information systems (GIS), computer aided design (CAD) and multimedia systems have increased. However, most of the research has dealt with only a part of spatial database systems such as data models, spatial indexes, spatial join algorithms, or cost models. There has been a little research on the spatial query optimization which can integrate them. Most of the spatial query optimization techniques published until now has not properly reflected the characteristics of the SDBs. This paper presents query optimization strategies which take the characteristics of SDBs into account. The application of standard query processing and optimization techniques in the context of an integrated SDB environment is discussed.

With rise in new technologies like Big Data, Internet of Things, Sensor Networks, etc. new query processing and optimization techniques have been evolved which handles the complex and interactive queries. Traditional query processing systems are not capable enough to handle queries on big data. In this paper, we represent many of the big data querying techniques along with the comparative analysis. Finally, the paper concludes with list of tools/techniques that are used for big data query optimization.

An efficient query optimization strategy for spatio-temporal queries in video databases

An efficient query optimization strategy for spatio-temporal queries in video databases

Query execution and optimization for streaming data revisits almost all aspects of query execution and optimization over traditional, disk-bound database systems. The reason is that two fundamental assumptions of disk-bound systems are dropped: (i) the data resides on disk, and (ii) the data is finite. As such, new evaluation algorithms and new optimization metrics need to be devised. The approaches can be broadly classified into two categories. First, there are static approaches that follow the traditional optimize-then-execute paradigm by assuming that optimization-time assumptions will continue to hold during execution; the environment is expected to be relatively static in that respect. Alternatively, there are adaptive approaches that assume the environment is completely dynamic and highly unpredictable. In this chapter we explore both approaches and present novel query optimization and evaluation techniques for queries over streaming sources.

Distributed query processing and optimization techniques for a hierarchically structured computer network

New Database Architecture for Smart Query Handler of Spatial Database

We study the problem of query optimization in federated relational database systems. The nature of federated databases explicitly decouples many aspects of the optimization process, often making it imperative for the optimizer to consult underlying data sources while doing cost-based optimization. This not only increases the cost of optimization, but also changes the trade-offs involved in the optimization process significantly. The dominant cost in the decoupled optimization process is the "cost of costing" that traditionally has been considered insignificant. The optimizer can only afford a few rounds of messages to the underlying data sources and hence the optimization techniques in this environment must be geared toward gathering all the required cost information with minimal communication. In this paper, we explore the design space for a query optimizer in this environment and demonstrate the need for decoupling various aspects of the optimization process. We present minimum-communication decoupled variants of various query optimization techniques, and discuss tradeoffs in their performance in this scenario. We have implemented these techniques in the Cohera federated database system and our experimental results, somewhat surprisingly, indicate that a simple two-phase optimization scheme performs fairly well as long as the physical database design is known to the optimizer, though more aggressive algorithms are required otherwise.

Query processing is that method or technique in area of mobile atmosphere which deal using mobile environment. This research in that way user can think advancement using query processing and it takes more use for speedily accessing global query processing and it is joint venture of query processing between dissimilar sites includes fixed server movable computer. The necessity saving of energy and also the existence of lopsided features in a moveable computing atmosphere, the predictable query processing for a scattered database cannot be straightly genuine to a moveable computing organization. The mobile environment is a collection of mobile diverse hosts, which are enabled to communicate using wireless links. These wireless links are change giving to the natures of mobile networks. Moreover, nodes in the ad-hoc network have to link without any centralized or help. The usability for the user also changes using query processing tool. Thus, this mechanism that allows the sharing of functionality among different devices in same environment change the way of user communication for searching fast query time in mobile computing environment. The aim of this research is to make innovation in query optimization technique in mobile computing environment. This research focuses on improvement of various query optimization techniques for highly effective and trustworthy query optimization methods.

Geography Markup Language (GML) has been extended for mobile and location-based applications. Three schemas are included in the extension, i.e., voice schema, tracking schema, and POI (Point Of Interest) schema. In order to handle effectively the extension, we also have designed a Spatial XQuery language and its processing modules. Because our work is based on a spatial database system, the Spatial XQuery statements are first translated into SQL statements. By working on an existing spatial database system, we can use of the existing amenities of database systems such as query optimization, spatial indexes, concurrency control, and crash recovery. Translation of the Spatial XQuery into SQL has been explained using examples. Because the results from the spatial database system are in the form of tables, we need to translate again the results into XML statements. A working example of the proposed system as an Emergency Support System is also presented. Prospected application areas of the proposed system are almost all mobile and location-based systems such as LBS (Location-based System), POI systems, tracking systems, ubiquitous systems, and distributed control and management systems.

S.70-79

In the big data community, Spark plays an important role and is used to process interactive queries. Spark employs a query optimizer, called Catalyst, to interpret SQL queries to optimized query execution plans. Catalyst contains a number of optimization rules and supports cost-based optimization. Although query optimization techniques have been well studied in the field of relational database systems, the effectiveness of Catalyst in Spark is still unclear. In this paper, we investigated the effectiveness of rule-based and cost-based optimization in Catalyst, meanwhile, we obtained a set of comparative experiments by varying the data volume and the number of nodes. It is found that even when applied query optimizations, the execution time of most TPC-H queries were slightly reduced. Some interesting observations were made on Catalyst, which can enable the community to have a better understanding and improvement of the query optimizer in Spark.

Recent sensor networks research has produced a class of data storage and query processing techniques called Data-Centric Storage that leverages locality-preserving distributed indexes like DIM, DIFS, and GHT to efficiently answer multi-dimensional range and rangeaggregate queries. These distributed indexes offer a rich design space of a) logical decompositions of sensor relation schema into indexes, as well as b) physical mappings of these indexes onto sensors. In this poster, we explore this space for energy-efficient data organizations (logical and physical mappings of tuples and attributes to sensor nodes) and devise purely local query optimization techniques for processing queries that span such decomposed relations. We propose four design techniques: (a) fully decomposing the base sensor relation into distinct sub-relations, (b) spatially partitioning these sub-relations across the sensornet, (c) localized query planning and optimization to find fully decentralized optimal join orders, and (d) locally caching join results. Together, these optimizations reduce the overall network energy consumption by 4 times or more when compared against the standard single multidimensional distributed index on a variety of synthetic query workloads simulated over both synthetic and real-world datasets. We validate the feasibility of our approach by implementing a functional prototype of our data organizer and query processor on Mica2 motes and observing comparable message cost savings.

Recent sensor networks research has produced a class of data storage and query processing techniques called Data-Centric Storage that leverages locality-preserving distributed indexes like DIM, DIFS, and GHT to efficiently answer multi-dimensional range and range-aggregate queries. These distributed indexes offer a rich design space of a) logical decompositions of sensor relation schema into indexes, as well as b) physical mappings of these indexes onto sensors. In this poster, we explore this space for energy-efficient data organizations (logical and physical mappings of tuples and attributes to sensor nodes) and devise purely local query optimization techniques for processing queries that span such decomposed relations. We propose four design techniques: (a) fully decomposing the base sensor relation into distinct sub-relations, (b) spatially partitioning these sub-relations across the sensornet, (c) localized query planning and optimization to find fully decentralized optimal join orders, and (d) locally caching join results. Together, these optimizations reduce the overall network energy consumption by 4 times or more when compared against the standard single multi-dimensional distributed index on a variety of synthetic query workloads simulated over both synthetic and real-world datasets. We validate the feasibility of our approach by implementing a functional prototype of our data organizer and query processor on Mica2 motes and observing comparable message cost savings.