Query Transformation Research Articles

Transformation of Continuous Aggregation Join Queries over Data Streams

Aggregation join queries are an important class of queries over data streams. These queries involve both join and aggregation operations, with window-based joins followed by an aggregation on the join output. All existing research address join query optimization and aggregation query optimization as separate problems. We observe that, by putting them within the same scope of query optimization, more efficient query execution plans are possible through more versatile query transformations. The enabling idea is to perform aggregation before join so that the join execution time may be reduced. There has been some research done on such query transformations in relational databases, but none has been done in data streams. Doing it in data streams brings new challenges due to the incremental and continuous arrival of tuples. These challenges are addressed in this paper. Specifically, we first present a query processing model geared to facilitate query transformations and propose a query transformation rule specialized to work with streams. The rule is simple and yet covers all possible cases of transformation. Then we present a generic query processing algorithm that works with all alternative query execution plans possible with the transformation, and develop the cost formulas of the query execution plans. Based on the processing algorithm, we validate the rule theoretically by proving the equivalence of query execution plans. Finally, through extensive experiments, we validate the cost formulas and study the performances of alternative query execution plans.

Element classification‐based transformation of XML queries

PurposeEngines have been built that execute queries against XML data. The aim of this paper is to describe a novel technique that can be used to improve the speed of execution of the queries based on semantics of the data in the XML document.Design/methodology/approachThe paper formally introduces algorithms for optimizing XML queries, implement the algorithms, and through experimentation demonstrate the improvement in speed.FindingsThree possible semantic query optimizations based on the values of elements were introduced and these demonstrate that two of the three optimizations improve query performance but the third does not. It is hypothesized why this is the case.Research limitations/implicationsA limitation is obviously the query engine and how it works. Future work includes, executing the experiments on a different engine and comparing results, building a system to automatically generate the characteristics that are necessary to do the optimization, describing the best way to represent and maintain the characteristics once they are found, compare the results of optimizations based on content with optimizations based on structure.Practical implicationsThe optimizations could be incorporated into new query engines.Originality/valueNovel algorithms for query optimization have been developed and proven to work. They are of value to people who are building database systems for XML data.

Closing the query processing loop in Oracle 11g

The role of a query optimizer in a database system is to find the best execution plan for a given SQL statement based on statistics about the objects related to the tables referenced in the statement. These statistics include the tables themselves, their indexes, and other derived objects. Statistics include the number of rows, space utilization on disk, distribution of column values, etc. Optimization also relies on system statistics, such as the I/O bandwidth of the storage sub-system. All of this information is fed into a cost model. The cost model is used to compute the cost whenever the query optimizer needs to make a decision for an access path, join method, join order, or query transformation. The optimizer picks the alternative that yields the lowest cost. The quality of the final execution plan depends primarily on the quality of the information fed into the cost model as well as the cost model itself. In this paper, we discuss two of the problems that affect the quality of execution plans generated by the query optimizer: the cardinality of intermediate results and host variable values. We will give details of the solutions we introduced in Oracle 11g. Our approach establishes a bridge from the SQL execution engine to the SQL compiler. The bridge brings valuable information to help the query optimizer assess the impact of its decisions and make better decisions for future executions of the SQL statement. We illustrate the merits of our solutions based on experiments using the Oracle E-Business Suite workload.

The POINT approach to represent now in bitemporal databases

Most modern database applications involve a significant amount of time dependent data and a significant portion of this data is now-relative. Now-relative data are a natural and meaningful part of every temporal database as well as being the focus of most queries. Previous studies indicate that the choice of the representation of now significantly influences the efficiency of accessing bitemporal data. In this paper we propose and experimentally evaluate a novel approach to represent now that we termed the POINT approach, in which now-relative facts are represented as points on the transaction-time and/or valid-time line. Furthermore, in the POINT approach we propose a logical query transformation that relies on the above representation and on the geometry features of spatial access methods. Such a logical query transformation enables off-the-shelf spatial indexes to be used. We empirically prove that the POINT approach is efficient on now-relative bitemporal data, outperforming the maximum timestamp approach that has been proven to the best approach to now-relative data in the literature, independently of the indexing methodology (B ?+?- tree vs R *- tree) being used. Specifically, if spatial indexing is used, the POINT approach outperforms the maximum timestamp approach to the extent of factor more than 10, both in number of disk accesses and CPU usage.

Enriching the class diagram concepts to capture natural language semantics for database access

Research on accessing databases using natural languages usually utilizes an intermediate logical form for the mapping process from natural languages to database query languages. However, there is still much that needs to be accomplished to bridge the gap between natural language constructs and database schemas. In this paper, we present a translation scheme for transforming natural language queries into relational algebra through the class diagram representations. Based on a logical form developed by extending the UML class diagram notations, a transformation model is presented to support the automatic transformation of natural language queries into relational algebra by employing appropriate natural language processing techniques and object-oriented analysis methods. The proposed logical form has the advantage that it can be mapped from natural language constructs by referring to the conceptual schema modeled by class diagrams, and can be efficiently transformed into relational algebra for query execution. We believe the whole process could offer a clear and natural framework for processing natural language queries to retrieve data from database systems.

Coupled Transformation of Schemas, Documents, Queries, and Constraints

Coupled transformation occurs when multiple software artifacts must be transformed in such a way that they remain consistent with each other. For instance, when a database schema is adapted in the context of system maintenance, the persistent data residing in the system's database needs to be migrated to conform to the adapted schema. Also, queries embedded in the application code and any declared referential constraints must be adapted to take the schema changes into account. As another example, in XML-to-relational data mapping, a hierarchical XML Schema is mapped to a relational SQL schema with appropriate referential constraints, and the XML documents and queries are converted into relational data and relational queries. The 2LT project is aimed at providing a formal basis for coupled transformation. This formal basis is found in data refinement theory, point-free program calculation, and strategic term rewriting. We formalize the coupled transformation of a data type by an algebra of information-preserving data refinement steps, each witnessed by appropriate data conversion functions. Refinement steps are modeled by so-called two-level rewrite rules on type expressions that synthesize conversion functions between redex and reduct while rewriting. Strategy combinators are used to composed two-level rewrite rules into complete rewrite systems. Point-free program calculation is applied to optimized synthesize conversion function, to migrate queries, and to normalize data type constraints. In this paper, we provide an overview of the challenges met by the 2LT project and we give a sketch of the solutions offered.

Determined: a system with novel techniques for XML query optimization and evaluation

PurposeEfficient processing of XML queries is critical for XML data management and related applications. Previously proposed techniques are unsatisfactory. The purpose of this paper is to present Determined – a new prototype system designed for XML query processing and optimization from a system perspective. With Determined, a number of novel techniques for XML query processing are proposed and demonstrated.Design/methodology/approachThe methodology emphasizes on query pattern minimization, logic‐level optimization, and efficient query execution. Accordingly, three lines of investigation have been pursued in the context of Determined: XML tree pattern query (TPQ) minimization; logic‐level XML query optimization utilizing deterministic transformation; and specialized algorithms for fast XML query execution.FindingsDeveloped and demonstrated were: a runtime optimal and powerful algorithm for XML TPQ minimization; a unique logic‐level XML query optimization approach that solely pursues deterministic query transformation; and a group of specialized algorithms for XML query evaluation.Research limitations/implicationsThe experiments conducted so far are still preliminary. Further in‐depth, thorough experiments thus are expected, ideally carried out in the setting of a real‐world XML DBMS system.Practical implicationsThe techniques/approaches proposed can be adapted to real‐world XML database systems to enhance the performance of XML query processing.Originality/valueThe reported work integrates various novel techniques for XML query processing/optimization into a single system, and the findings are presented from a system perspective.

Query Transformation Based on Semantic Centrality in Semantic Social Network1

Query Transformation Based on Semantic Centrality in Semantic Social Network1

Data mining of search engine logs

AbstractThis article reports on the development of a novel method for the analysis of Web logs. The method uses techniques that look for similarities between queries and identify sequences of “query transformation”. It allows sequences of query transformations to be represented as graphical networks, thereby giving a richer view of search behavior than is possible with the usual sequential descriptions. We also perform a basic analysis to study the correlations between observed transformation codes, with results that appear to show evidence of behavior habits. The method was developed using transaction logs from the Excite search engine to provide a tool for an ongoing research project that is endeavoring to develop a greater understanding of Web‐based searching by the general public.

In this issue

In this issue

Database querying under changing preferences

We present here a formal foundation for an iterative and incremental approach to constructing and evaluating preference queries. Our main focus is query modification: a query transformation approach which works by revising the preference relation in the query. We provide a detailed analysis of the cases where the order-theoretic properties of the preference relation are preserved in the revision. We consider a number of different revision operators: union, prioritized and Pareto composition. We also formulate algebraic laws that enable incremental evaluation of preference queries. Finally, we consider two variations of the basic framework: finite restrictions of preference relations and weak-order extensions of strict partial order preference relations.

Taxonomy alignment for interoperability between heterogeneous virtual organizations

Resources in virtual organizations are classified based on their local taxonomies. However, heterogeneity between these taxonomies is a serious problem for efficient cooperation processes (e.g., knowledge sharing and querying-based interactions). In order to overcome this problem, we propose a novel framework based on aligning the taxonomies of virtual organizations. Thereby, the best mapping between two organization taxonomies has to be discovered to maximize the summation of a set of partial similarities between concepts in the taxonomies. We can consider two levels of alignment processes; (i) intra-alignment in a virtual organization for building an organizational taxonomy and (ii) inter-alignment between organizational taxonomies. Particularly, for intra-alignment, features extracted from resources are exploited to enhance the precision of similarity measurement between concepts. For experimentation, twelve virtual organizations have been built with different local taxonomies. The proposed inter-alignment method has shown about 76% of precision and 68% of recall. Also, feature-based intra-alignment improved those performance, during resource retrieval by query transformation. In addition, we found out that alignment results are dependent on some characteristics of taxonomies (e.g., depth and number of classes).

관계 데이터베이스에서 수평 뷰 테이블에 대한 PIVOT 기반의 질의 최적화 방법

For effective analyses in various business applications, OLAP(On-Line Analytical Processing) systems represent the multidimensional data as the horizontal format of tables whose columns are corresponding to values of dimension attributes. Because the traditional RDBMSs have the limitation on the maximum number of attributes in table columns(MS SQLServer and Oracle permit each table to have up to 1,024 columns), horizontal tables cannot be directly stored into relational database systems. In this paper, we propose various efficient optimization strategies in transforming horizontal queries to equivalent vertical queries. To achieve this goral, we first store a horizontal table using an equivalent vertical table, and then develop various query transformation rules for horizontal table queries using the PIVOT operator. In particular, we propose various alternative query transformation rules for the basic relational operators, selection, projection, and join. Here, we note that the transformed queries can be executed in several ways, and their execution times will differ from each other. Thus, we propose various optimization strategies that transform the horizontal queries to the equivalent vertical queries when using the PIVOT operator. Finally, we evaluate these methods through extensive experiments and identify the optimal transformation strategy when using the PIVOT operator.

Semantic Query Optimization: Correctness and Control

We developed a semantic query optimization framework for deductive databases based on equivalent transformation (ET) rules. ET rules, prepared from the semantic knowledge of databases, such as integrity constraints, transform given queries into syntactically different but semantically equivalent and more efficient forms. We formally prove the correctness of query transformations by ET rules. For efficiency, we propose a two-phase heuristic-based strategy to guide query transformations and introduce a condition-based control strategy to prevent unwanted, unnecessary transformations. We give examples demonstrating the possible optimization.

Incorporating a virtual union catalogue into the wider information environment through the application of middleware

PurposeTo explore the use of middleware to provide access to a virtual catalogue via Z39.50, allowing its integration with other services. To consider issues arising from the cross‐database search process and the problem of interoperability between systems.Design/methodology/approachMiddleware was used to establish a Z39.50 server for an existing virtual union catalogue, InforM25. The Copac physical union catalogue was used to carry out cross‐database search tests, investigating the value of query transformation by the middleware and illustrating many of the issues associated with interoperability in cross‐database searching.FindingsThe test results demonstrate the viability and benefits of using middleware to allow the integration of virtual catalogues with other services. The results also illustrate the range of issues affecting the performance of the virtual catalogue, with supporting evidence from the use of the Copac physical union catalogue for cross‐database searching via Z39.50. In particular the discussion highlights the issue of semantic interoperability, as well as emphasising the value of centralised virtual catalogue support.Research limitations/implicationsThe research was limited to work on a specific virtual union catalogue, that is InforM25, but the results highlight issues with general relevance in the development of any virtual catalogue. The need for more work on response times in the virtual union catalogue is apparent, along with the related issue of result set post processing.Originality/valueThis paper provides a practical demonstration of the issues important to those involved in making their local catalogue accessible to services, and individuals, outside their own institution, as well as those working on virtual union catalogue development.

Hybrid Query and Data Ordering for Fast and Progressive Range-Aggregate Query Answering

Data analysis systems require range-aggregate query answering of large multidimensional datasets. We provide the necessary framework to build a retrieval system capable of providing fast answers with progressively increasing accuracy in support of range-aggregate queries. In addition, with error forecasting, we provide estimations on the accuracy of the generated approximate results. Our framework utilizes the wavelet transformation of query and data hypercubes. While prior work focused on the ordering of either the query or the data coefficients, we propose a class of hybrid ordering techniques that exploits both query and data wavelets in answering queries progressively. This work effectively subsumes and extends most of the current work where wavelets are used as a tool for approximate or progressive query evaluation. The results of our experimental studies show that independent of the characteristics of the dataset, the data coefficient ordering, contrary to the common belief, is the inferior approach. Hybrid ordering, on the other hand, performs best for scientific datasets that are inter-correlated. For an entirely random dataset with no inter-correlation, query ordering is the superior approach.

Merging Results for Distributed Content Based Image Retrieval

Searching information through the Internet often requires users to separately contact several digital libraries, use each library interface to author the query, analyze retrieval results and merge them with results returned by other libraries. Such a solution could be simplified by using a centralized server that acts as a gateway between the user and several distributed repositories: The centralized server receives the user query, forwards the user query to federated repositories—possibly translating the query in the specific format required by each repository—and fuses retrieved documents for presentation to the user. To accomplish these tasks efficiently, the centralized server should perform some major operations such as: resource selection, query transformation and data fusion. In this paper we report on some aspects of MIND, a system for managing distributed, heterogeneous multimedia libraries (MIND, 2001, http://www.mind-project.org). In particular, this paper focusses on the issue of fusing results returned by different image repositories. The proposed approach is based on normalization of matching scores assigned to retrieved images by individual libraries. Experimental results on a prototype system show the potential of the proposed approach with respect to traditional solutions.

Scalable QSF-Trees

Many database applications require effective representation of regional objects in high-dimensional spaces. By applying an original query transformation, a recently proposed access method for regional data, called the simple QSF-tree (sQSF-tree), effectively attacks the limitations of traditional spatial access methods in spaces with many dimensions. Nevertheless, sQSF-trees are not immune to all problems associated with high data dimensionality. Based on the analysis of sQSF-trees, this paper presents a new variant of sQSF-trees, called the scalable QSF-tree (cQSF-tree), which relies on a heuristic optimization to reduce the number of false drops into pages that contain no object satisfying the query. By increasing the selectivity of search predicates, cQSF-trees improve the performance of multi-dimensional selections. Experimental evidence shows that cQSF-trees are more scalable than sQSF-trees to the growing data dimensionality. The performance improvements also increase with more skewed data distribution.

Learning to find answers to questions on the Web

We introduce a method for learning to find documents on the Web that contain answers to a given natural language question. In our approach, questions are transformed into new queries aimed at maximizing the probability of retrieving answers from existing information retrieval systems. The method involves automatically learning phrase features for classifying questions into different types, automatically generating candidate query transformations from a training set of question/answer pairs, and automatically evaluating the candidate transformations on target information retrieval systems such as real-world general purpose search engines. At run-time, questions are transformed into a set of queries, and reranking is performed on the documents retrieved. We present a prototype search engine, Tritus , that applies the method to Web search engines. Blind evaluation on a set of real queries from a Web search engine log shows that the method significantly outperforms the underlying search engines, and outperforms a commercial search engine specializing in question answering. Our methodology cleanly supports combining documents retrieved from different search engines, resulting in additional improvement with a system that combines search results from multiple Web search engines.

Bringing order to query optimization

A variety of developments combine to highlight the need for respecting order when manipulating relations. For example, new functionality is being added to SQL to support OLAP-style querying in which order is frequently an important aspect. The set- or multiset-based frameworks for query optimization that are currently being taught to database students are increasingly inadequate.This paper presents a foundation for query optimization that extends existing frameworks to also capture ordering. A list-based relational algebra is provided along with three progressively stronger types of algebraic equivalences, concrete query transformation rules that obey the different equivalences, and a procedure for determining which types of transformation rules are applicable for optimizing a query. The exposition follows the style chosen by many textbooks, making it relatively easy to teach this material in continuation of the material covered in the textbooks, and to integrate this material into the textbooks.