Computation Of Queries Research Articles

Optimizing the execution of multiple data analysis queries on parallel and distributed environments

We investigate techniques for efficiently executing multiquery workloads from data and computation-intensive applications in parallel and/or distributed computing environments. In this context, we describe a database optimization framework that supports data and computation reuse, query scheduling, and active semantic caching to speed up the evaluation of multiquery workloads. Its most striking feature is the ability of optimizing the execution of queries in the presence of application-specific constructs by employing a customizable data and computation reuse model. Furthermore, we discuss how the proposed optimization model is flexible enough to work efficiently irrespective of the parallel/distributed environment underneath. In order to evaluate the proposed optimization techniques, we present experimental evidence using real data analysis applications. For this purpose, a common implementation for the queries under study was provided according to the database optimization framework and deployed on top of three distinct experimental configurations: a shared memory multiprocessor, a cluster of workstations, and a distributed computational Grid-like environment.

PSoup: a system for streaming queries over streaming data

Abstract.Recent work on querying data streams has focused on systems where newly arriving data is processed and continuously streamed to the user in real time. In many emerging applications, however, ad hoc queries and/or intermittent connectivity also require the processing of data that arrives prior to query submission or during a period of disconnection. For such applications, we have developed PSoup, a system that combines the processing of ad hoc and continuous queries by treating data and queries symmetrically, allowing new queries to be applied to old data and new data to be applied to old queries. PSoup also supports intermittent connectivity by separating the computation of query results from the delivery of those results. PSoup builds on adaptive query-processing techniques developed in the Telegraph project at UC Berkeley. In this paper, we describe PSoup and present experiments that demonstrate the effectiveness of our approach.

Computable Queries for Object Oriented Databases

A relational database can be considered as a finite structure for a finite relational signature in first-order logic, i.e., there are no function symbols. Interpreting the logic over this signature in such structures allows the expressiveness and complexity of queries to be studied in detail. This is the starting point for finite model theory which has proven to be a viable tool to study relational database theory. In particular, it is known that computable queries expressed as isomorphism-preserving partial recursive functions can be formalized by Reflective Relational Machines. These are extended Turing Machines with an additional relational store, a query tape and the facility to evaluate the query on the tape against the database in the store in a single step.In this paper we start to generalize the theory to post-relational databases. We first consider the case of having set-based complex values and references such that the semantics can still be expressed in finite sets. Following the approach that object oriented databases in general including those, where the underlying type systems does no longer allow the semantics defined by sets, can be expressed as theories in higher-order intuitionistic logic, we use such a logic instead of first-order logic. However, as we are not yet exploiting the full power of such logics, we can interpret the logic in the category FINSET of finite sets, i.e., again in a structure defined by a database.Having done this the definition of computable queries and the model of Reflective Relational Machines carry over easily. We can show that the new model of Reflective Object Machines guarantees completeness, i.e., all computable queries can be expressed by the model.

Abstract State Machines and Computationally Complete Query Languages

Abstract state machines (ASMs) form a relatively new computation model holding the promise that they can simulate any computational system in lockstep. In particular, an instance of the ASM model has recently been introduced for computing queries to relational databases. This model, to which we refer as the BGS model, provides a powerful query language in which all computable queries can be expressed. In this paper, we show that when one is only interested in polynomial- time computations, BGS is strictly more powerful than both QL and while new , two well-known computationally complete query languages. We then show that when a language such as while new is extended with a duplicate elimination mechanism, polynomial-time simulations between the language and BGS become possible.

Endoscopy reports, databases, and computers in 2001

Endoscopy reports, databases, and computers in 2001

Iceberg-cube computation with PC clusters

In this paper, we investigate the approach of using low cost PC cluster to parallelize the computation of iceberg-cube queries. We concentrate on techniques directed towards online querying of large, high-dimensional datasets where it is assumed that the total cube has net been precomputed. The algorithmic space we explore considers trade-offs between parallelism, computation and I/0. Our main contribution is the development and a comprehensive evaluation of various novel, parallel algorithms. Specifically: (1) Algorithm RP is a straightforward parallel version of BUC [BR99]; (2) Algorithm BPP attempts to reduce I/0 by outputting results in a more efficient way; (3) Algorithm ASL, which maintains cells in a cuboid in a skiplist, is designed to put the utmost priority on load balancing; and (4) alternatively, Algorithm PT load-balances by using binary partitioning to divide the cube lattice as evenly as possible. We present a thorough performance evaluation on all these algorithms on a variety of parameters, including the dimensionality of the cube, the sparseness of the cube, the selectivity of the constraints, the number of processors, and the size of the dataset. A key finding is that it is not a one-algorithm-fit-all situation. We recommend a “recipe” which uses PT as the default algorithm, but may also deploy ASL under specific circumstances.

Disease reporting from an automated laboratory-based reporting system to a state health department via local county health departments.

The authors assessed the completeness of disease reporting from a managed care organization's automated laboratory-based reporting system to the California Department of Health Services (CDHS) via local public health departments. The authors identified all positive laboratory tests for 1997 from the computerized database of Kaiser Permanente Northern California for seven infections for which there are statutory reporting requirements: Campylobacter jejuni, Chlamydia trachomatis, Cryptosporidium parvum, hepatitis A, Neisseria meningitidis, Neisseria gonorrhoeae, and Salmonella (N = 7,331 reports). Cases were then matched by computer query to records of cases reported to CDHS. To determine why cases were not found in CDHS records, a sample of un-matched cases was searched at two county health departments. Overall, 84.5% (95% CI 83.4, 85.6) of the laboratory reports submitted with accompanying demographic information were successfully matched with cases in the CDHS disease surveillance database. Frequency of matching for specific diseases ranged from 79.4% (95% CI 75.6, 83.3) for N. gonorrhoeae to 88.4% (95% CI 85.3, 91.6) for C. jejuni. Reports were more likely to be matched when the county of residence was the same as the county of the health care facility. At the county level, reasons for failure of cases to be forwarded to CDHS included: errors due to manual data entry, failure to forward information from the county of diagnosis to the county of residence, and incorrect disease coding. Automated laboratory-based reporting is highly effective, but some data are lost with off-line transfer of information. To optimize surveillance accuracy and completeness, reporting at all levels should be done via direct electronic data transfer.

On the expressibility and the computability of untyped queries

The work of Chandra and Harel contained in Chandra and Harel (J. Comput. System Sci. 21 (1980) 156–178) can be considered as the beginning of the construction of a theoretical framework in which the computability and the complexity of queries to relational databases could be studied. In the definition of the class CQ of computable queries, the authors included untyped queries, that is, queries whose answers are relations of possibly different arities on different relational structures or databases. However, it seems that in the quite wide and important work which followed on the subject, untyped queries were not considered at all, neither in the abstract machines side ( relational machines and reflective relational machines), nor in the logic framework. We propose to re-introduce these queries in the study of query computability and complexity without the need to leave the relational model. One important application which we consider in the theoretical setting is the computation of numerical queries, that is, queries which range over the naturals. Numerical queries do not fit in the current state of the theory regarding the two formalisms referenced above, since they can result in numerical values which are not necessarily in the domain of the given structure. We define an extension of the reflective relational machine of Abiteboul et al. (Proc. 9th IEEE Symp. on Logic in Computer Science, 1994), which we call untyped reflective relational machine, and we prove that this model is complete considering the whole class CQ (i.e., both typed and untyped queries). In the logic framework, we define a new quantifier, which we call conditional quantifier, and we build with it an infinitary logic which we denote by L ω 1ω c . The difference between this logic and the well-known infinitary logic L ω 1ω is that a formula of our logic can induce relations of different arities on different structures or databases. Then we prove that L ω 1ω c strictly includes the whole sub-class of the total computable queries, including untyped queries. Finally, we define a fragment of L ω 1ω c which we prove that exactly captures the sub-class of the total computable queries, but which is undecidable. We also define an undecidable fragment of L ω 1ω which exactly captures the sub-class of the total and typed computable queries.

A study of homogeneity in relational databases

We define four different properties of relational databases which are related tothe notion of homogeneity in classical model theory. The main question for their definition is, for any given database to determine the minimum integer i>k, such that whenever two i>k-tuples satisfy the same properties which are expressible in first order logic with up to i>k variables (FOi>k), then there is an automorphism which maps each of these i>k-tuples onto each other. We study these four properties as a means to increase the computational power of subclasses of the reflective relational machines (RRMs) of bounded variable complexity. These were introduced by S. Abiteboul, C. Papadimitriou and V. Vianu and are known to be incomplete. For this sake we first give a semantic characterization of the subclasses of total RRM with variable complexity i>k (RRMi>k) for every natural number i>k. This leads to the definition of classes of queries denoted as i>Qi>Ci>Qi>k. We believe these classes to be of interest in their own right. For each i>k>0, we define the subclass i>Qi>Ci>Qi>k as the total queries in the class i>Ci>Q of computable queries which preserve realization of properties expressible in FOi>k. The nature of these classes is implicit in the work of S. Abiteboul, M. Vardi and V. Vianu. We prove i>Qi>Ci>Qi>kei>total(RRMi>k) for every i>k>0. We also prove that these classes form a i>strict hierarchy within a strict subclass of i>total(i>Ci>Q). This hierarchy is orthogonal to the usual classification of computable queries in time-space-complexity classes. We prove that the computability power of RRMi>k machines is much greater when working with classes of databases which are homogeneous, for three of the properties which we define. As to the fourth one, we prove that the computability power of RRM with sublinear variable complexity also increases when working on databases which satisfy that property. The strongest notion, pairwise i>k-homogeneity, allows RRMi>k machines to achieve i>completeness.

Queries and computation on the web

The paper introduces a model of the Web as an infinite, semistructured set of objects. We reconsider the classical notions of genericity and computability of queries in this new context and relate them to styles of computation prevalent on the Web, based on browsing and searching. We revisit several well-known declarative query languages (first-order logic, Datalog, and Datalog with negation) and consider their computational characteristics in terms of the notions introduced in this paper. In particular, we are interested in languages or fragments thereof which can be implemented by browsing, or by browsing and searching combined. Surprisingly, stratified and well-founded semantics for negation turn out to have basic shortcomings in this context, while inflationary semantics emerges as an appealing alternative.

Optimizing path query performance: graph clustering strategies

Path queries over transportation networks are operations required by many Geographic Information Systems applications. Such networks, typically modeled as graphs composed of nodes and links and represented as link relations, can be very large and hence often need to be stored on secondary storage devices. Path query computation over such large persistent networks amounts to high I/O costs due to having to repeatedly bring in links from the link relation from secondary storage into the main memory buffer for processing. This paper is the first to present a comparative experimental evaluation of alternative graph clustering solutions in order to show their effectiveness in path query processing over transportation networks. Clustering optimization is attractive because it does not incur any run-time cost, requires no auxiliary data structures, and is complimentary to many of the existing solutions on path query processing. In this paper, we develop a novel clustering technique, called spatial partition clustering (SPC), that exploits unique properties of transportation networks such as spatial coordinates and high locality. We identify other promising candidates for clustering optimizations from the literature, such as two-way partitioning and approximate topological clustering. We fine-tune them to optimize their I/O behavior for path query processing. Our experimental evaluation of the performance of these graph clustering techniques using an actual city road network as well as randomly generated graphs considers variations in parameters such as memory buffer size, length of the paths, locality, and out-degree. Our experimental results are the foundation for establishing guidelines to select the best clustering technique based on the type of networks. We find that our SPC performs the best for the highly interconnected city map; the hybrid approach for random graphs with high locality; and the two-way partitioning based on link weights for random graphs with no locality.

On the expressive power of the relational algebra with partially ordered domains

Assuming data domains are partially ordered, we define the partially ordered relational algebra (PORA) by allowing the ordering predicate ⊑ to be used in formulae of the selection operator σ. We apply Paredaens and Bancilhon's Theorem to examine the expressiveness of the PORA, and show that the PORA expresses exactly the set of all possible relations which are invariant under order-preserving automorphisms of databases. The extension is consistent with the two important extreme cases of unordered and linearly ordered domains. We also investigate the three hierarchies of: (1) computable queries, (2) query languages and (3) partially ordered domains, and show that there is a one-to-one correspondence between them.

Complete Geometric Query Languages

We extend Chandra and Harel's seminal work on computable queries for relational databases to a setting in which also spatial data may be present, using a constraint-based data model. Concretely, we introduce both coordinate-based and point-based query languages that are complete in the sense that they can express precisely all computable queries that are generic with respect to certain classes of transformations of space, corresponding to certain geometric interpretations of spatial data. The languages we introduce are obtained by augmenting basic languages with a “while” construct. We also show that the respective basic point-based languages are complete, relative to the subclass of the corresponding generic queries consisting of those that are expressible in the relational calculus with real polynomial constraints.

Querying network directories

Heirarchically structured directories have recently proliferated with the growth of the Internet, and are being used to store not only address books and contact information for people, but also personal profiles, network resource information, and network and service policies. These systems provide a means for managing scale and heterogeneity, while allowing for conceptual unity and autonomy across multiple directory servers in the network, in a way for superior to what conventional relational or object-oriented databases offer. Yet, in deployed systems today, much of the data is modeled in an ad hoc manner, and many of the more sophisticated “queries” involve navigational access. In this paper, we develop the core of a formal data model for network directories, and propose a sequence of efficiently computable query languages with increasing expressive power. The directory data model can naturally represent rich forms of heterogeneity exhibited in the real world. Answers to queries expressible in our query languages can exhibit the same kinds of heterogeneity. We present external memory algorithms for the evaluation of queries posed in our directory query languages, and prove the efficiency of each algorithm in terms of its I/O complexity. Our data model and query languages share the flexibility and utility of the recent proposals for semi-structured data models, while at the same time effectively addressing the specific needs of network directory applications, which we demonstrate by means of a representative real-life example.

Formal models of Web queries

We present a new formal model of query and computation on the Web. We focus on two important aspects that distinguish the access to Web data from the access to a standard database system: the navigational nature of the access and the lack of concurrency control. We show that these two issues have significant effects on the computability of queries. To illustrate the ideas and how they can be used in practice for designing appropriate Web query languages, we consider a particular query language, the Web calculus, an abstraction and extension of the practical Web query language WebSQL.

The Expressive Power of Stratified Logic Programs with Value Invention

The expressive power of the family wILOG (⌝) of relational query languages is investigated. The languages are rule based, with value invention and stratified negation. The semantics for value invention is based on Skolem functor terms. We study a hierarchy of languages based on the number of strata allowed in programs. We first show that, in presence of value invention, the class of stratified programs made of two strata has the expressive power of the whole family, thus expressing the computable queries. We then show that the language wILOG ≠ of programs with nonequality and without negation expresses the mono- tone computable queries, and that the language wILOG 1/2, ⌝ of semipositive programs expresses the semimonotone computable queries.

Deductive query languages for recursively typed complex objects

Deductive database query languages for recursively typed complex objects based on the set and tuple constructs are studied. A fundamental characteristic of such complex objects is that in them, sets may contain members with arbitrarily deep nesting of tuple and set constructs. Relative to mappings from flat relations to flat relations, two extensions of COL in this context (with stratified semantics and inflationary semantics, respectively) are shown to have the expressive power of computable queries. Although the deductive calculus of Bancilhon and Khoshafian has the ability to simulate Turing machines, when restricted to flat input and output its expressive power is characterized by a weak variant of the conjunctive queries.

Δ-languages for sets and LOGSPACE computable graph transformers

We discuss several versions of a set theoretic Δ-language as a reasonable prototype for “nested” database query language where database states and queries are considered, respectively, as hereditarily finite sets and set theoretic operations. In a previous work such a language exactly corresponding to PTIME-computability was introduced. It is supposed that HF-sets are naturally presented by vertices of acyclic graphs. Here we consider a number of languages for Sub PTIME computable set operations via corresponding graph transformers. Two such languages lead to a notion of NLOGSPACE and, respectively, DLOGSPACE computable queries over HF which appear the most natural, at our present knowledge, among others considered here. Unlike the “flat” relational databases the problem of finding sufficiently good corresponding approach for HF proves to be more intricate and, furthermore, gives rise to some interesting questions in finite model theory (cf. Section 13).

Queries and Algorithms Computable by Polynomial Time Existential Reflective Machines

We consider two kinds of reflective relational machines: the usual ones, which use first order queries, and existential reflective machines, which use only first order existential queries. We compare these two computation models. We build on already existing results for standard relational machines, obtained by Abiteboul, Pa-padimitriou and Vianu [2], so we prove only results for existential machines. First we show that for both standard and existential reflective machines the set of polynomial time computable Boolean queries consists precisely of all PSPACE computable queries. Then we go further and compare which classes of algorithms both kinds of machines represent. Unless PSPACE = P NP, there are PSPACE queries which cannot be computed by polynomial time existential reflective machines which use only polynomial amount of relational memory, while it is possible for standard reflective machines. We conclude that existential reflective machines, being equivalent in computational power to unrestricted machines, implement substantially worse algorithms than the latter. Concerning deciding P classes of structures, every fixpoint query can be evaluated by a polynomial time unrestricted reflective machine using constant number of variables, while existential reflective machines need [n/2] variables to implement the graph connectivity query. So again the algorithms represented by existential reflective machines are worse. Finally, for each k we get a characterization of the Δ k+1 p level in the polynomial time hierarchy in terms of reflective relational machines which use either P k 0 or S k p queries.

Oversize shelves: a storage management technique for large spatial data objects

In this paper we present a new technique to improve the performance of spatial access methods by minimizing redundancy: the oversize shelf. Oversize shelves are additional disk pages that are attached to the interior nodes of a treebased spatial access method (such as the R-tree or the cell tree). These pages are used to accommodate very large data objects in order to avoid their excessive fragmentation. Whenever inserting a new object into the tree, one now has to decide whether to store it on an oversize shelf or insert it into the corresponding subtrees. For this purpose, we developed an analytic model for the behaviour of dynamic spatial access methods under insertion and deletions. The model yields a threshold value for the size of an object, such that it is more favourable to put it on the oversize shelf if and only if its size is greater than the threshold value. Otherwise the insertion into the corresponding subtrees is preferable. Practical experiments indicate that this policy results in a faster computation of search queries, and that the cell tree with oversize shelves is one of the most efficient spatial access methods currently available.