Parallel Database Research Articles

Parallel execution of hash joins in parallel databases

We explore two important issues, processor allocation and the use of hash filters, to improve the parallel execution of hash joins. To exploit the opportunity of pipelining for hash join execution, a scheme to transform a bushy execution tree to an allocation tree is first devised. In an allocation tree, each node denotes a pipeline. Then, using the concept of synchronous execution time, processors are allocated to the nodes in the allocation tree in such a way that inner relations in a pipeline can be made available at approximately the same time. Also, the approach of hash filtering is investigated to further improve the parallel execution of hash joins. Extensive performance studies are conducted via simulation to demonstrate the importance of processor allocation and to evaluate various schemes using hash filters. It is experimentally shown that processor allocation is, in general, the dominant factor of performance, and the effect of hash filtering becomes more prominent as the number of relations in a query increases.

MACRO—DB: a decision-support tool for assessing pesticide fate and mobility in soils

A decision-support tool (MACRO—DB) for predicting pesticide fate and mobility in soils is described. MACRO—DB consists of soil, pesticide, climate and crop databases linked to parameter estimation routines and a simulation model (MACRO). The system currently allows access to three soils databases: SEISMIC ( c. 400 UK benchmark soils), MARKDATA (26 Swedish soils), and a database that the user can develop independently. Automatic estimation procedures (pedo-transfer functions) translate the soil information into model parameter values. Two pesticide databases are available. The PETE database contains information for over 600 common compounds. The user can also develop a parallel database for new compounds. Sorption and degradation constants are calculated automatically, combining soil and compound properties. A weather database contains long-term daily meteorological data and a weather generator enables synthetic daily weather data to be derived from long-term average climatic data. A separate database contains information on typical planting and harvest dates, root depths, etc. for some common agricultural crops. To account for parameter uncertainty, the user can simulate ‘worst-case’ or ‘average-case’ soil scenarios, utilizing available information on the mean and typical ranges of soil organic carbon content and pH. The user can view and analyse simulation results with flexible, in-built, graphical procedures.

Graph-based rules and transactions for parallel and active database systems

Active database management systems are becoming increasingly popular because of their relevance to several advanced and complex database applications. The need for user-defined execution orders (or control structures) for rules is well recognized by researchers of active database management systems. Priority-based approaches (e.g., numeric priorities) have been used to specify a desired control structure among rules. However, due to the fact that fixed priorities are assigned to rules, independent of different contexts in which they may be triggered, the existing approaches are not able to allow rules to be executed following different control structures when they are triggered by different events. More flexible and expressive control mechanisms are often needed for rules in advanced database applications such as CAD/CAM, CASE, CIM and flexible manufacturing systems. Since rules in database environments are executed in a transaction framework, an expressive transaction model is needed to model complex control structures among rulesuniformly. In this work, we separate the event part from the condition-action parts of a rule and associate it with a rule graph which represents a set of rules (actually a set of condition-action pairs) sharing the same control structure. Different rule graphs can be defined under different event specifications thereby enabling a set of rules to follow different control structures when triggered by different events. We also use an expressive graph-based transaction model to incorporate the control structures of rule graphs uniformly in a transaction framework. The proposed rule and transaction modeling and execution techniques have been implemented and verified on a shared-nothing multiprocessor computer nCUBE2. In this paper, we also describe the client-server architecture and different parallel transaction and rule execution techniques that have been used for the implementation. Finally, we analyze the speedup and scaleup of the implemented system.

Hierarchical query execution in a parallel object-oriented database system

We present a hierarchical query execution strategy for a parallel object-oriented database (OODB) system. The system, named PRACTIC, is based on a concurrent active class management model and is mapped to an abstract hierarchical multiprocessor architecture. The proposed strategy is studied analytically and by simulation on a transputer-based machine, verifying the theoretical results. Although the analysis suits both main-memory and disk-based database systems, it becomes significant for main-memory systems where the multiprocessor initialization and communication overheads are comparable to the actual workload. The hierarchical query execution strategy is proved much better than the usual flat strategy of parallel database systems, except some clearly identified extreme cases, where flat processing is better. Furthermore, we propose a declustering scheme for space optimization to improve processor utilization and single-class query performance, by having different classes share memory and computation power of neighboring processing elements.

Open issues in parallel query optimization

We provide an overview of query processing in parallel database systems and discuss several open issues in the optimization of queries for parallel machines.

Efficiently extendible mappings for balanced data distribution

In data storage applications, a large collection of consecutively numbered data “buckets” are often mapped to a relatively small collection of consecutively numbered storage “bins.” For example, in parallel database applications, buckets correspond to hash buckets of data and bins correspond to database nodes. In disk array applications, buckets correspond to logical tracks and bins correspond to physical disks in an array. Measures of the “goodness” of a mapping method include: One contribution of this paper is to give a new mapping method, theInterval-Round-Robin (IRR) method. The IRR method has optimal balance and relocation cost, and its time complexity and storage requirements compare favorably with known methods. Specifically, ifm is the number of times that the number of bins and/or buckets has increased, then the time complexity isO(logm) and the storage isO(m 2). Another contribution of the paper is to identify the concept of ahistory-independent mapping, meaning informally that the mapping does not “remember” the past history of expansions to the number of buckets and bins, but only the current number of buckets and bins. Thus, such mappings require very little information to be stored. Assuming that balance and relocation are optimal, we prove that history-independent mappings are possible if the number of buckets is fixed (so only the number of bins can increase), but not possible if the number of bins and buckets can both increase.

Two techniques for on-line index modification in shared nothing parallel databases

Whenever data is moved across nodes in the parallel database system, the indexes need to be modified too. Index modification overhead can be quite severe because there can be a large number of indexes on a relation. In this paper, we study two alternatives to index modification, namely OAT (One-At-a-Time page movement) and BULK (bulk page movement). OAT and BULK are two extremes on the spectrum of the granularity of data movement. OAT and BULK differ in two respects: first, OAT uses very little additional disk space (at most one extra page), whereas BULK uses a large amount of disk space. Second, BULK uses sequential prefetch I/O to optimize on the number of I/Os during index modification, while OAT does not. Using an experimental testbed, we show that BULK is an order of magnitude faster than OAT. In terms of the impact on transaction performance during reorganization, BULK and OAT perform differently: when the number of indexes to be modified is either one or two, OAT has a lesser impact on the transaction performance degradation. However, when the number of indexes is greater than two, both techniques have the same impact on transaction performance.

Multi-dimensional resource scheduling for parallel queries

Scheduling query execution plans is an important component of query optimization in parallel database systems. The problem is particularly complex in a shared-nothing execution environment, where each system node represents a collection of time-shareable resources (e.g., CPU(s), disk(s), etc.) and communicates with other nodes only by message-passing. Significant research effort has concentrated on only a subset of the various forms of intra-query parallelism so that scheduling and synchronization is simplified. In addition, most previous work has focused its attention on one-dimensional models of parallel query scheduling, effectively ignoring the potential benefits of resource sharing. In this paper, we develop an approach that is more general in both directions, capturing all forms of intra-query parallelism and exploiting sharing of multi-dimensional resource nodes among concurrent plan operators. This allows scheduling a set of independent query tasks (i.e., operator pipelines) to be seen as an instance of the multi-dimensional bin-design problem. Using a novel quantification of coarse grain parallelism, we present a list scheduling heuristic algorithm that is provably near-optimal in the class of coarse grain parallel executions (with a worst-case performance ratio that depends on the number of resources per node and the granularity parameter). We then extend this algorithm to handle the operator precedence constraints in a bushy query plan by splitting the execution of the plan into synchronized phases. Preliminary performance results confirm the effectiveness of our scheduling algorithm compared both to previous approaches and the optimal solution. Finally, we present a technique that allows us to relax the coarse granularity restriction and obtain a list scheduling method that is provably near-optimal in the space of all possible parallel schedules.

Optimizing the NASA Technical Report Server

Explains how the NASA Technical Report Server, a World Wide Web report distribution NASA technical publications service, has been modified for performance enhancement, greater protocol support and human interface optimization. Results include: parallel database queries, significantly decreasing user access times by an average factor of 2.3; access from clients behind firewalls and/or proxies which truncate excessively long Uniform Resource Locators; access to non‐Wide Area Information Server (WAIS) databases and compatibility with the Z39‐50.3 protocol; and a streamlined user interface.

Distributed and parallel database systems

article Free AccessDistributed and parallel database systems Authors: M. Tamer Özsu Department of Computing Science, University of Alberta, Edmonton, Canada Department of Computing Science, University of Alberta, Edmonton, CanadaView Profile , Patrick Valduriez INRIA, Rocquencourt, Le Chesnay Cedex, France INRIA, Rocquencourt, Le Chesnay Cedex, FranceView Profile Authors Info & Claims ACM Computing SurveysVolume 28Issue 1March 1996 pp 125–128https://doi.org/10.1145/234313.234368Published:01 March 1996Publication History 61citation4,639DownloadsMetricsTotal Citations61Total Downloads4,639Last 12 Months812Last 6 weeks78 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Extending a multi-set relational algebra to a parallel environment

Parallel database systems will very probably be the future for high-performance data-intensive applications. In the past decade, many parallel database systems have been developed, together with many languages and approaches to specify operations in these systems. A common background is still missing, however. This paper proposes an extended relational algebra for this purpose, based on the well-known standard relational algebra. The extended algebra provides both complete database manipulation language features, and data distribution and process allocation primitives to describe parallelism. It is defined in terms of multi-sets of tuples to allow handling of duplicates and to obtain a close connection to the world of high-performance data processing. Due to its algebraic nature, the language is well suited for optimization and parallelization through expression rewriting. The proposed language can be used as a database manipulation language on its own, as has been done in the PRISMA parallel database project, or as a formal basis for other languages, like SQL.

Modelling test data for performance evaluation of large parallel database machines

Parallel servers offer improved processing power for relational database systems and provide system scalability. In order to support the users of these systems, new ways of assessing the performance of such machines are required. If these assessments are to show how the machines perform under commercial workloads they need to be based upon models which have a real commercial basis. This paper shows how a realistic model of a financial application has been developed and how a set of tools has been created which allow the implementation of the model on any commercial database system. The tools allow the generation of large quantities of test data in a manner which renders it amenable to subsequent independent analysis. The test data thus generated forms the basis for the performance tuning of parallel database machines.

Practical throughput estimation for parallel databases

Methods for estimating the performance of database management systems can aid the design of database systems by identifying potential performance bottle-necks or by predicting the relative performance of different designs. Performance estimation is critical in parallel database systems with distributed memory, where an effective overall performance depends on a good choice among a wide range of ways of placing data. An approach is described for performance estimation for shared-nothing parallel database systems. It estimates system throughput for a given benchmark or set of queries, and can exercise different data placement schemes to determine the data layout that provides the best throughput value.

Parallelism and its price

We describe the use of parallel execution techniques and measure the price of parallel execution in NonStop SQL/MP, a commercial parallel database system from Tandem Computers. Non-Stop SQL uses intra-operator parallelism to parallelize joins, groupings and scans. Parallel execution consists of starting up several processes and communicating data between them. Our measurements show (a) Startup costs are negligible when processes are reused rather than created afresh (b) Communication costs are significant — they may exceed the costs of operators such as scan, grouping or join. We also show two counter-examples to the common intuition that parallel execution reduces response time at the expense of increased work — parallel execution may reduce work or may increase response time depending on communication costs.

Beyond software performance visualization

AbstractPerformance visualization tools of the past decade have yielded new insights into the behavior of sequential, parallel and distributed programs. However, they have three inherent limitations: (1) they only display what happened in one execution of a program (this is dangerous when analyzing concurrent applications, which are prone to non‐deterministic behavior); (2) a human uses one or more bandwidth‐limited senses with a visualization tool (this limits the scalability of a visualization tool); (3) the relationship of ‘interesting’ program events is often separated in time by other events; thus discerning time‐dependent behavior often hinges on finding the ‘right’ visualization—a possibly time‐consuming activity. CHITRA93 complements visualization systems, while alleviating these limitations, and analyzes a set (or ensemble) of traces by combining the visualization of a few traces with a statistical analysis of the entire ensemble (overcoming (1)). It reduces the ensemble to empirical models that capture the time‐dependent relationships of ‘interesting’ program events through application, programming language and computer architecture independent analysis techniques (addressing (2) and (3)). It also incorporates the following transforms, such as aggregation, that simplify the ensemble and reduce the state‐space size of the models generated; a user interface that allows certain transforms to be selected by editing the visualization with a mouse; homogeneity tests that allow partitioning of an ensemble; an efficient semi‐Markov model generation algorithm whose computation time is linear in the sum of the lengths of the traces comprising the ensemble; and a CHAID‐based model that can fathom non‐Markovian relationships among transitions in the traces. The use of CHITRA93 is demonstrated by partitioning ten parallel database traces with nearly 8,000 states into two homogeneous subsets, each modeled by an irreducible, periodic and hierarchical stochastic process with as few as four states.

Uniform partitioning of relations using histogram equalization framework: An efficient parallel hash-based join

Many parallel join algorithms have been proposed for parallel relational database systems. Among them, the parallel hash-based join algorithm has been found to be superior to other ones for the uniform distribution of data. This paper proposes an efficient algorithm, called skew resolution join algorithm (SRJA), for parallel join algorithms with skewed data. We propose a methodology for partitioning relations evenly across all processors in a parallel database system. We performed simulation experiments under the Zipf-like distribution of hashed values for join attributes.

PRACTIC: A concurrent object data model for a parallel object-oriented database system

A concurrent object data model for a parallel object-oriented database system, named PRACTIC, and its abstract machine are presented. PRACTIC means PaRallel ACTIve Classes and is based on the vertical partitioning and concurrent management of the database schema classes and metaclasses, which are collectively called active objects. Active objects are permanent processes in memory that encapsulate their definitions, methods, and management procedures. Semiactive and passive objects exist to realize abstract classes and instances (the actual data), respectively. The object model gives rise to a query/method execution model that provides parallelism on all levels of the instantiation hierarchy. The abstract PRACTIC machine directly maps the model to a MIMD machine. The performance of one of the proposed parallel query/method execution schemes is measured by simulation on the abstract machine.

Parallel evaluation of multi-join queries

A number of execution strategies for parallel evaluation of multi-join queries have been proposed in the literature; their performance was evaluated by simulation. In this paper we give a comparative performance evaluation of four execution strategies by implementing all of them on the same parallel database system, PRISMA/DB. Experiments have been done up to 80 processors. The basic strategy is to first determine an execution schedule with minimum total cost and then parallelize this schedule with one of the four execution strategies. These strategies, coming from the literature, are named: Sequential Parallel, Synchronous Execution, Segmented Right-Deep, and Full Parallel. Based on the experiments clear guidelines are given when to use which strategy.

Informix online XPS

Parallel database systems have delivered on the promise of providing excellent, linear performance speedups for both decision support and transaction processing workloads across a variety of parallel system platforms. The challenge is now to extend those results across the entire open systems spectrum of loosely coupled environments while realizing the inherent high availability of such systems. The Informix eXtended Parallel Server (XPS) will accomplish that goal.XPS is the third and latest in the series of Online database servers that are based upon the Informix Dynamic Scalable Architecture (DSA). Earlier Online DSA servers have proven the effectiveness of the SMP-based high performance parallel data query (PDQ) technology that is the foundation of XPS: multi-threaded process groups, table partitioning, pipelined hash-partitioned operators, light access methods, and parallel resource management. XPS extends that PDQ technology across the continuum from small clusters of large SMP systems to massively parallel clusters of SMP or uniprocessor systems.

Paradise

The goal of the Paradise project is to apply object-oriented and parallel database technology to the task of implementing a parallel GIS system capable of managing extremely large (multi-terabyte) data sets such as those that will be produced by the upcoming NASA EOSDIS project [Car92]. The project is focusing its resources on algorithms, processing, and storage techniques, and not on making new contributions to the data modeling, query language, or user interface domains.At the outset, we organized the Paradise project as two phases. The goal of first phase was to produce a client-server version of Paradise. The second phase of the project is to add support for tertiary storage and extend the software to run on clusters of workstations and "shared nothing" multiprocessors [Sto86]. Phase one of the project is now complete and has produced a usable, client-server version of the system whose performance and functionality is comparable to other integrated GIS systems [DKL + 94].