Parallel Database Systems Research Articles

Multicore Processors and Database Systems

Database management systems are necessary for transaction processing and query processing. Today, parallel database systems can be run on multicore platforms. Presented within is an overview of how multicore machines have impacted the design and implementation of database management systems.

Novel Distributed Query Optimization Model and Hybrid Query Optimization Algorithm

Query optimization is the most critical phase in query processing. Query optimization in distributed databases explicitly needed in many aspects of the optimization process, this is not only increases the cost of optimization, but also changes the trade-offs involved in the optimization process significantly .This paper describes the synthetically evolution of query optimization methods from uniprocessor relational database systems to parallel database systems. We point out a set of parameters to characterize and compare query optimization methods, mainly: (i) type of algorithm (static or dynamic), (ii) working environments (re-optimization or re-scheduling) and (iii) level of modification. The major contributions of this paper are: (I) Understanding the mechanisms of query optimization methods with respect to the considered environments and their constraints (e.g. parallelism, distribution, heterogeneity, large scale, dynamicity of nodes). (ii) Study the problem of query optimization particular in term of heterogeneously environment and pointing out their main characteristics, which allow comparing them and help to Implement new query optimization algorithm and model. These contributions is led to performance enhancement of query optimization in distributed database system through classify by different QEPs and minimize the response time.

Adaptive and big data scale parallel execution in oracle

This paper showcases some of the newly introduced parallel execution methods in Oracle RDBMS. These methods provide highly scalable and adaptive evaluation for the most commonly used SQL operations - joins, group-by, rollup/cube, grouping sets, and window functions. The novelty of these techniques is their use of multi-stage parallelization models, accommodation of optimizer mistakes, and the runtime parallelization and data distribution decisions. These parallel plans adapt based on the statistics gathered on the real data at query execution time. We realized enormous performance gains from these adaptive parallelization techniques. The paper also discusses our approach to parallelize queries with operations that are inherently serial. We believe all these techniques will make their way into big data analytics and other massively parallel database systems.

Об оценке коммуникационных затрат при обработке фрагментированного отношения для равномерного распределения

Query processing in the shared-nothing parallel database systems demands a data exchange between processor nodes. In the paper, we present a theorem, which gives estimation for the amount of tuples which have to be transferred during processing partitioned relation. We consider the case when the transfer function is functionally dependent on an attribute, which is uniformly distributed relative to the partitioning attribute.

Simulation of hierarchical multiprocessor database systems

The paper is dedicated to issues concerning simulation and analysis of hierarchical multiprocessor systems oriented to database applications. Requirements for a parallel database system model are given. A survey and comparative analysis of known parallel database system models are presented. A new multiprocessor database system model is introduced. This model allows us to simulate and evaluate arbitrary hierarchical multiprocessor configurations in the context of the OLTP class database applications. Examples of using the database multiprocessor model for simulation study of multiprocessor database systems are presented.

Обработка запросов на кластерных вычислительных системах с многоядерными ускорителями

The paper is devoted to the problem of modeling query execution process in multiprocessors of parallel database systems. Original approaches to the query execution process on GPU, MIC and multicore CPU are presented. Based on this approach, a simulator of parallel DBMS is developed. Results of computational experiments are presented, and analysis of efficiency of the proposed approaches is performed.

ASTERIX

At UC Irvine, we are building a next generation parallel database system, called ASTERIX, as our approach to addressing today's "Big Data" management challenges. ASTERIX aims to combine time-tested principles from parallel database systems with those of the Web-scale computing community, such as fault tolerance for long running jobs. In this demo, we present a whirlwind tour of ASTERIX, highlighting a few of its key features. We will demonstrate examples of our data definition language to model semi-structured data, and examples of interesting queries using our declarative query language. In particular, we will show the capabilities of ASTERIX for answering geo-spatial queries and fuzzy queries, as well as ASTERIX' data feed construct for continuously ingesting data.

A Compound Parallel Btree for High Scalability and Availability on Chained Declustering Parallel Systems

Scalability and availability are the key features of parallel database systems. To realize scalability, many dynamic load-balancing methods with data placement and parallel index structures on shared-nothing parallel infrastructure have been proposed. Data migration with range-partitioned placement using a parallel Btree is one solution. The combination of range partitioning and chained declustered replicas provides high availability (HA) while preserving scalability. However, independent treatment of the primary and backup data in each node requires long failover times. We propose a novel method for the compound treatment of chained declustered replicas using a parallel Btree, termed the Fat-Btree. In the proposed method, a single Fat-Btree provides access paths to both the primary and backup data of all processor elements (PEs), which greatly reduces failover time. Moreover, these access paths overlap between two neighboring PEs, which enables dynamic load balancing without physical data migration by dynamically redirecting the access paths. In addition, this compound treatment improves memory space utilization to enable index processing with good scalability. Experiments using PostgreSQL on a 160-node PC cluster demonstrate the effectiveness of the high scalability and availability of our proposed method.

The performance of MapReduce

MapReduce has been widely used for large-scale data analysis in the Cloud. The system is well recognized for its elastic scalability and fine-grained fault tolerance although its performance has been noted to be suboptimal in the database context. According to a recent study [19], Hadoop, an open source implementation of MapReduce, is slower than two state-of-the-art parallel database systems in performing a variety of analytical tasks by a factor of 3.1 to 6.5. MapReduce can achieve better performance with the allocation of more compute nodes from the cloud to speed up computation; however, this approach of "renting more nodes" is not cost effective in a pay-as-you-go environment. Users desire an economical elastically scalable data processing system, and therefore, are interested in whether MapReduce can offer both elastic scalability and efficiency. In this paper, we conduct a performance study of MapReduce (Hadoop) on a 100-node cluster of Amazon EC2 with various levels of parallelism. We identify five design factors that affect the performance of Hadoop, and investigate alternative but known methods for each factor. We show that by carefully tuning these factors, the overall performance of Hadoop can be improved by a factor of 2.5 to 3.5 for the same benchmark used in [19], and is thus more comparable to that of parallel database systems. Our results show that it is therefore possible to build a cloud data processing system that is both elastically scalable and efficient.

Massively parallel data analysis with PACTs on Nephele

Large-scale data analysis applications require processing and analyzing of Terabytes or even Petabytes of data, particularly in the areas of web analysis or scientific data management. This trend has been discussed as "web-scale data management" in a panel at VLDB 2009. Formerly, parallel data processing was the domain of parallel database systems. Today, novel requirements like scaling out to thousands of machines, improved fault-tolerance, and schema free processing have made a case for new approaches.

MAD skills

As massive data acquisition and storage becomes increasingly affordable, a wide variety of enterprises are employing statisticians to engage in sophisticated data analysis. In this paper we highlight the emerging practice of Magnetic, Agile, Deep (MAD) data analysis as a radical departure from traditional Enterprise Data Warehouses and Business Intelligence. We present our design philosophy, techniques and experience providing MAD analytics for one of the world's largest advertising networks at Fox Audience Network, using the Greenplum parallel database system. We describe database design methodologies that support the agile working style of analysts in these settings. We present dataparallel algorithms for sophisticated statistical techniques, with a focus on density methods. Finally, we reflect on database system features that enable agile design and flexible algorithm development using both SQL and MapReduce interfaces over a variety of storage mechanisms.

Architectural Implications for Spatial Object Association Algorithms.

Spatial object association, also referred to as crossmatch of spatial datasets, is the problem of identifying and comparing objects in two or more datasets based on their positions in a common spatial coordinate system. In this work, we evaluate two crossmatch algorithms that are used for astronomical sky surveys, on the following database system architecture configurations: (1) Netezza Performance Server®, a parallel database system with active disk style processing capabilities, (2) MySQL Cluster, a high-throughput network database system, and (3) a hybrid configuration consisting of a collection of independent database system instances with data replication support. Our evaluation provides insights about how architectural characteristics of these systems affect the performance of the spatial crossmatch algorithms. We conducted our study using real use-case scenarios borrowed from a large-scale astronomy application known as the Large Synoptic Survey Telescope (LSST).

Technologies of parallel database systems for hierarchical multiprocessor environments

For the multiprocessor systems of the hierarchical-architecture relational databases, a new approach to data layout and load balancing was proposed. Described was a database multiprocessor model enabling simulation and examination of arbitrary multiprocessor hierarchical configurations in the context of the on-line transaction processing applications. An important subclass of the symmetrical multiprocessor hierarchies was considered, and a new data layout strategy based on the method of partial mirroring was proposed for them. The disk space used to replicate the data was evaluated analytically. For the symmetrical hierarchies having certain regularity, theorems estimating the laboriousness of replica formation were proved. An efficient method of load balancing on the basis of the partial mirroring technique was proposed. The methods described are oriented to the clusters and Grid-systems.

The reliability design and availability analysis of a distributed storage system

Current distributed parallel file systems and database systems can not satisfy the demands of data-intensive applications, such as storage capacity, access performance, reliability, scalability, and so on. Cluster-based storage systems have some shortcomings, too. To solve this kind of problems, a novel PC storage cluster solution is proposed, a distributed storage system based on 3-tiered agent architecture is designed, the system reliability model based on the master-slave backup mode is built, and the system availability is analyzed with the Markov model. According to the system availability formula and the values of the system parameters, the novel system can provide higher reliability and availability to satisfy users' requirements.

A Parallel Implementation Scheme of Relational Tables Based on Multidimensional Extendible Array

In this article, an efficient parallel implementation scheme of relational tables is proposed and evaluated. The scheme implements a relational table by employing an extendible multidimensional array. Data allocation is a key performance factor for parallel database systems. This holds especially for data warehousing environments in which huge amounts of data have to be dealt with. In our scheme, an efficient data allocation technique is used, based on the notion of extendible array. The dynamic load balancing is conducted when load on each processor is not uniformly distributed in order to maximize processor utilization.

A Tool for Supporting the Teaching of Parallel Database Systems

Parallel database systems are complex entities. When teaching about these systems as part of a course in a limited time, one has difficulty providing useful practical experience that gives a deep insight into system behavior and operation. This paper describes a tool for performance prediction that has been developed to aid the visualization of parallel database systems and that is currently being used to support teaching. The tool enables students to experiment with different hardware and software configurations and to view the effects of changes on the performance of the system. The tool provides insight into how data can be placed among the nodes of a parallel machine according to predefined strategies, as well as manually, and provides feedback on the effect of these on throughput and response time. This tool is able to provide a good appreciation of the concepts in a relatively short period of time.

Survey of Architectures of Parallel Database Systems

The paper is devoted to the classification, design, and analysis of architectures of parallel database systems. A formalization of the notion “parallel database system” is suggested, which relies on a concept of a virtual machine. Based on this formalization, a new approach to the classification of architectures of parallel database systems is suggested. Requirements to parallel database systems are formulated, which serve as criteria for comparing various architectures. Various classes of architectures of parallel database systems are considered and compared.

Performance analysis of “Groupby-After-Join” query processing in parallel database systems

Queries containing aggregate functions often combine multiple tables through join operations. This query is subsequently called “ Groupby-Join”. There is a special category of this query whereby the group-by operation can only be performed after the join operation. This is known as “ Groupby-After-Join” queries––the focus of this paper. In parallel processing of such queries, it must be decided which attribute is used as a partitioning attribute, particularly join attribute or group-by attribute. Based on the partitioning attribute, two parallel processing methods, namely join partition method (JPM) and aggregate partition method (APM) are discussed. The behaviours of these parallelization methods are described in terms of cost models. Experiments are performed based on simulations. The simulation results show that the aggregate partition method performs better than the join partition method.

Analytical response time estimation in parallel relational database systems

Techniques for performance estimation in parallel database systems are well established for parameters such as throughput, bottlenecks and resource utilisation. However, response time estimation is a complex activity which is difficult to predict and has attracted research for a number of years. Simulation is one option for predicting response time but this is a costly process. Analytical modelling is a less expensive option but requires approximations and assumptions about the queueing networks built up in real parallel database machines which are often questionable and few of the papers on analytical approaches are backed by results from validation against real machines. This paper describes a new analytical approach for response time estimation that is based on a detailed study of different approaches and assumptions. The approach has been validated against two commercial parallel DBMSs running on actual parallel machines and is shown to produce acceptable accuracy.

Global parallel index for multi-processors database systems

In this paper, we propose a global B + indexing tree structure for parallel database systems. A global B + indexing tree is a tree structure used for indexing where the complete index tree is partially partitioned into multi-processors but the overall structure of global index is still maintained. We also present algorithms for maintenance of global B + indexing trees (e.g. insertion and deletion of nodes), and describe operation algorithms (e.g. search and join) on tables that are indexed using global B + indexing trees.