Data Parallel Programming Model Research Articles

Sequence Similarity Parallelization over Heterogeneous Computer Clusters Using Data Parallel Programming Model

Sequence similarity, as a special case of data intensive applications, is one of the neediest applications for parallelization. Clustered commodity computers as a cost-effective platform for distributed and parallel processing, can be leveraged to parallelize sequence similarity. However, manually designing and developing parallel programs on commodity computers is a time-consuming, complex and error-prone process. In this paper, we present a sequence similarity parallelization technique using the Apache Storm as a stream processing framework with a data parallel programming model. Storm automatically parallelizes computations via a special user-defined topology that is represented as a directed acyclic graph. The proposed technique collects streams of data from a disk and sends them sequence by sequence to clustered computers for parallel processing. We also present a dispatching policy for balancing the cluster workload and managing the cluster heterogeneity to achieve more than 99 percent parallelism. An alignment-free method, known as n-gram modeling, is used to calculate similarities between the sequences. To show the cost-performance superiority of our method on clustered commodity computers over serial processing in powerful computers, we simply use UniProtKB/SwissProt dataset for evaluation of the performance of sequence similarity as an interesting large-scale Bioinformatics application.

DySel

The rising pressure for simultaneously improving performance and reducing power is driving more diversity into all aspects of computing devices. An algorithm that is well-matched to the target hardware can run multiple times faster and more energy efficiently than one that is not. The problem is complicated by the fact that a program's input also affects the appropriate choice of algorithm. As a result, software developers have been faced with the challenge of determining the appropriate algorithm for each potential combination of target device and data. This paper presents DySel, a novel runtime system for automating such determination for kernel-based data parallel programming models such as OpenCL, CUDA, OpenACC, and C++AMP. These programming models cover many applications that demand high performance in mobile, cloud and high-performance computing. DySel systematically deploys candidate kernels on a small portion of the actual data to determine which achieves the best performance for the hardware-data combination. The test-deployment, referred to as micro-profiling, contributes to the final execution result and incurs less than 8% of overhead in the worst observed case when compared to an oracle. We show four major use cases where DySel provides significantly more consistent performance without tedious effort from the developer.

DySel

The rising pressure for simultaneously improving performance and reducing power is driving more diversity into all aspects of computing devices. An algorithm that is well-matched to the target hardware can run multiple times faster and more energy efficiently than one that is not. The problem is complicated by the fact that a program's input also affects the appropriate choice of algorithm. As a result, software developers have been faced with the challenge of determining the appropriate algorithm for each potential combination of target device and data. This paper presents DySel, a novel runtime system for automating such determination for kernel-based data parallel programming models such as OpenCL, CUDA, OpenACC, and C++AMP. These programming models cover many applications that demand high performance in mobile, cloud and high-performance computing. DySel systematically deploys candidate kernels on a small portion of the actual data to determine which achieves the best performance for the hardware-data combination. The test-deployment, referred to as micro-profiling, contributes to the final execution result and incurs less than 8% of overhead in the worst observed case when compared to an oracle. We show four major use cases where DySel provides significantly more consistent performance without tedious effort from the developer.

A Survey on Programmable LDPC Decoders

Low-density parity-check (LDPC) block codes are popular forward error correction schemes due to their capacity-approaching characteristics. However, the realization of LDPC decoders that meet both low latency and high throughput is not a trivial challenge. Usually, this has been solved with the ASIC and FPGA technology that enables meeting the decoder design constraints. But the rise of parallel architectures, such as graphics processing units, and the scaling of CPU streaming extensions has shown that multicore and many-core technology can provide a flexible alternative to the development of dedicated LDPC decoders for the compute-intensive prototyping phase of the design of new codes. Under this light, this paper surveys the most relevant publications made in the past decade to programmable LDPC decoders. It looks at the advantages and disadvantages of parallel architectures and data-parallel programming models, and assesses how the design space exploration is pursued regarding key characteristics of the underlying code and decoding algorithm features. This paper concludes with a set of open problems in the field of communication systems on parallel programmable and reconfigurable architectures.

Discovery of semantic associations in an RDF graph using bi-directional BFS on massively parallel hardware

Resource description framework (RDF) data model provides a framework to capture the meaning of an entity by specifying how it relates to other entities. Large RDF graph involving millions of entities are common in many semantic graph applications and are challenging to process. For example, finding complex relationships called semantic associations between two entities in an RDF graph is a tedious process. Graphics processing units (GPUs) provides high computation power at low price. Today, the GPUs expose a general data-parallel programming model in the form of CUDA. In this paper, we present the implementation of bi-directional breadth-first-search algorithm to discover the semantic association in RDF graph using CUDA programming model. In the experimental evaluation, we prove that our proposed algorithm is faster than the existing algorithms.

Discovering Big Data Modelling for Educational World

With the advancement in internet technology all over the world, the demand for online education is growing. Many educational institutions are offering various types of online courses and e-content. The analytical models from data mining and computer science heuristics help in analysis and visualization of data, predicting student performance, generating recommendations for students as well as teachers, providing feedback to students, identifying related courses, e-content and books, detecting undesirable student behaviours, developing course contents and in planning various other educational activities. Today many educational institutions are using data analytics for improving the services they provide. The data access patterns about students, logged and collected from online educational learning systems could be explored to find informative relationships in the educational world. But a major concern is that the data are exploding, as numbers of students and courses are increasing day by day all over the world. The usage of Big Data platforms and parallel programming models like MapReduce may accelerate the analysis of exploding educational data and computational pattern finding capability. The paper focuses on trial of educational modelling based on Big Data techniques.

ForOpenCL: transformations exploiting array syntax in Fortran for accelerator programming

Emerging GPU architectures for high performance computing are well suited to a data-parallel programming model. This paper presents preliminary work examining a programming methodology that provides Fortran programmers with access to these emerging systems. We use array constructs in Fortran to show how this infrequently exploited, standardised language feature is easily transformed to lower-level accelerator code. The transformations in ForOpenCL are based on a simple mapping from Fortran to OpenCL. We demonstrate, using a stencil code solving the shallow-water fluid equations, that the performance of the ForOpenCL compiler-generated transformations is comparable with that of hand-optimised OpenCL code.

A Parallel, Reconstructed Discontinuous Galerkin Method for the Compressible Flows on Arbitrary Grids

AbstractA reconstruction-based discontinuous Galerkin method is presented for the solution of the compressible Navier-Stokes equations on arbitrary grids. In this method, an in-cell reconstruction is used to obtain a higher-order polynomial representation of the underlying discontinuous Galerkin polynomial solution and an inter-cell reconstruction is used to obtain a continuous polynomial solution on the union of two neighboring, interface-sharing cells. The in-cell reconstruction is designed to enhance the accuracy of the discontinuous Galerkin method by increasing the order of the underlying polynomial solution. The inter-cell reconstruction is devised to remove an interface discontinuity of the solution and its derivatives and thus to provide a simple, accurate, consistent, and robust approximation to the viscous and heat fluxes in the Navier-Stokes equations. A parallel strategy is also devised for the resulting reconstruction discontinuous Galerkin method, which is based on domain partitioning and Single Program Multiple Data (SPMD) parallel programming model. The RDG method is used to compute a variety of compressible flow problems on arbitrary meshes to demonstrate its accuracy, efficiency, robustness, and versatility. The numerical results demonstrate that this RDG method is third-order accurate at a cost slightly higher than its underlying second-order DG method, at the same time providing a better performance than the third order DG method, in terms of both computing costs and storage requirements.

Transformer: A New Paradigm for Building Data-Parallel Programming Models

Cloud computing drives the design and development of diverse programming models for massive data processing. the transformer programming framework aims to facilitate the building of diverse data-parallel programming models. transformer has two layers: a common runtime system and a model-specific system. using transformer, the authors show how to implement three programming models: dryad-like data flow, MapReduce, and All-Pairs.

Multi-Level Graph Layout on the GPU

This paper presents a new algorithm for force directed graph layout on the GPU. The algorithm, whose goal is to compute layouts accurately and quickly, has two contributions. The first contribution is proposing a general multi-level scheme, which is based on spectral partitioning. The second contribution is computing the layout on the GPU. Since the GPU requires a data parallel programming model, the challenge is devising a mapping of a naturally unstructured graph into a well-partitioned structured one. This is done by computing a balanced partitioning of a general graph. This algorithm provides a general multi-level scheme, which has the potential to be used not only for computation on the GPU, but also on emerging multi-core architectures. The algorithm manages to compute high quality layouts of large graphs in a fraction of the time required by existing algorithms of similar quality. An application for visualization of the topologies of ISP (Internet Service Provider) networks is presented.

A defect tolerant self-organizing nanoscale SIMD architecture

The continual decrease in transistor size (through either scaled CMOS or emerging nano-technologies) promises to usher in an era of tera to peta-scale integration. However, this decrease in size is also likely to increase defect densities, contributing to the exponentially increasing cost of top-down lithography. Bottom-up manufacturing techniques, like self assembly, may provide a viable lower-cost alternative to top-down lithography, but may also be prone to higher defects. Therefore, regardless of fabrication methodology, defect tolerant architectures are necessary to exploit the full potential of future increased device densities.This paper explores a defect tolerant SIMD architecture. A key feature of our design is the ability of a large number of limited capability nodes with high defect rates (up to 30%) to self-organize into a set of SIMD processing elements. Despite node simplicity and high defect rates, we show that by supporting the familiar data parallel programming model the architecture can execute a variety of programs. The architecture efficiently exploits a large number of nodes and higher device densities to keep device switching speeds and power density low. On a medium sized system (~1cm 2 area), the performance of the proposed architecture on our data parallel programs matches or exceeds the performance of an aggressively scaled out-of-order processor (128-wide, 8k reorder buffer, perfect memory system). For larger systems (>1cm 2 ), the proposed architecture can match the performance of a chip multiprocessor with 16 aggressively scaled out-of-order cores.

A defect tolerant self-organizing nanoscale SIMD architecture

The continual decrease in transistor size (through either scaled CMOS or emerging nano-technologies) promises to usher in an era of tera to peta-scale integration. However, this decrease in size is also likely to increase defect densities, contributing to the exponentially increasing cost of top-down lithography. Bottom-up manufacturing techniques, like self assembly, may provide a viable lower-cost alternative to top-down lithography, but may also be prone to higher defects. Therefore, regardless of fabrication methodology, defect tolerant architectures are necessary to exploit the full potential of future increased device densities.This paper explores a defect tolerant SIMD architecture. A key feature of our design is the ability of a large number of limited capability nodes with high defect rates (up to 30%) to self-organize into a set of SIMD processing elements. Despite node simplicity and high defect rates, we show that by supporting the familiar data parallel programming model the architecture can execute a variety of programs. The architecture efficiently exploits a large number of nodes and higher device densities to keep device switching speeds and power density low. On a medium sized system (~1cm 2 area), the performance of the proposed architecture on our data parallel programs matches or exceeds the performance of an aggressively scaled out-of-order processor (128-wide, 8k reorder buffer, perfect memory system). For larger systems (>1cm 2 ), the proposed architecture can match the performance of a chip multiprocessor with 16 aggressively scaled out-of-order cores.

A defect tolerant self-organizing nanoscale SIMD architecture

The continual decrease in transistor size (through either scaled CMOS or emerging nano-technologies) promises to usher in an era of tera to peta-scale integration. However, this decrease in size is also likely to increase defect densities, contributing to the exponentially increasing cost of top-down lithography. Bottom-up manufacturing techniques, like self assembly, may provide a viable lower-cost alternative to top-down lithography, but may also be prone to higher defects. Therefore, regardless of fabrication methodology, defect tolerant architectures are necessary to exploit the full potential of future increased device densities.This paper explores a defect tolerant SIMD architecture. A key feature of our design is the ability of a large number of limited capability nodes with high defect rates (up to 30%) to self-organize into a set of SIMD processing elements. Despite node simplicity and high defect rates, we show that by supporting the familiar data parallel programming model the architecture can execute a variety of programs. The architecture efficiently exploits a large number of nodes and higher device densities to keep device switching speeds and power density low. On a medium sized system (~1cm 2 area), the performance of the proposed architecture on our data parallel programs matches or exceeds the performance of an aggressively scaled out-of-order processor (128-wide, 8k reorder buffer, perfect memory system). For larger systems (>1cm 2 ), the proposed architecture can match the performance of a chip multiprocessor with 16 aggressively scaled out-of-order cores.

Extending the data parallel paradigm with data-dependent operators

This paper presents a new scheme for software support of image analysis. It extends the data parallel programming model with the notion of a special data-dependent operand. Thanks to its ability of storing the location of pixel aggregates (connected components), manipulations on multiple image areas can be expressed in terms of whole image transformations. This general model, called the associative net extension, gives programmers flexible ways of handling regions and contours, as shown by a set of examples and it has been implemented on single processor machines to verify its performance and to test its effectiveness in real cases. Implementations on special purpose hardware or multi-processor machines are being tested and look promising, too.

Mapping strategies in data parallel programming models; the projection methods

We Analyze data parallel programming of some general purpose methods in linear algebra. Specifically, the projection methods to solve very large linear system and/or eigenproblem. The expensive parts of these methods are their projection phases. This portion of these algorithms has generally, a very simple structure for such a programming model. It is composed essentially of matrix-vector multiplications and inner-products. Then, we simply need to find a good data distribution in order to obtain a well adapted communication pattern and not loose too much storage space. We begin with a survey of data parallel behavior of some projection methods such as Arnoldi, GMRES, Lanczos, PRR,... After analyzing some methods of data mapping onto virtual processors we point out that for a fixed number of physical processors, the performances are a function of the mapping method. We will see also that the maximum size of the problems which can be solved on the architectures supporting the data parallel programming model is a function of the data mapping method. In conclusion we present the performances obtained on a Connection Machine 5 (CM5).

An integrated performance analysis tool for SPMD data-parallel programs

A new generation of data parallel languages have been proposed whereby a user specifies how data structures are to be distributed amongst the processor nodes of a distributed-memory machine. Based on this information, the compiler then generates code for the parallel application. Although this approach significantly simplifies the development of the initial version of a parallel application, selection of good data distributions leading to efficient computations is often quite difficult. Therefore, performance debuggers are needed to yield insights into the data distribution effects. On the other hand, most of the existing approaches to performance debugging are very general and thus do not provide the user feedback in terms of the high level programming model or the source code of the parallel application. In this paper, we describe a novel approach to performance debugging of data parallel programs. The design and implementation of a visual performance debugger is described that is specifically targeted to meet the performance debugging requirements of a data-parallel programming model based on user-specified data distributions. The performance debugger is part of an integrated programming environment, called EPPP, which also supports a data parallel compiler and a parallel architecture simulator. Thus development and performance debugging of an application may be done either on the real hardware or by using the simulator. The code for EPPP can be obtained free of cost (contact web address, http://www.crim.ca/apar).

Sparse Matrix Operations In Vector And ParallelProcessors

Vector computers have been extensively used for years in matrix algebra to treat with large dense matrix problems. However, if matrices are sparse and we use special storage schemes for them, vectorization provides a poor performance due to the great amount of indirections in the code. An alternative option is the utilization of a multiprocessor (or a cluster of workstations); in this case, a data parallel programming model also fails because of the reason pointed out for vector computers. Therefore, the best choice is to parallelize the corresponding algorithms using message passing routines. In order to discuss these features, we will focus on solving sparse linear least squares problems, which appear in several scientific areas such as structural analysis, geodetic survey, molecular structure and many others. Experimental results are obtained for vector and parallel computer architectures.

Adaptive Smooth Particle Hydrodynamics and Particle-Particle Coupled Codes: Energy and Entropy Conservation

We present and test a general purpose code, called PPASPR, for evolving self-gravitating fluids in astrophysics, both with and without a collisionless component. In PPASPH, hydrodynamical properties are computed by using the SPH (smoothed particle hydrodynamics) method while, unlike most previous implementations of SPH, gravitational forces are computed by a PP (particle-particle) approach. Other important features of this code are as follows: (1) PPASPR takes into account the contributions of all particles to the gravitational and hydrodynamical forces on any other particle. This results in a better energy conservation. (2) Smoothing lengths are updated by an iterative procedure that ensures an exactly constant number of neighbors around each gas particle. (3) Cooling processes have been implemented in an integrated form that includes a special treatment to avoid a nonphysical catastrophic cooling phenomenon. Such a procedure ensures that cooling does not limit the time step. (4) Hydrodynamics equations optionally include the correction terms (hereafter ∇h terms) appearing when h(t, r) is not constant. <P />Our code has been implemented by using the data parallel programming model on the Connection Machine (CM), which allows for an efficient unification of the SPH and PP methods with costs per time step growing as ∼N. <P />PPASPR has been applied to study the importance of adaptive smoothing correction terms on the entropy conservation. We confirm Hernquist's interpretation of the entropy violation observed in previous SPR simulations as a result of having neglected these terms. An improvement on the entropy conservation is not found by merely considering larger numbers of particles or different N<SUB>s</SUB> choices. The correct continuum description is only obtained if the ∇h correction terms are included. Otherwise, the entropy conservation is always rather poor as compared to that found for the total energy.

PDDP, A Data Parallel Programming Model

PDDP, the parallel data distribution preprocessor, is a data parallel programming model for distributed memory parallel computers. PDDP implements high-performance Fortran-compatible data distribution directives and parallelism expressed by the use of Fortran 90 array syntax, the FORALL statement, and the WHERE construct. Distributed data objects belong to a global name space; other data objects are treated as local and replicated on each processor. PDDP allows the user to program in a shared memory style and generates codes that are portable to a variety of parallel machines. For interprocessor communication, PDDP uses the fastest communication primitives on each platform.

Optimized communication patterns on workstation clusters

The limited communication bandwidth and high startup latencies of clustered workstations restrict their use to problems with sparse communication patterns or good concurrency between calculation and communication. First we describe our modifications to the popular PVM [3] message passing library, and report on performance improvements using the PVM package on an FDDI-ring. Applications developed with a parallel communications architecture in mind perform poorly when ported to a message passing library running on workstations with sequential communication. In the second part, we present a dynamic loop scheduling algorithm for the data parallel programming model which optimizes the network usage on such clusters. As a proof of concept we have implemented a basic matrix multiplication and find a significant increase in parallel efficiency.