Task Of Parallel Programming Research Articles

Performance and programmability of GrPPI for parallel stream processing on multi-cores

GrPPI library aims to simplify the burdening task of parallel programming. It provides a unified, abstract, and generic layer while promising minimal overhead on performance. Although it supports stream parallelism, GrPPI lacks an evaluation regarding representative performance metrics for this domain, such as throughput and latency. This work evaluates GrPPI focused on parallel stream processing. We compare the throughput and latency performance, memory usage, and programmability of GrPPI against handwritten parallel code. For this, we use the benchmarking framework SPBench to build custom GrPPI benchmarks and benchmarks with handwritten parallel code using the same backends supported by GrPPI. The basis of the benchmarks is real applications, such as Lane Detection, Bzip2, Face Recognizer, and Ferret. Experiments show that while performance is often competitive with handwritten parallel code, the infeasibility of fine-tuning GrPPI is a crucial drawback for emerging applications. Despite this, programmability experiments estimate that GrPPI can potentially reduce the development time of parallel applications by about three times.

Improving the Efficiency of Calculating B-splines in parallel Programming Tasks

The use of parallel computing tools can significantly reduce the execution time of calculations in many engineer­ing tasks. One of the main difficulties in the development of multithreaded programs remains the organization of simultaneous access from different threads to shared data. The most common solution to this problem is to use locking facilities when accessing shared data. There are a number of tasks where data sharing is not needed, but you need to synchronize access to a limited resource, such as a temporary buffer. In such tasks, there is no data exchange between different threads, but there is an object that at a given time can be used by the code of only one thread. One such task is calculating the value of a B-spline. The software implementation of the functions for calculating B-splines, performed according to classical algorithms, requires the use of blocking objects when accessing the common array of intermediate data from different threads. This reduces the degree of parallelism and reduces the efficiency of computational programs using B-splines running on multiprocessor computing systems. The article discusses a way to improve the efficiency of calculating B-splines in parallel programming tasks by eliminating locks when accessing general modified data. A soft­ware implementation is presented in the form of a C++ class template, which provides placement of a temporary array used for calculating a B-spline into a local buffer of a given size with the possibility of increasing it if necessary. Using the developed template in conjunction with the threadlocal qualifier reduces the number of requests for increasing the buffer for high degree B-splines (larger than the initially specified buffer size). It is also possible to implement this scheme using the std::vector template of the C++ STL Standard Library. The results of the application of the developed class when calculating the values of B-splines in a multithreaded environment, showing a reduction in the calculation time in proportion to an increase in the number of computational processors, are presented. The methods of specifying arrays for storing intermediate calculation results considered in this article can be used in other parallel programming tasks.

A Comparison of Scheduling parallel program tasks based on Java Applet

A Comparison of Scheduling parallel program tasks based on Java Applet

Using Thread Local Memory for Calculating B-Splines in Parallel Programming Tasks

Using Thread Local Memory for Calculating B-Splines in Parallel Programming Tasks

English

Bioinformatics is an emerging field, where information technology usage can significantly accelerate life science research. It is a relatively new field and the scope of exploring new tools and techniques seems immense. One major field where bioinformatics plays important role is next generation sequence analysis (NGS), in which an unknown genome is shuttered into pieces and tried to align it to a reference known genome to decipher its functions using sequence comparison. The first well known application of this technology is the human genome project which took nearly 10 years to finish. With advancements in central processing units (CPUs), the alignment time has improved, but has not reached optimal. There seems a constant need to improve this computing time, which made the scope for using graphics processing units (GPUs) and parallel programming tasks to replace CPUs. With access to high performance multi-thread, multi-core parallel computing supercomputers, several GPU based sequence alignment tools have been published recently, some of the major tools are BarraCUDA, CUSHAW, GPU-BWT, SOAP3, and SARUMAN, which claim to speed up the processes anywhere between 2x and 10x times. Most of these tools can be compiled on GCC 4.3 compilers with CUDA. This paper focuses on compiling the current GPU based alignment tools on 70.7 million read pairs (Illumina HiSeq 2000) to align them on a human genome and check its efficiency (time sensitivity and alignment specificity) compared to traditional CPU based alignment (Bowtie) tool. Resulting observations would help researchers choose the appropriate GPU alignment tool to suffice their computing needs. Key words: CUDA, sequencing, alignment, graphics processing units (GPUs), central processing units (CPUs).

Parallel programming with Easy Java Simulations

Nearly all of today's processors are multicore, and ideally programming and algorithm development utilizing the entire processor should be introduced early in the computational physics curriculum. Parallel programming is often not introduced because it requires a new programming environment and uses constructs that are unfamiliar to many teachers. We describe how we decrease the barrier to parallel programming by using a java-based programming environment to treat problems in the usual undergraduate curriculum. We use the easy java simulations programming and authoring tool to create the program's graphical user interface together with objects based on those developed by Kaminsky [Building Parallel Programs (Course Technology, Boston, 2010)] to handle common parallel programming tasks. Shared-memory parallel implementations of physics problems, such as time evolution of the Schrödinger equation, are available as source code and as ready-to-run programs from the AAPT-ComPADRE digital library.

Program Transformation to Identify List-Based Parallel Skeletons

Algorithmic skeletons are used as building-blocks to ease the task of parallel programming by abstracting the details of parallel implementation from the developer. Most existing libraries provide implementations of skeletons that are defined over flat data types such as lists or arrays. However, skeleton-based parallel programming is still very challenging as it requires intricate analysis of the underlying algorithm and often uses inefficient intermediate data structures. Further, the algorithmic structure of a given program may not match those of list-based skeletons. In this paper, we present a method to automatically transform any given program to one that is defined over a list and is more likely to contain instances of list-based skeletons. This facilitates the parallel execution of a transformed program using existing implementations of list-based parallel skeletons. Further, by using an existing transformation called distillation in conjunction with our method, we produce transformed programs that contain fewer inefficient intermediate data structures.

SpiceC

In this paper we present an approach to parallel programming called SpiceC. SpiceC simplifies the task of parallel programming through a combination of an intuitive computation model and SpiceC directives. The SpiceC parallel computation model consists of multiple threads where every thread has a private space for data and all threads share data via a shared space. Each thread performs computations using its private space thus offering isolation which allows for speculative computations. SpiceC provides easy to use SpiceC compiler directives using which the programmers can express different forms of parallelism. It allows developers to express high level constraints on data transfers between spaces while the tedious task of generating the code for the data transfers is performed by the compiler. SpiceC also supports data transfers involving dynamic data structures without help from developers. SpiceC allows developers to create clusters of data to enable parallel data transfers. SpiceC programs are portable across modern chip multiprocessor based machines that may or may not support cache coherence. We have developed implementations of SpiceC for shared memory systems with and without cache coherence. We evaluate our implementation using seven benchmarks of which four are parallelized speculatively. Our compiler generated implementations achieve speedups ranging from 2x to 18x on a 24 core system.

Predicting the execution times of parallel-independent programs using Pearson distributions

Predicting the execution time of parallel programs involves computing the maximum or minimum of the execution times of the tasks involved in the parallel computation. We present a method to accurately compute the distribution of the largest (Max) and the smallest (Min) execution time of the composite of a number of parallel programming tasks, each having an independent, stochastic, arbitrary workload. The Max function applies to the general case that the composite task completes at the time its longest constituent task terminates. The Min function applies when the completion of its shortest task terminates the whole parallel process, such as in a parallel searching program. Both the Min and Max density function of a constituent task are characterized in terms of a Pearson distribution. Due to its accuracy, the presented method is especially of interest when the performance of time critical parallel applications must be derived. Both prediction methods are tested against three well-known distributions. Furthermore, the Max prediction method is also tested against a number of measured real-life data parallel programs with different degree of parallelism. The results show excellent accuracy of better than 1% with a very few exceptions in extreme situations.

Main sequences genetic scheduling for multiprocessor systems using task duplication

This paper addresses the problem of scheduling parallel program tasks onto multiprocessors to minimize its execution time. The scheduling problem is known to be NP-complete, whether task duplication is used or not. To overcome the weakness of the existing genetic scheduling algorithms, a novel chromosome encoding scheme is adopted in the proposed scheduling algorithms. Moreover, the knowledge of the main sequences (MSs) in a task graph is integrated into the proposed algorithm to improve its performance. We compare the TOK algorithm proposed by Tsuchiya et al. with our proposed algorithms in terms of schedule length and running time to find the schedule. Experimental results show the effectiveness of main sequences genetic scheduling algorithm.

Scheduling tasks of a parallel program in two-processor systems with use of cellular automata

In this paper, a cellular automaton (CA) is proposed as a tool for designing distributed scheduling algorithms for allocating parallel program tasks in multiprocessor systems. For this purpose, a program graph is considered as a CA containing elementary automata interacting locally according to some rules. In the first phase of the algorithm, effective rules for the CA are discovered by a genetic algorithm. In the second phase, the CA works as a distributed scheduler. In this phase, for any initial allocation of tasks in a multiprocessor system, the CA-based scheduler finds an allocation minimizing the total execution time of the program in a given system topology. The effectiveness of the proposed scheduling algorithm is shown for a number of program graphs scheduled in a two-processor system.

Genetics-based multiprocessor scheduling using task duplication

We propose a new genetics-based approach to scheduling parallel program tasks on multiprocessors. In the presence of communication delays, it is shown that task duplication is a useful technique for shortening the length of schedules. Though some genetic algorithms (GAs) for multiprocessor scheduling have been proposed so far, none of them allows task duplication. To overcome this deficiency, we develop a new GA, incorporating new genetic operators to control the degree of replication of tasks. Through simulation studies, we show that the proposed GA works effectively, especially when communication delays are relatively small.

A Parallel Matrix Class Library in C++ for Computational Mechanics Applications

The object–oriented design and implementation of a matrix class library in C++ for parallel computing is described. The main goal of this library is to support the management of matrix data in a distributed environment and the parallel solution of linear systems of algebraic equations typical of computational mechanics applications. By using the object–oriented paradigm, a clean and consistent interface to the library is provided, and the implementation details are hidden from the user. These features improve the clarity and expressiveness of the client code and consequently ease the task of parallel programming. The library was developed and tested originally using a network of Sun Sparc 10 workstations. Test examples subsequently were run on the Intel Paragon XP/S. In these examples, the portability of the library among different distributed environments was demonstrated, and some preliminary performance studies of the library were carried out. Finally, the utilization of the class library in parallel finite–element computations is discussed. It is shown that the present library greatly simplifies the implementation of matrix operations intrinsic to parallel finite–element applications.

A message-passing class library C++ for portable parallel programming

An object-oriented message-passing class library in C++, called PPI++, for portable parallel programming has been developed. PPI++ (parallel portability interface in C++) is designed to serve as a stable (unchanging) interface between the client parallel code and the rapidly evolving distributed computing environments. By taking advantage of encapsulation, inheritance, and polymorphism supported by C++, PPI++ provides a clean and consistent programming interface, which helps improve the clarity and expressiveness of client parallel codes and hides implementation details and complexity from the user to ease parallel programming tasks. In addition, the use of strong type-checking in C++ allows the detection of potential misuses of the library at compile time, and thus promotes code reliability. This paper describes the object-oriented design and implementation of PPI++. Evaluation of PPI++ on important performance issues, such as portability, ease-of-use, extensibility, and efficiency, is also discussed.

Object-oriented parallel programming tools for structural engineering applications

The principal objective of this work is to develop portable and extensible programming tools for the development of object-oriented parallel finite element codes for structural engineering applications. An object-oriented parallel portability interface for message-passing operations has been designed and implemented. An existing object-oriented matrix library is currently being extended to support the management of distributed matrix data and parallel solution of linear systems of algebraic equations. By taking advantage of C++ object-oriented programming, both the class libraries provide clean and consistent user interfaces, which not only help to improve the clarity and expressiveness of the client parallel codes, but also hide implementation details and complexity from the user to ease parallel programming tasks. In this paper, the object-oriented design and implementation of the class libraries are discussed. The libraries were first developed and tested using a network of Sun SPARC 10 workstations. Application examples were then studied on two commercial parallel computers: the IBM SP1 and the Intel Paragon XP/S 10, for evaluation of the portability and efficiency of the present class libraries.

A quasi-optimal cluster allocation strategy for parallel program execution in distributed systems using genetic algorithms

This paper shows an approach to find quasi-optimal solutions to the first optimization stage of parallel program tasks allocation problem, in an internet distributed system. The user initiates program execution from an arbitrary node in an arbitrary cluster, the parallel tasks comprising the program migrate to quasi-optimal clusters using an strategy that tries to minimize intercluster traffic of the parallel program execution.Genetic algorithms (GAs) [11] are used to provide a set of timely, quasi-optimal solutions.

Scheduling of precedence-constrained parallel program tasks on multiprocessors

The classical multiprocessor scheduling problem is the problem of scheduling the tasks of a precedence-constrained task graph (representing a parallel program) onto the processors of a multiprocessor in a way that minimizes the completion time. This problem has proven difficult both in theory and in practice. In this paper we present a heuristic algorithm for the multiprocessor scheduling problem considering non-negligible inter-task (inter-processor) communication. The algorithm incorporates a better ‘tie-resolution’ and considers contention in the communication channels of the multiprocessor system, thereby producing realistic schedules. Our performance studies of the algorithm show that it displays promising results. Finally, we propose a simple and novel idea of ‘reverse scheduling’ and demonstrate its effectiveness.

Translating from FP to Occam for systolic algorithms

Programming multiprocessor systems is not a simple task and is often error-prone. In particular, programming a message-passing parallel computer usually demands large amounts of work in development and debugging to avoid deadlock. It is, therefore, essential to have software to support the task of parallel programming, or more desirably to have automatic parallel program generators to greatly reduce the labour. In this paper, we present a novel method of automatically generating Occam programs; or more precisely, we present an automatic FP-to-Occam translator called AFO. Occam is a parallel, message-passing programming language executed on a network of processors called transputers, but is basically a sequential language extended with primitive parallel and communication constructs. In contrast, FP is a functional language that contains no communication details and supports parallel thinking naturally with higher-level parallel constructs. FP has been used to derive many systolic algorithms. Systolic algorithms can be implemented in Occam as soft-systolic algorithms. With this knowledge, we developed AFO to automatically generate Occam programs from FP programs that have corresponding systolic algorithms, in the hope of generating parallel programs without needing to know the details of low-level communications, and thus greatly improving the productivity of parallel programming.

Scheduling parallel program tasks onto arbitrary target machines

We extend previous results for optimally scheduling parallel program tasks on a finite number of parallel processors. We introduce a new scheduling heuristic (MH) that schedules program modules represented as nodes in a precedence task graph with communication onto arbitrary machine topology taking contention into consideration. The results for scheduling simulated task graphs on ring, star, mesh, hypercube, and fully connected networks are also introduced. We also present Task Grapher, a tool for studying optimal parallel program task scheduling on arbitrarily interconnected parallel processors. Given a parallel program represented as a precedence-constrained task graph, and an interconnect topology of a target machine, Task Grapher uses one or more of its seven scheduling heuristics to produce the following displays: (1) Gantt Chart Schedule, (2) Speedup Line Graph, (3) Critical Path in Task Graph, (4) Processor Utilization Chart, (5) Processor Efficiency Chart, and (6) Dynamic Activity Display.