Real Parallel Machines Research Articles

New BSP/CGM algorithms for spanning trees

Computing a spanning tree (ST) and a minimum ST (MST) of a graph are fundamental problems in graph theory and arise as a subproblem in many applications. In this article, we propose parallel algorithms to these problems. One of the steps of previous parallel MST algorithms relies on the heavy use of parallel list ranking which, though efficient in theory, is very time-consuming in practice. Using a different approach with a graph decomposition, we devised new parallel algorithms that do not make use of the list ranking procedure. We proved that our algorithms are correct, and for a graph [Formula: see text], [Formula: see text], and [Formula: see text], the algorithms can be executed on a Bulk Synchronous Parallel/Coarse Grained Multicomputer (BSP/CGM) model using [Formula: see text] communications rounds with [Formula: see text] computation time for each round. To show that our algorithms have good performance on real parallel machines, we have implemented them on graphics processing unit. The obtained speedups are competitive and showed that the BSP/CGM model is suitable for designing general purpose parallel algorithms.

P systems with proteins on membranes characterize PSPACE

The paper studies algorithmic properties of operations with membrane proteins modeled within the framework of membrane systems (also called P systems). Membrane systems are biologically inspired models of parallel and distributed computing based on the information processing in cells and cellular membranes. We show that the computational potential of P systems with proteins on membranes is equivalent to that of parallel computing models as the alternating Turing machine or the PRAM. These abstract machines characterize by their polynomial time-bounded computations the class PSPACE, and simultaneously they serve as idealized models of real parallel machines. Therefore, this and other related results suggest the existence of a homology between the potential of silicon and biological parallel information processing.

A Dynamic Load Balancing Framework for Real-time Applications in Message Passing Systems

Load balancing algorithms are designed essentially to equally distribute the load on processors and maximize their utilities while minimizing the total task execution time. In order to achieve these goals, the load-balancing mechanism should be “fair” in distributing the load across the different processors. This implies that the difference between the heaviest-loaded and the lightest-loaded processors should be minimized. Therefore, the load information on each processor must be updated such that the load-balancing mechanism can be more effective. In this work, we present an application independent dynamic algorithm for scheduling tasks and load- balancing in message passing systems. We propose a DAG-based Dynamic Load Balancing algorithm for Real time applications (DAG-DLBR) that is designed to work dynamically to cope with possible changes in the load that might occur during runtime. This algorithm addresses the challenge of devising a load balancing scheme which judicially deals with the hybrid execution of existing real-time application (represented by a Direct Acyclic Graph (DAG)) together with newly arriving jobs. The main objective of this algorithm is to reduce response times of the newly arriving jobs while maintaining the time constrains of the existing DAG. To evaluate the performance of the DAG-DLBR algorithm, a comparison with the performance of two common dynamic load balancing algorithms is presented. This comparison is performed by evaluating, experimentally, the execution time of different load balancing algorithms on a homogenous real parallel machine. In addition, the values of load imbalance, the execution time, and the communication overhead time are evaluated analytically using different benchmarks as test-bed workloads. These workloads cover a wide range of dynamic applications with different task types. Experimental results illustrate the improved performance of the DAG-DLBR algorithm compared to both distributed and hierarchal based algorithms by at least 12 and 19%, respectively. This improvement is true for all workloads, even with highly dependent workload. The DAG-DLBR algorithm achieves lower computation time than its corresponding values of both the distributed and the hierarchical-based algorithms for 4, 8, 12 and 16 processors.

An Experimental Analysis of Parallel Sorting Algorithms

We have developed a methodology for predicting the performance of parallel algorithms on real parallel machines. The methodology consists of two steps. First, we characterize a machine by enumerating the primitive operations that it is capable of performing along with the cost of each operation. Next, we analyze an algorithm by making a precise count of the number of times the algorithm performs each type of operation. We have used this methodology to evaluate many of the parallel sorting algorithms proposed in the literature. Of these, we selected the three most promising, Batcher's bitonic sort, a parallel radix sort, and a sample sort similar to Reif and Valiant's flashsort, and implemented them on the connection Machine model CM-2. This paper analyzes the three algorithms in detail and discusses the issues that led us to our particular implementations. On the CM-2 the predicted performance of the algorithms closely matches the observed performance, and hence our methodology can be used to tune the algorithms for optimal performance. Although our programs were designed for the CM-2, our conclusions about the merits of the three algorithms apply to other parallel machines as well.

Integrating parallel algorithm design with parallel machine models

In teaching of Parallel Algorithms, emphasis is usually placed on algorithms that are based on parallel random access machine (PRAM) model, but those algorithms may or may not be efficiently implementable on real parallel machines. In this paper we design parallel algorithms in relation with the actual parallel machine models and show the importance of integrating parallel algorithms with parallel machine models. We discuss this approach using traveling salesperson problem (TSP) as an example.

Parallax: a tool for parallel program scheduling

Parallax, a scheduling tool that incorporates seven traditional and nontraditional scheduling heuristics and lets developers compare their performance for real applications on real parallel machines, is discussed. Of the seven heuristics, two simple ones consider only task execution time, two consider both task execution and message-passing delay times, two use task duplication to reduce communication delay, and one considers communication delays, task execution time, and target machine characteristics such as interconnection network topology and overhead due to message-passing and process creation. Two examples of parallel applications of Parallax are described. >

Commentary—Speedups and Insights

It is a pleasure to be invited to participate in the discussion of this thoughtful paper, “Reporting Computational Experiments with Parallel Algorithms: Issues, Measures, and Experts' Opinions” by Richard S. Barr and Betty L. Hickman. As one who has published many a table of speedups, I feel very much in the authors' audience, perhaps even in their crosshairs. Their paper focuses attention on an issue faced by every reader as well as writer of articles that report the results of experiments with parallel programs running on real parallel machines. I believe the paper can contribute to more accurate communication in the parallel processing community through a heightened awareness of the issues surrounding reporting of experiments. The survey is a novel addition to the discussion, and seems to indicate that the field's participants are seriously grappling with this problem, although no consensus has developed. What I hope to contribute here are a few opinions on why there is still a quite legitimate variation in what results are reported and how they are measured and presented, as well as some experiences in my own research that might reinforce some of the conclusions in the paper. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

Towards a single model of efficient computation in real parallel machines

We propose a model of parallel computation, the YPRAM, that allows general parallel algorithms to be designed for a wide class of parallel models. The basic model captures locality among processors, which is measured as a function of two parameters; latency and bandwidth. We design YPRAM algorithms for solving several fundamental problems: parallel prefix, sorting, sorting numbers from a bounded range, and list ranking. We show that our model predicts, reasonably accurately, the actual known performances of several basic parallel models — PRAM, hypercube, mesh and tree — when solving these problems.

A comparative study of five parallel programming languages

Many different paradigms for parallel programming exist, nearly each of which is employed in dozens of languages. Several researchers have tried to compare these languages and paradigms by examining the expressivity and flexibility of their constructs. Few attempts have been made, however, at practical studies based on actual programming experience with multiple languages. Such a study is the topic of this paper. We will look at five parallel languages, all based on different paradigms. The languages are: SR (based on message passing), Emerald (concurrent objects), Parlog (parallel Horn clause logic), Linda (Tuple Space), and Orca (logically shared data). We have implemented the same parallel programs in each language, using real parallel machines. The paper reports on our experiences in implementing three frequently occurring communication patterns: message passing through a mailbox, one-to-many communication, and access to replicated shared data.

Simulating parallel architectures in a distributed environment

This paper describes a simulation tool called the Simulated Parallel Architectures in a Distributed Environment (SPADE). SPADE is currently implemented on a network of workstations with UNIX (trademark of AT&T Bell Laboratories) and NFS (trademark of Sun Microsystems). It is developed to provide an affordable and flexible environment for the verification and analysis of parallel algorithms. With SPADE, a user can experiment with various existing or hypothetical parallel machine models and carry out preliminary studies on aspects such as communication pattern and parallelism granularity without requiring access to a real parallel machine. As an example, a SIMD object labeling algorithm is implemented on SPADE. The simulation results are analyzed and their accuracy is validated against the performance of the same labeling algorithm on the Connection Machine 2.

Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel Supercomputers

This paper proposes guided self-scheduling, a new approach for scheduling arbitrarily nested parallel program loops on shared memory multiprocessor systems. Utilizing loop parallelism is clearly most crucial in achieving high system and program performance. Because of its simplicity, guided self-scheduling is particularly suited for implementation on real parallel machines. This method achieves simultaneously the two most important objectives: load balancing and very low synchronization overhead. For certain types of loops we show analytically that guided self-scheduling uses minimal overhead and achieves optimal schedules. Two other interesting properties of this method are its insensitivity to the initial processor configuration (in time) and its parameterized nature which allows us to tune it for different systems. Finally we discuss experimental results that clearly show the advantage of guided self-scheduling over the most widely known dynamic methods.

Multiprocessing the Sieve of Eratosthenes

The Sieve of Eratosthenes for finding prime numbers in recent years has seen much use as a benchmark algorithm for serial computers while its intrinsically parallel nature has gone largely unnoticed. The implementation of a parallel version of this algorithm for a real parallel computer, the Flex/32, is described and its performance discussed. It is shown that the algorithm is sensitive to several fundamental performance parameters of parallel machines, such as spawning time, signaling time, memory access, and overhead of process switching. Because of the nature of the algorithm, it is impossible to get any speedup beyond 4 or 5 processors unless some form of dynamic load balancing is employed. We describe the performance of our algorithm with and without load balancing and compare it with theoretical lower bounds and simulated results. It is straightforward to understand this algorithm and to check the final results. However, its efficient implementation on a real parallel machine requires thoughtful design, especially if dynamic load balancing is desired. The fundamental operations required by the algorithm are very simple: this means that the slightest overhead appears prominently in performance data. The Sieve thus serves not only as a very severe test of the capabilities of a parallel processor but is also an interesting challenge for the programmer.