Distributed-memory MIMD Computers Research Articles

An efficient parallel termination detection algorithm

In this paper, we present a new, easy to implement algorithm for detecting the termination of a parallel asynchronous computation on distributed-memory MIMD computers. We demonstrate that it operates concurrently with the main computation, adding minimal overhead, and we prove that it correctly detects termination when it occurs. Experimental results confirm that the termination detection routine imposes an overhead smaller than the experimental uncertainty.

The PMESC Programming Library for Distributed-Memory MIMD Computers

Efficient programming of task-parallel problems, where the number and execution times of the computational tasks can vary unpredictably, demands an asynchronous and adaptive approach. In this sort of approach, however, such fundamental programming issues as load sharing, data sharing, and termination detection can present difficult programming problems. This paper presents the PMESC library for managing task-parallel problems on distributed-memory MIMD computers within the context of the SPMD (single program, multiple data) programming model. PMESC offers support for all of the application-independent programming issues involved in SPMD task-parallel computation in a portable and efficient way while still allowing users to customize their codes. Because different problems may require different strategies to achieve good performance, PMESC is based on a straightforward model in which different building blocks can be easily put together and changed to accommodate the particular needs of the different applications. The library provides an interface that allows users to program a virtual machine and thereby ignore the details associated with message passing and machine architecture. These features make PMESC accessible to a wide variety of users.

Parallel Learning of Belief Networks in Large and Difficult Domains

Learning belief networks from large domains can be expensive even with single-link lookahead search (SLLS). Since a SLLS cannot learn correctly in a class of problem domains, multi-link lookahead search (MLLS) is needed which further increases the computational complexity. In our experiment, learning in some difficult domains over more than a dozen variables took days. In this paper, we study how to use parallelism to speed up SLLS for learning in large domains and to tackle the increased complexity of MLLS for learning in difficult domains. We propose a natural decomposition of the learning task for parallel processing. We investigate two strategies for job allocation among processors to further improve load balancing and efficiency of the parallel system. For learning from very large datasets, we present a regrouping of the available processors such that slow data access through the file system can be replaced by fast memory access. Experimental results in a distributed memory MIMD computer demonstrate the effectiveness of the proposed algorithms.

Parallel adaptive hp-refinement techniques for conservation laws

We describe an adaptive hp-refinement local finite element procedure for the parallel solution of hyperbolic systems of conservation laws on rectangular domains. The local finite element procedure utilizes spaces of piecewise-continuous polynomials of arbitrary degree and coordinated explicit Runge-Kutta temporal integration. A solution limiting procedure produces monotonic solutions near discontinuities while maintaining high-order accuracy near smooth extrema. A modified tiling procedure maintains processor load balance on parallel, distributed-memory MIMD computers by migrating finite elements between processors in overlapping neighborhoods to produce locally balanced computations. Grids are stored in tree data structures, with finer grids being offspring of coarser ones. Within each grid, AVL trees simplify the transfer of information between neighboring processors and the insertion and deletion of elements as they migrate between processors. Computations involving Burgers' and Euler's equations of inviscid flow demonstrate the effectiveness of the hp-refinement and balancing procedures relative to non-balanced adaptive and balanced non-adaptive procedures.

Molecular dynamics on distributed-memory MIMD computers with load balancing

We report two aspects of a computational molecular dynamics study of large-scale problems on a distributed-memory MIMD parallel computer: (1) efficiency and scalability results on Intel Paragon parallel computers with up to 1024 nodes for 400 million particles and (2) a new method for dynamic load balancing.

Implementation of parallel block preconditionings on a transputer-based multiprocessor

In this paper we are concerned with the implementation on distributed memory MIMD computers of parallel block preconditionings for solving large and sparse symmetric positive definite linear systems. For this purpose, a recently proposed ordering strategy is used. Experimental tests performed on a transputer-based multiprocessor computer are reported, displaying that good performance can be achieved in exchange for some optimization effort and on to the assumption that the number of processors is smaller by orders of magnitude than the size of the problem to solve, which agrees with observations made elsewhere.

Compiling Fortran 90D/HPF for Distributed Memory MIMD Computers

This paper describes the design of the Fortran90D/HPF compiler, a source-to-source parallel compiler for distributed memory systems being developed at Syracuse University. Fortran 90D/HPF is a data parallel language with special directives to specify data alignment and distributions. A systematic methodology to process distribution directives of Fortran 90D/HPF is presented. Furthermore, techniques for data and computation partitioning, communication detection and generation, and the run-time support for the compiler are discussed. Finally, initial performance results for the compiler are presented. We believe that the methodology to process data distribution, computation partitioning, communication system design, and the overall compiler design can be used by the implementors of compilers for HPF.

OPTIMIZING THE MATRIX MULTIPLICATION PARALLEL ALGORITHMS ON A DISTRIBUTED-MEMORY MIMD MULTIPROCESSOR

In the past few years, there have been significant developments in the area of distributed and parallel processing. More powerful and new hardware architectures are being produced at a rapid rate, such as distributed-memory MIMD computers, which have provided enormous computing power to the software engineers. These multiprocessors may provide a significant speed-up over the serial execution of an algorithm. However, this requires careful partitioning and allocation of data and control to the processor set. Matrix multiplication is a fundamental parallel algorithm which can be effectively executed on a distributed-memory multiprocessor and can show significant improvement in the speed-up over the serial execution. Ideally, we should be able to achieve a linear speed up with increase in the number of processors, but in practice the speed up is much less, and in fact increasing the number of processors beyond a certain number may result in degradation of the completion time. This degradation is caused by in...

The calculation of separated flows using a distributed memory mimd computer

AbstractA brief description of distributed memory MIMD computers, and in particular, a transputer based computing surface, is presented. The factors to be considered in their application to computationally intensive CFD problems are discussed with reference to the discrete vortex method used for the solution of separated flow about bluff bodies. Three parallel algorithms for its implementation are presented, and the results are discussed in the context of the speed‐ups obtained using up to sixty‐four parallel processors.

Molecular Dynamics Simulation of Aerogel Silica on Parallel Computers

AbstractMolecular dynamics (MD) simulations of porous silica, in the density range 2.2 - 0.1 g/cm3, are carried out on a 41,472 particle system using two- and three-body interatomic potentials. Calculated results for fractal dimension and small-angle neutron scattering data are in good agreement with neutron scattering experiments. Results for structural correlations reveal crossovers from the short- to intermediate range (< 8 Å) and fractal to large-scale regime (10 ∼ 100 Å).The MD program simulations are carried out on distributed-memory MIMD computers using a domain-decomposition algorithm. The algorithm employs the linked-celllist method and separable three-body force calculation. The force calculation is accelerated by the multiple-time-step method. The parallel algorithm is highly efficient (parallel efficiency = 0.974), as it involves only 3 % communication overhead.

Parallel Triangular System Solving on a Mesh Network of Transputers

A parallel algorithm is presented for triangular system solving on a distributed-memory MIMD computer with a square mesh topology. The algorithm is based on the square grid (scattered) distribution of matrix elements across the processors. The theoretical time complexity is $n^2/ p + O(n)$, for p processors and an $n \times n$ matrix. Experimental timings of an implementation in occam 2 on a square mesh of $p = 36$ transputers confirm the theoretical time model. The scaled speedup achieved for $n = 1200$ is 24 on a 36 transputer mesh. This corresponds to a computing rate of 11.7 Mflop/s.

Parallel Triangular System Solving on a Mesh Network of Transputers

A parallel algorithm is presented for triangular system solving on a distributed-memory MIMD computer with a square mesh topology. The algorithm is based on the square grid (scattered) distribution...

A MIMD implementation of the Buchberger Algorithm for Boolean polynomials

In this note we present two methods to compute Gröbner basis in parallel, both based on Buchberger's sequential algorithm. A distributed memory MIMD computer (the FPS T40) gives experimental results obtained with the Boolean polynomials. The algorithms were implemented on the FPS T40 connected as a ring and as a hypercube of processors. The first implementation shows the interest of the parallelization. The second one, based on a divide and conquer strategy, has a behaviour very close to the sequential algorithm.

A Parallel Finite Element Software Package:Design And Implementation

This paper describes the design and implementation of a parallel finite element software package capable of solving linear boundary value problems on a network of transputers. Many of the design and development issues are discussed that had to be considered in the evolution of efficient software for running on a distributed memory MIMD computer. Typical experimental results are presented and discussed.