Distributed Memory Multiprocessor Systems Research Articles

Multidomain implicit numerical scheme

A multidomain method for the solution of elliptic CFD problems with an ADI scheme is described. Two methods of treatment of internal boundary conditions for ADI functions are discussed, namely an explicit and a semi‒implicit method. Stability conditions for the proposed methods are derived theoretically. The semi‒implicit scheme is more stable than the explicit scheme, leading to improved numerical efficiency for multidomain computations. Numerical computations for a linear convection-diffusion equation, for buoyancy‒driven recirculating flow in a square cavity and for turbulent flow in a square duct confirmed the theoretical results. Computer runs of the multidomain code in a distributed memory multiprocessor system were successful and efficient and produced reliable results. © 1997 John Wiley & Sons, Ltd.

A parallel automatic differentiation algorithm for simulation models

Automatic differentiation is a technique to compute the exact values of derivatives numerically, without using any formula manipulation software. Recently it has been included in several simulation algorithms especially those which require real-time computation. This paper presents new software to compute first and second-order derivatives in parallel on a distributed memory multiprocessor system. The code has been extensively tested on a Transputer network; the results are very encouraging, showing a substantial speed-up for several test problems.

A Compiler-Directed Approach to Network Latency Reduction for Distributed Shared Memory Multiprocessors

In distributed shared memory multiprocessor systems, parallel tasks communicate through sharing memory data. As the system size increases, such communication cost becomes the main factor that limits the overall parallelism and performance. In this paper, we propose a new solution to the problem through judiciously managing the relevant resource, namely, the shared data and the interconnection network (IN) through which the sharing is carried out. In this approach, communication cost is minimized by means of data migration/allocation which is based on analyzing general layered task graphs, sharing behavior of parallel tasks, and network topology. Our method is not applicable for read only variables. Further, for the time being, the usefulness of the method is limited to multiprocessors where no cache coherence mechanism is implemented. Four typical interconnection topologies for multiprocessors are considered, namely, shared-bus, hierarchical-bus, 2-D mesh, and fat-tree structures. Efficient data allocation algorithms for each of the four network topologies are developed that make decision on data allocation/migration at the compile time. The complexity of one algorithm isO(np) for shared-bus andO(n2p) for the remaining three in a system withnprocessors executing ap-layer task graph for one shared variable. We have also given an algorithm to determine optimal allocation/migration scheme for multiple shared variables. However, the cost of the algorithm become prohibitive when the number of shared variables is high. Therefore, a heuristic of low complexity is suggested. The heuristic is optimal for some topologies.

A parallel implementation of automatic differentiation for partially separable functions using PVM

In this paper we present a parallel implementation of Automatic Differentiation (AD for short) for computing, numerically, first and second order derivatives, without using any formula manipulation software, on a distributed memory multiprocessor system using PVM. The code is written in Fortran 77 and tested on a cluster of workstations connected via FDDI links. With the aim to evaluate the efficiency and effectiveness of the proposed approach, the tool has been embedded in standard numerical codes for solving nonlinear optimization problems. The performance has been evaluated by comparing the results with those obtained by using a parallel routine implementing finite-difference approximation formulas. The approach seems to be very encouraging, since the results show a substantial speed-up for several test problems when we use AD.

Parallel asynchronous team algorithms: convergence and performance analysis

This paper formalizes a general technique to combine different methods in the solution of large systems of nonlinear equations using parallel asynchronous implementations on distributed-memory multiprocessor systems. Such combinations of methods, referred to as team algorithms, are evaluated as a way of obtaining desirable properties of different methods and a sufficient condition for their convergence is derived. The load flow problem of electrical power networks is presented as an example problem that, under certain conditions, has the characteristics to make a team algorithm an appealing choice for its solution. Experimental results of an implementation on an Intel iPSC/860 Hypercube are reported, showing that considerable speedup and robustness can be obtained using team algorithms.

Comparative study of task duplication static scheduling versus clustering and non‐clustering techniques

AbstractOne of the major issues that needs to be addressed in distributed memory multiprocessor (DMM) systems is the program task partitioning and scheduling problems, i.e. mapping of an application program's precedence related task threads among the processing elements of a DMM system. The optimal task partitioning and scheduling problem, with the goal of minimizing the program execution time and interprocessor communication overhead, is known to be an NP‐complete problem. The paper addresses the design, development and performance evaluation of a novel static task partitioning and scheduling method called linear clustering with task duplication (LCTD). LCTD employs the linear (sequential) execution of tasks and task duplication heuristics in achieving minimized computation and interprocessor communication delays in DMMs. The superiority of the proposed LCTD algorithm is demonstrated through simulation studies and comparison against several of the existing static scheduling schemes, such as heavy node first (HNF) and linear clustering. We show that the proposed method can obtain an average of 33% improvement in program execution time and 21% improvement in processor utilization compared to linear clustering and HNF methods.

A scalable scheduling scheme for functional parallelism on distributed memory multiprocessor systems

We attempt a new variant of the scheduling problem by investigating the scalability of the schedule length with the required number of processors, by performing scheduling partially at compile time and partially at run time. Assuming infinite number of processors, the compile time schedule is found using a new concept of the threshold of a task that quantifies a trade-off between the schedule-length and the degree of parallelism. The schedule is found to minimize either the schedule length or the number of required processors and it satisfies: A feasibility condition which guarantees that the schedule delay of a task from its earliest start time is below the threshold, and an optimality condition which uses a merit function to decide the best task-processor match for a set of tasks competing for a given processor. At run time, the tasks are merged producing a schedule for a smaller number of available processors. This allows the program to be scaled down to the processors actually available at run time. Usefulness of this scheduling heuristic has been demonstrated by incorporating the scheduler in the compiler backend for targeting Sisal (Streams and Iterations in a Single Assignment Language) on iPSC/860. >

SHARP: A shape recognition system and its parallel implementation

A parallel algorithm for shape recognition is presented along with its implementation on a distributed memory multiprocessor. Shape recognition is one of the fundamental problems of computer vision. We consider a shape to be composed of a set of small straight line segments tangential to the object. The recognition problem is to determine whether the test image contains a specified reference shape or not. The straight line Hough transform (SLHT) has been used to detect reference shapes. A signature based parallel algorithm called SHARP is developed for shape recognition using SLHT on a distributed memory multiprocessor system. In the SHARP algorithm, the (θ, r) space is divided among processors. The SHARP algorithm has been implemented on a Meiko transputer with 32 nodes. We analyse the performance of the parallel algorithm using both theoretical and experimental techniques.

THE RANDOM SEARCH GLOBAL OPTIMIZATION METHOD FOR PARALLEL COMPUTERS

In this paper we present a strategy for random search global optimization for parallel computers. The implementation on a shared memory parallel computer is coded in FORTRAN and the implementation on a distributed memory multiprocessor system is coded in C using the processor farming technique. Results on known test examples confirm the conclusions.

Parallel implementations of convolution and moments algorithms on a multi-transputer system

This paper describes the implementations of convolution and regular moments on a transputer network. Discrete convolution is the principal spatial domain method for digital image enhancement. Moments are by far the most popular descriptors for image regions and boundary segments. Both convolution and moments are computationally expensive and difficult to accomplish in real time. To reduce computational time, parallel implementations of convolution and moments were investigated and the details of the best two implementations employing different interprocessor communication topologies on a multi-transputer system are described. Two theoretical performance models based on the implementations are used to predict the number of processors needed to satisfy the requirements for a real-time image processing system. The methodology presented for parallel processing can be easily adapted for other distributed memory multiprocessor systems.

Language Constructs for Data Partitioning and Distribution

This article presents a survey of language features for distributed memory multiprocessor systems (DMMs), in particular, systems that provide features for data partitioning and distribution. In these systems the programmer is freed from consideration of the low‐level details of the target architecture in that there is no need to program explicit processes or specify interprocess communication. Programs are written according to the shared memory programming paradigm but the programmer is required to specify, by means of directives, additional syntax or interactive methods, how the data of the program are decomposed and distributed.

Compiling CIL rewriting language for multiprocessors

The high-level Compiler Intermediate Language CIL is a general-purpose description language of parallel graph rewriting computational model intended for parallel implementation of declarative languages on multiprocessor systems. In this paper, we first outline a new Hybrid Execution Model(HEM) and corresponding parallel abstract machine PAM/TGR based on Extended parallel Graph Rewriting Computational Model EGRCM for implementing CIL language on distributed memory multiprocessor systems. Then we focus on the compiling CIL language with various optimizing techniques such as pattern matching, rule indexing, node ordering and compile-time partial scheduling. The experimental results on a 16-node Transputer Array demonstrates the effectiveness of our model and strategies.

Analysis of processor allocation in multiprogrammed, distributed-memory parallel processing systems

A main objective of scheduling independent jobs composed of multiple sequential tasks in shared-memory and distributed-memory multiprocessor computer systems is the assignment of these tasks to processors in a manner that ensures efficient operation of the system. Achieving this objective requires the analysis of a fundamental tradeoff between maximizing parallel execution, suggesting that the tasks of a job be spread across all system processors, and minimizing synchronization and communication overheads, suggesting that the job's tasks be executed on a single processor. The authors consider a class of scheduling policies that represent the essential aspects of this processor allocation tradeoff, and model the system as a distributed fork-join queueing system. They derive an approximation for the expected job response time, which includes the important effects of various parallel processing overheads (such as task synchronization and communication) induced by the processor allocation policy.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An adaptive switching architecture for multiprocessor networks

The communication architecture in a distributed memory multi-processor system must provide reliable and flexible communications with minimal latency to fully utilize the performance advantage offered by the inherent parallelism. An inefficiently designed architecture introduces ‘hot spots’ in the inter-processor communications and degrades the potential speed-up possible. This paper compares existing switching strategies on multi-processor systems and describes the development of an adaptive switching architecture for a circuit-switched multi- processor system with an arbitrary interconnection topology. The switching architecture employs dynamic reconfiguration of the communications path to provide highly efficient and fault-tolerant inter-processor communications.

Transputer-Based Multibody System, Dynamic Simulation, Part II: Parallel Implementation—Results∗

Abstract This paper continues demonstration of the benefits of parallel multibody simulation. In Part I implicit integration schemes combined with a modified inverse dynamic formulation were proposed for parallel multibody simulation. In Part II the modified inverse dynamics algorithm and computation of the integrator's Jacobian are split up into independent work packages for parallelization on medium-and coarse-grain levels, respectively. The farming concept, which organizes the flow of work and result packages through a network of processors, is presented. Transputers are used to build a distributed memory multiprocessor system, where both the medium- and coarse-grain approaches are implemented, resulting in the PARSIM-package for parallel multibody dynamics. The simulation of an off-road vehicle and a multibody model for a cable demonstrate the considerable savings in simulation times achieved by the new software package.

A development environment for rapid prototyping of user applications on distributed memory multiprocessor systems

There is a well acknowledged need for convenient prototype development environments for parallel applications software on loosely coupled, distributed memory multiprocessor systems which support synchronized message passing as the primary means of interprocess communication. In addition, such an environment should provide facilities forexperimenting with the allocation of processors to user processes and for subsequent user program monitoring with a view to achieving performance optimization and rapid prototyping. This paper describes the principles behind an actual design of such a system and highlights the major implementation issues.

PRIP – A Parallel Raster Image Processor

AbstractLike colour video displays and laser printers, laser photoplotters are raster scan devices. For such devices, the pixel stream representing the image must be generated in real time, and in the (scan line) order required. However, the typical size of the images photoplotters produce is an order of magnitude higher than that of video displays and laser printers, precluding the use of full‐size bitmap memories. These requirements pose particular implementation problems for the raster image processor generating the pixel stream.The parallel RIP system presented here is aimed at high‐resolution laser photoplotters, and features a largely scalable performance ranging from 40 to several hundred megapixels per second. It is built with standard components such as graphics microprocessors and VRAM memories. Its architecture is that of a distributed memory multiprocessor system with a global ring‐like topology. And most importantly, it can be programmed using the traditional sequential programming paradigm. Only minor additions are needed to sequential graphical algorithms to be executable on the system with an arbitrary number of processors.A prototype 8‐processor PRIP system has been built and tested generating printed circuit board images for a direct imaging photoplotter. The prototype exhibits a near‐linear speedup with respect to a monoprocessor solution. Architectural simulations indicate that the system can be expanded to well over 10 processors.

Coprocessor design for multilayer surface-mounted PCB routing

The authors present the issues involved in the design of a special-purpose processing array system, called HAM, which accelerates computationally intensive wire routing tasks. It is especially suited for double-sided surface-mounted boards, which require complex three-dimensional search operations over multiple wiring planes. The novel features of the design include a hexagonal interconnection scheme to improve workload distributions during multilayer concurrent search operations and the VLSI custom design of the processors. Particular emphasis has been placed on the demands of maze routing. A cell-address propagation scheme, which is quite different from the traditional grid-coordinate approach, is discussed. It provides rapid lookup of pertinent routing information and can be extended to any distributed memory multiprocessor system. A global pipelining scheme of cell updates and expands is discussed. Experimental results are presented relating the speedup to various criteria for two different modes of parallel wave propagation. >

Supernodal symbolic Cholesky factorization on a local-memory multiprocessor

In this paper, we consider the symbolic factorization step in computing the Cholesky factorization of a sparse symmetric positive definite matrix on distributed-memory multiprocessor systems. By exploiting the supernodal structure in the Cholesky factor, the performance of a previous parallel symbolic factorization algorithm is improved. Empirical tests demonstrate that there can be drastic reduction in the execution time required by the new algorithm on an Intel iPSC/2 hypercube.

Analysis of minimal path routing schemes in the presence of faults

The design and analysis of fault tolerant message routing schemes for large parallel systems has been the focus of much recent research. In this paper, we present a framework for the analysis of routing schemes in distributed memory multiprocessor systems containing faulty or unusable components. We introduce techniques for the derivation of the probabilities of succesfully routing a single message using minimal path routing schemes. Using this framework, we derive closed form solutions for a wide range of routing schemes on the hypercube and on the two- dimensional mesh. The results obtained show the surprising resilience of the hypercube to a potentially large number of faults while demonstrating the inability of the mesh to tolerate a comparatively smaller number of faults.