Distributed Memory Parallel System Research Articles

Iterative solution of elliptic collocation systems on a cognitive parallel computer

The performance of parallel algorithms implementing the block SOR iterative method, used for the solution of linear systems arising from the discretization of elliptic BVPs by the finite element collocation method, is investigated in this paper. By making use of a recently introduced block tridiagonal partitioning of the collocation matrix, which yields faster rates of convergence, and its associated modified red-black ordering of unknowns and equations, we succeeded to increase the scalability of the problem. The study of the communicational and computational needs of the problem on a virtual parallel machine made possible the mapping of the resulted SOR-algorithm on a fixed size distributed memory parallel system, utilizing a simple pipeline architecture among its processors, in an optimum way. The proposed parallel schemes are being realized on a Parsytec cognitive computer via an SPMD programming model. Speedup measurements are used to reveal the efficiency of our implementation.

Parallel matrix multiplication on a linear array with a reconfigurable pipelined bus system

The known fast sequential algorithms for multiplying two N/spl times/N matrices (over an arbitrary ring) have time complexity O(N/sup /spl alpha//), where 2</spl alpha/<3. The current best value of /spl alpha/ is less than 2.3755. We show that, for all 1/spl les/p/spl les/N/sup /spl alpha//, multiplying two N/spl times/N matrices can be performed on a p-processor linear array with a reconfigurable pipelined bus system (LARPBS) in O(N/sub m//P+(N/sup 2//p/sup 2//spl alpha//)log p) time. This is currently the fastest parallelization of the best known sequential matrix multiplication algorithm on a distributed memory parallel system. In particular, for all 1/spl les/p/spl les/N/sup 2.3755/, multiplying two N/spl times/N matrices can be performed on a p-processor LARPBS in O(N/sup 2.3755//p+(N/sup 2/)/p/sup 0.8419/log p) time and linear speedup can be achieved for p as large as O(N/sup 2.3755//(log N)/sup 6.3262/). Furthermore, multiplying two N/spl times/N matrices can be performed on an LARPBS with O(N/sup /spl alpha//) processors in O(log N) time. This compares favorably with the performance on a PRAM.

Pajé, an interactive visualization tool for tuning multi-threaded parallel applications

This paper describes Pajé, an interactive visualization tool for displaying the execution of parallel applications where a potentially large number of communicating threads of various life-times execute on each node of a distributed memory parallel system. Pajé is capable of representing a wide variety of interactions between threads. The main characteristics of Pajé, interactivity and scalability, are exemplified by the performance tuning of a molecular dynamics application. In order to be easily extensible, the architecture of the system was based on components which are connected in a data flow graph to produce a given visualization tool. Innovative components were designed, in addition to “classical” components existing in similar visualization systems, to support scalability and interactivity.

Automatic code generation of overlapped communications in a parallelisation tool

This paper addresses the exploitation of overlapping communication with calculation within parallel FORTRAN 77 codes for computational fluid dynamics (CFD) and computational structured dynamics (CSD). The obvious objective is to overlap interprocessor communication with calculation on each processor in a distributed memory parallel system and so improve the efficiency of the parallel implementation. A general strategy for converting synchronous to overlapped communication is presented together with tools to enable its automatic implementation in FORTRAN 77 codes. This strategy is then implemented within the parallelisation toolkit, CAPTools, to facilitate the automatic generation of parallel code with overlapped communications. The success of these tools are demonstrated on two codes from the NAS-PAR and PERFECT benchmark suites. In each case, the tools produce parallel code with overlapped communications which is as good as that which could be generated manually. The parallel performance of the codes also improve in line with expectation.

A high-speed network interface for distributed-memory systems

Distributed-memory systems have traditionally had great difficulty performing network I/O at rates proportional to their computational power. The problem is that the network interface has to support network I/O for a supercomputer, using computational and memory bandwidth resources similar to those of a workstation. As a result, the network interface becomes a bottleneck. In this article we present an I/O architecture that addresses these problems and supports high-speed network I/O on distributed-memory systems. The key to good performance is to partition the work appropriately between the system and the network interface. Some communication tasks are performed on the distributed-memory parallel system, since it is more powerful and less likely to become a bottleneck than the network interface. Tasks that do not parallelize well are performed on the network interface, and hardware support is provided for the most time-critical operations. This architecture has been implemented for the iWarp distributed-memory system and has been used by a number of applications. We describe this implementaiton, present performance results, and use application examples to validated the main features of the I/O architecture.

Operating system support for parallel computation

Parallel systems are not necessarily special-purpose machines. At present, most departmental servers already resort to shared memory multiprocessing as a means to increase performance, while a network of workstations can also be regarded as a distributed memory parallel system. This paper examines the support offered by the operating system to exploit such parallelism. After discussing the design of multi-threaded programs in a Unix environment, a comparison is made with their distributed counterparts. Some performance figures obtained on a SparcCenter 2000 multiprocessor. on a network of workstations and on a transputer-based system are presented.

Architecture and evaluation of a high-speed networking subsystem for distributed-memory systems

Achieving high-speed network I/O on distributed-memory systems is difficult because their architecture is in general ill-suited for communication processing. Some of the common problems are: inability to do protocol processing, inefficient handling of data distribution, and poor management of the I/O. In this paper we present an I/O architecture that addresses these problems and supports high-speed network I/O on distributed-memory systems. The key to good performance is to partition the work appropriately between the system and the network interface. We perform some communication tasks on the distributed-memory parallel system since it is more powerful, and less likely to become a bottleneck than the network interface. Tasks that do not parallelize well are performed on the network interface and hardware support is provided for the most time-critical operations. We emphasize the use of simple I/O mechanisms that can be used by programming tools that map applications on the distributed-memory system to implement efficient I/O for the class of applications they support.This architecture has been implemented for the iWarp distributed-memory system. We describe this implementation and present performance results.

Computation migration

We describe computation migration , a new technique that is based on compile-time program transformations, for accesing remote data in a distributed-memory parallel system. In contrast with RPC-style access, where the access is performed remotely, and with data migration, where the data is moved so that it is local, computation migration moves part of the current thread to the processor where the data resides. The access is performed at the remote processor, and the migrated thread portion continues to run on that same processor; this makes subsequent accesses in the thread portion local. We describe an implementation of computation migration that consists of two parts: an implementation that migrates single activation frames, and a high-level language annotation that allows a programmer to express when migration is desired. We performed experiments using two applications; these experiments demonstrate that computation migration is a valuable alternative to RPC and data migration.

A performance analysis exemplar: Parallel ray tracing

Among the many techniques in computer graphics, ray tracing is prized because it can render realistic images, albeit at great computational expense. Ray tracing's large computation requirements, coupled with its inherently parallel nature, make ray tracing algorithms attractive candidates for parallel implementation. In this paper we illustrate the utility and the importance of a suite of performance analysis tools when exploring the performance of several approaches to ray tracing on a distributed memory parallel system. These ray tracing algorithm variations introduce parallelism based on both ray and object partitions. Traditional timing analysis can quantify the performance effects of different algorithm choices (i.e. when an algorithm is best matched to a given problem), but it cannot identify the causes of these performance differences. We argue, by example, that a performance instrumentation system is needed that can trace the execution of distributed memory parallel programs by recording the occurrence of parallel program events. The resulting event traces can be used to compile summary stapistics that provide a global view of program performance. In addition, visualization tools permit the graphic display of event traces. Visual presentation of performance data is particularly useful, indeed, necessary for large-scale, parallel computations; assimilating the enormous volume of performance data mandates visual display.

Ray tracing on distributed memory parallel systems

Among the many techniques in computer graphics, ray tracing is prized because it can render realistic images, albeit at great computational expense. In this note we explore the performance of several approaches to ray tracing on a distributed memory parallel system. A set of performance instrumentation tools and their associated visualization software are used to identify the underlying causes of performance differences.