Memory Multiple Instruction Multiple Data Research Articles

A comparative workload-based methodology for performance evaluation of parallel computers

A practical methodology for evaluating and comparing the performance of distributed memory Multiple Instruction Multiple Data (MIMD) systems is presented. The methodology determines machine parameters and program parameters separately, and predicts the performance of a given workload on the machines under consideration. Machine parameters are measured using benchmarks that consist of parallel algorithm structures. The methodology takes a workload-based approach in which a mix of application programs constitutes the workload. Performance of different systems are compared, under the given workload, using the ratio of their speeds. In order to validate the methodology, an example workload has been constructed and the time estimates have been compared with the actual runs, yielding good predicted values. Variations in the workload are analysed in terms of increase in problem sizes and changes in the frequency of particular algorithm groups. Utilization and scalability are used to compare the systems when the number of processors is increased. It has been shown that performance of parallel computers is sensitive to the changes in the workload and therefore any evaluation and comparison must consider a given user workload. Performance improvement that can be obtained by increasing the size of a distributed memory MIMD system depends on the characteristics of the workload as well as the parameters that characterize the communication speed of the parallel system.

Parallel computation in atmospheric chemical modeling

The porting of atmospheric chemical dynamic models to massively parallel computers presents interesting computational challenges. Strategies for the parallelization of the transport and chemistry operators of atmospheric models are outlined. The use of parallel buffers to perform input/output operations is described. Results are given for implementation of the CIT urban air pollution model on distributed memory multiple instruction/multiple data (MIMD) machines ranging from a cluster of workstations to a 512 node Intel Paragon. A speed-up factor of 94.9 is achieved when the I/O, transport, and chemistry portions of the model are performed in parallel using 256 nodes of the Intel Paragon.

Performance Issues for the Iterative Solution of Markov Decision Processes on Parallel Computers

This paper analyses the implementation of an iterative solution method for Markov decision processes on distributed memory multiple instruction multiple data (MIMD) parallel processors. To preserve the good convergence properties of this method a parallel algorithm must be synchronous and the aim of this paper is to understand the factors which influence the efficiency of synchronous parallel algorithms for iterative methods. Models are developed for processor communication time, processor calculation time and overall run time which are also appropriate for other iterative methods. Such iterative methods are used in many other problem areas, including dynamic programming and the solution of linear and differential equations. The timing models guide the development of a phased pipeline algorithm. For 60,000 state sparse Markov decision processes using 121 processors, this algorithm gives 60-fold speed-ups relative to the best sparse serial algorithm. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

Boundary element analysis on vector and parallel computers

Boundary element analysis (BEA) can be characterized as a numerical technique that generally shifts the computational burden in the analysis toward numerical integration and the solution of nonsymmetric and either dense or blocked sparse systems of algebraic equations. Researchers have explored the concept that the fundamental characteristics of BEA can be exploited to generate effective implementations on vector and parallel computers. In this paper, the results of some of these investigations are discussed. The performance of overall algorithms for BEA on vector supercomputers, massively data parallel single instruction multiple data (SIMD), and relatively fine grained distributed memory multiple instruction multiple data (MIMD) computer systems is described. Some general trends and conclusions are discussed, along with indications of future developments that may prove fruitful in this regard.

Air quality modeling on massively parallel computers

The use of massively parallel computers provides an avenue to overcome the computational requirements of air quality modeling. General considerations on parallel implementation of air quality models are outlined including domain decomposition. The implementation of the CIT urban photochemical model on the Intel Touchstone Delta, a distributed memory multiple instruction/multiple data (MIMD) machine is described. When both the transport and chemistry portions of the model are parallelized, a speed-up of about 30 is achieved using 256 processors.

Parallel dynamic load-balancing algorithm for three-dimensional adaptive unstructured grids

Adaptive local grid refinement/coarsening results in unequal distribution of work load among the processors of a parallel system. A novel method for balancing the load in cases of dynamically changing tetrahedral grids is developed. The approach employs local exchange of cells among processors to redistribute the load equally. An important part of the load-balancing algorithm is the method employed by a processor to determine which cells within its subdomain are to be exchanged. Two such methods are presented and compared. The strategy for load balancing is based on the divide-and-conquer approach that leads to an efficient parallel algorithm. This method is implemented on a distributed-memory multiple instruction multiple data system. I. Introduction C OMPUTATIONAL fluid dynamics (CFD) has advanced rapidly over the last two decades, and it is recognized as a valuable tool for engineering design. However, numerical simulation of three-dimensi onal flowfields remains very expensive even with the use of current vector supercomputers. Vector computers have accelerated computations only by one or two orders of magnitude compared with scalar machines. An evolving approach in computer architectures is the design of scalable massively parallel computers, wherein a number of processors are involved in executing different portions of the job. Parallel computing appears to be a promising approach for future design applications of CFD. State-of-the-art parallel architectures can be broadly classified into single instruction multiple data (SIMD), shared memory multiple instruction multiple data (MIMD), and partitioned memory MIMD architectures. Shared memory MIMD architectures such as the Cray Y-MP are extensions of pipelined vector processors with additional facilities to enable effective utilization of the available multiple processors. However, the scalability of such architectures is severely limited due to the inherent bottleneck of the common, shared memory. On the other hand, SIMD architectures such as the CM-2 are based on the lockstep paradigm of parallel computation wherein a large number of processors execute the same instructions on local data. Although this enhances scalability, the overhead associated with communication among these processors can sometimes prove to be a major bottleneck.1 Partitioned memory MIMD architectures provide a good compromise between the other two types of systems. The user has the flexibility to allocate data as well as tasks to each individual processor, thereby enabling fine tuning of the application to the underlying architecture. There is a price to be paid, however, in terms of additional user responsibility to coordinate the processors by exchanging the relevant information via message passing. In the case of an adaptive grid algo

A Parallel Algorithm for Reducing Symmetric Banded Matrices to Tridiagonal Form

An algorithm is presented for reducing symmetric banded matrices to tridiagonal form via Householder transformations. The algorithm is numerically stable and is well suited to parallel execution on distributed memory multiple instruction multiple data (MIMD) computers. Numerical experiments on the iPSC/860 hypercube show that the new method yields nearly full speedup if it is run on multiple processors. In addition, even on a single processor the new method usually will be several times faster than the corresponding EISPACK and LAPACK routines.

Impact of mapping and sparsity on parallelized finite element method modules

The efficient parallelization of a flexible h-version finite element method (FEM) algorithm is investigated. The actual implementation and benchmarking are performed on a distributed memory-multiple instruction multiple data (MIMD) machine (NCUBE). After an initial modularization and benchmarking, three specific parameters that could influence the efficiency of a concurrent finite element analysis (FEA) solver are identified: (i) sparsity of matrices, (ii) storage format for stiffness matrix, and (iii) mapping of domain onto processors. These factors are interdependent and they heavily influence the overall efficiency of the implemented FE code. This study includes a performance evaluation of the FE modules using the techniques outlined below. The results demonstrate the importance of these parameters, and the advantages of exploiting different levels of concurrency in a parallel FEA code. Three direct methods of solving systems of linear equations are analyzed, a Block Cholesky method exhibiting a coarse grain concurrency, and Gauss Elimination and Cholesky Decomposition methods with inherit medium grain parallelism. Additionally, work on two Assembly/Storage Modules (One-Way Dissection and Shifted Banded), and their corresponding simple Domain Decomposers (Vertical Strip and Wrap Mapping) is performed.