Distributed Memory MIMD Architectures Research Articles

A parallel algorithm for large systems of Volterra integral equations of Abel type

A significative number of recent applications require numerical solution of large systems of Abel–Volterra integral equations. Here we propose a parallel algorithm to numerically solve a class of these systems, designed for a distributed-memory MIMD architecture. In order to achieve a good efficiency we employ a fully parallel and fast convergent waveform relaxation (WR) method and evaluate the lag term by using FFT techniques. To accelerate the convergence of the WR method and to best exploit the parallel architecture we develop special strategies. The performances of the resulting code, NSWR4, are illustrated on some examples.

Motion-compensated wavelet packet zerotree video coding on multicomputers

In this work we describe and analyze algorithms for advanced video coding on distributed memory MIMD architectures. In particular, we consider a wavelet packet based codec using the concept of zerotree encoding. The main contribution of this work is the design of a parallel motion-compensated video coder composed of a wavelet packet decomposition in conjunction with the best basis algorithm followed by zerotree coding. Whereas two sensible parallelization techniques can be employed for the wavelet packet decomposition (subband based partitioning and stripe partitioning), the zerotree coding and motion compensation stages only allow one reasonable parallelization method (stripe partitioning). We investigate the advantages and drawbacks of the resulting different overall data distribution strategies and show experimental results obtained on a Siemens hpcLine cluster and a Cray T3E.

Parallel transient dynamic non-linear analysis of reinforced concrete plates

This article describes how parallel processing may improve the computational efficiency of the transient dynamic non-linear analysis of reinforced concrete plates subjected to blast or seismic loading. A parallel scheme for the time marching procedure is presented using the explicit Newmark’s algorithm. The finite element formulation with the material modelling (of a strain-rate sensitive hardening–softening, elasto–viscoplastic model accounting for cracking and crushing) is described. The complexity of the material laws and their modelling, induced by the extreme loading conditions, requires vast computational effort per time step. It will be shown that very high computational efficiency may be obtained by decomposing the finite element mesh into a number of sub-domains for distributed analysis on multiple processors. This high efficiency is achieved using distributed memory MIMD architectures (including transputer networks), where the routing of the messages represents significant overhead in terms of inter-processor communication because of limitations resulting from the processor network topology. Through examples it will be demonstrated how this efficiency depends on the problem size (i.e. the level of refinement of the problem idealisation), the number of sub-domains and the status of the analysis with regards to the states of the material.

Multiple sequential staging of tasks: a new approach to parallel computations

Developing an efficient algorithm for solving a large linear system in a parallel computing environment is the major problem associated with the application of parallel processing to the numerical solution of large-scale engineering problems. This paper presents a new algorithm called Multiple Sequential Staging of Tasks (MSST) to speed up the solution of a large linear system. The technique of Sequential Staging of Tasks (SST) is a highly efficient approach to the parallel solution of a large linear system, but it is not suitable for middle- and large-scale parallel computers due to the idle periods of processors. The MSST technique partitions processors into groups and makes each group start its operation from a different row of a large linear system to remove the idle period. Therefore, MSST can be performed effectively on middle- and large-scale parallel computers and achieves a higher speed-up. Numerical results were obtained from computer experiments performed with a numerical solution method of the Poisson equation on a Dawning-1000 supercomputer (a distributed-memory MIMD architecture). The parallel speed-up is satisfactory. Copyright © 1999 John Wiley & Sons, Ltd.

An Efficient Parallel Algorithm for Computing the Gaussian Convolution of Multi-dimensional Image Data

In this paper, we propose a parallel convolution algorithm for estimating the partial derivatives of 2D and 3D images on distributed-memory MIMD architectures. Exploiting the separable characteristics of the Gaussian filter, the proposed algorithm consists of multiple phases such that each phase corresponds to a separated filter. Furthermore, it exploits both the task and data parallelism, and reduces communication through data redistribution. We have implemented the proposed algorithm on the Intel Paragon and obtained a substantial speedup using more than 100 processors. The performance of the algorithm is also evaluated analytically. The analytical results confirming with the experimental results indicate that the proposed algorithm scales very well with the problem size and number of processors. We have also applied our algorithm to the design and implementation of an efficient parallel scheme for the 3D surface tracking process. Although our focus is on 3D image data, the algorithm is also applicable to 2D image data, and can be useful for a myriad of important applications including medical imaging, magnetic resonance imaging, ultrasonic imagery, scientific visualization, and image sequence analysis.

Parallel Simulated Annealing by Mixing of States

We report the results of testing the performance of a new, efficient, and highly general-purpose parallel optimization method, based upon simulated annealing. This optimization algorithm was applied to analyze the network of interacting genes that control embryonic development and other fundamental biological processes. We found several sets of algorithmic parameters that lead to optimal parallel efficiency for up to 100 processors on distributed-memory MIMD architectures. Our strategy contains two major elements. First, we monitor and pool performance statistics obtained simultaneously on all processors. Second, we mix states at intervals to ensure a Boltzmann distribution of energies. The central scientific issue is the inverse problem, the determination of the parameters of a set of nonlinear ordinary differential equations by minimizing the total error between the model behavior and experimental observations.

Real-Time Automatic Visual Inspection of High-Speed Plane Products by Means of Parallelism

Parallel processing can effectively satisfy the real-time constraints required by on-line machine vision applications. This paper describes a real-time automatic visual inspection (AVI) system for high-speed plane products, which is based on a reconfigurable and scalable coarse-grain distributed memory MIMD architecture, and a unique application programming interface. The code of application algorithms are source level machine independent. As our prototype is organized as a test bed, new algorithms and even new dedicated hardware processing elements can quickly be evaluated on it. The system can easily be tailored for various automatic visual surface inspection applications. An example for inspecting whether separate small mosaics (textures) in a scene are in normal shapes is described. The algorithm combines the connected component labeling, the moment calculation and the pattern recognition. This key task for most applications is well suited for the “divide and conquer” parallel paradigm. The real-time performance has been achieved on a TMS320C40 array.

Parallel image understanding algorithms on MIMD multicomputers

The heterogeneous nature of data types and computational structures involved in Computer Vision algorithms make the design and implementation of massively parallel image processing systems a not yet fully solved problem. It is common belief that in the next future MIMD architectures with their high degree of flexibility will play a very important role in this research area, by using a limited number of identical but powerful processing elements. The aim of this paper is to show how a selected list of algorithms in which a unique Image Understanding process can be decomposed could map onto a distributed-memory MIMD architecture. The operative modalities we adopt are the SPMD modality for the low level processing and the MIMD modality for the intermediate and high levels of processing. Either efficient parallel formulations of the algorithms with respect to the interconnection topology of processors and their optimized implementations on a target transputer-based architecture are reported.

Mapping the synthetic aperture radar signal processor on a distributed-memory MIMD architecture

This paper describes the processing of raw data, gathered with a synthetic aperture radar (SAR), by the parallel computer nCUBE 2 to form an electromagnetic image of the sensed scene. The initial goal of our work has been the evaluation of the use of a Distributed-Memory Multiple-Instruction stream Multiple-Data stream (DM-MIMD) parallel computer for SAR processing, with respect to the use of a conventional monoprocessor RISC workstation. To meet this goal, a study of some algorithms of the radix-2 n class for the one-dimensional (1D) and two-dimensional (2D) FFTs has been conducted. A good algorithm candidate to be parallelized and used as kernel component of our SAR processor has been selected.

Parallel finite element analysis using Jacobi-conditioned conjugate gradient algorithm

In this paper a modified parallel Jacobi-conditioned conjugate gradient (CG) method is proposed for solving linear elastic finite element system of equations. The conventional element-by-element and diagonally conditioned approaches are discussed with respect to parallel implementation on distributed memory MIMD architectures. The effects of communication overheads on the efficiency of the parallel CG solver are considered and it is shown that for the efficient performance of a parallel CG solver, the interprocessor communication has to be carried out concurrently. A concurrent communication scheme is proposed by relating the semi-bandwidth of the stiffness matrix with the number of independent degrees of freedom and the number of processors and inducing directionalization of communication within the processor pipeline. With the aid of two examples the effectiveness of the proposed method is demonstrated showing that the cost of communication remains low and relatively insensitive to the increase in the number of processors.

Distributed Logic Objects: A Fragment of Rewriting Logic and its Implementation

This paper presents a logic language (called Distributed Logic Objects, DLO for short) that supports objects, messages and inheritance. The operational semantics of the language is given in terms of rewriting rules acting upon the (possibly distributed) state of the system. In this sense, the logic underlying the language is Rewriting Logic. In the paper we discuss the implementation of this language on distributed memory MIMD architectures, and we describe the advantages achieved in terms of flexibility, scalability and load balancing. In more detail, the implementation is obtained by translating logic objects into a concurrent logic language based on multi-head clauses, taking advantage from its distributed implementation on a massively parallel architecture. In the underlying implementation, objects are clusters of processes, objects' state is represented by logical variables, message-passing communication between objects is performed via multi-head clauses, and inheritance is mapped into clause union. Some interesting features such as transparent object migration and intensional messages are easily achieved thanks to the underlying support. In the paper, we also sketch a (direct) distributed implementation supporting the indexing of clauses for single-named methods.

Parallelizing Strassen's method for matrix multiplication on distributed-memory MIMD architectures

We present a parallel method for matrix multiplication on distributed-memory MIMD architectures based on Strassen's method. Our timing tests, performed on a 56-node Intel Paragon, demonstrate the realization of the potential of the Strassen's method with a complexity of 4.7 M 2.807 at the system level rather than the node level at which several earlier works have been focused. The parallel efficiency is nearly perfect when the processor number is the power of 7. The parallelized Strassen's method seems always faster than the traditional matrix multiplication methods whose complexity is 2 M 3 coupled with the BMR method and the Ring method at the system level. The speed gain depends on matrix order M: 20% for M ≈ 1000 and more than 100% for M ≈ 5000.

Evaluation of parallelization strategies for an incremental Delaunay triangulator in e3

AbstractThe paper deals with the parallelization of Delaunay triangulation, a widely used space partitioning technique. Two parallel implementations of a three‐dimensional incremental construction algorithm are presented. The first is based on the decomposition of the spatial domain, while the second relies on the master‐slaves approach. Both parallelization strategies are evaluated, stressing practical issues rather than theoretical complexity. We report on the exploitation of two different parallel environments: a tightly coupled distributed memory MIMD architecture and a network of workstations co‐operating under the Linda environment Then, a third hybrid solution is proposed, specifically addressed to the exploitation of higher parallelism. It combines the other two solutions by grouping the processing nodes of the multicomputer into clusters and by exploiting parallelism at two different levels.

Parallel rendering of volumetric data set on distributed‐memory architectures

AbstractA solution is proposed to the problem of interactive visualization and rendering of volume data. Designed for parallel distributed memory MIMD architectures, the volume rendering system is based on the ray tracing (RT) visualization technique, the Sticks representation scheme (a data structure exploiting data coherence for the compression of classified data sets), the use of a slice‐partitioning technique for the distribution of the data between the processing nodes and the consequent ray‐data‐flow parallelizing strategy.The system has been implemented on two different architectures: an inmos Transputer network and a hypercube nCUBE 6400 architecture. The high number of processors of this latter machine has allowed us to exploit a second level of parallelism (parallelism on image space, or parallelism on pixels) in order to arrive at a higher degree of scalability. In both proposals, the similarities between the chosen data‐partitioning strategy, the communications pattern of the visualization processes and the topology of the physical system architecture represent the key points and provide improved software design and efficiency. Moreover, the partitioning strategy used and the network interconnection topology reduce the communications overhead and allow for an efficient implementation of a static load‐balancing technique based on the prerendering of a low resolution image.Details of the practical issues involved in the parallelization process of volumetric RT, commonly encountered problems (i.e. termination and deadlock prevention) and the sw migration process between different architectures are discussed.

An efficient distributed thinning algorithm

We present a detailed solution for the problem of computing a skeleton of a digital image by a distributed system. The proposed algorithm uses domain decomposition of the image to be thinned rather than computing one pixel per processor. We will show that the rectangular tesselation with 4-subfields technique is especially appropriate for the distributed thinning. The theoretical model as well as a distributed implementation on a distributed memory MIMD architecture are presented.