Distributed Memory Multicomputers Research Articles

An Efficient Parallel Algorithm to Solve Block?Toeplitz Systems

In this paper, we present an efficient parallel algorithm to solve Toeplitz–block and block–Toeplitz systems in distributed memory multicomputers. This algorithm parallelizes the Generalized Schur Algorithm to obtain the semi-normal equations. Our parallel implementation reduces the communication cost and optimizes the memory access. The experimental analysis on a cluster of personal computers shows the scalability of the implementation. The algorithm is portable because it is based on standard tools and libraries, such as ScaLAPACK and MPI.

A Compressed Diagonals Remapping Technique for Dynamic Data Redistribution on Banded Sparse Matrix

In many scientific applications, dynamic data redistribution is used to enhance algorithm performance and achieve data locality in parallel programs on distributed memory multi-computers. In this paper, we present a new method, Compressed Diagonals Remapping (CDR) technique aims to the efficiency of runtime data redistribution on banded sparse matrices. The main idea of the proposed technique is first to compress the source matrix into a Compressed Diagonal Matrix (CDM) form. Based on the compressed diagonal matrix, a one-dimensional local and global index transformation method can be applied to carry out data redistribution on the compressed diagonal matrix. This process is identical to redistribute data in the two-dimensional banded sparse matrix. The CDR technique uses an efficient one-dimensional indexing scheme to perform data redistribution on banded sparse matrix. A significant improvement of this approach is that a processor does not need to determine the complicated sending or receiving data sets for dynamic data redistribution. The indexing cost is reduced significantly. The second advantage of the present techniques is the achievement of optimal packing/unpacking stages consequent upon the consecutive attribute of column elements in a compressed diagonal matrix. Another contribution of our methods is the ability to handle sparse matrix redistribution under two disjoint processor grids in the source and destination phases. A theoretical model to analyze the performance of the proposed technique is also presented in this paper. To evaluate the performance of our methods, we have implemented the present techniques on an IBM SP2 parallel machine along with the v2m algorithm and a dense redistribution strategy. The experimental results show that our technique provides significant improvement for runtime data redistribution of banded sparse matrices in most test samples.

Efficient data parallel algorithms for multidimensional array operations based on the ekmr scheme for distributed memory multicomputers

Array operations are useful in a large number of important scientific codes, such as molecular dynamics, finite element methods, climate modeling, atmosphere and ocean sciences, etc. In our previous work, we have proposed a scheme of extended Karnaugh map representation (EKMR) for multidimensional array representation. We have shown that sequential multidimensional array operation algorithms based on the EKMR scheme have better performance than those based on the traditional matrix representation (TMR) scheme. Since parallel multidimensional array operations have been an extensively investigated problem, we present efficient data parallel algorithms for multidimensional array operations based on the EKMR scheme for distributed memory multicomputers. In a data parallel programming paradigm, in general, we distribute array elements to processors based on various distribution schemes, do local computation in each processor, and collect computation results from each processor. Based on the row, column, and 2D mesh distribution schemes, we design data parallel algorithms for matrix-matrix addition and matrix-matrix multiplication array operations in both TMR and EKMR schemes for multidimensional arrays. We also design data parallel algorithms for six Fortran 90 array intrinsic functions: All, Maxval, Merge, Pack, Sum, and Cshift. We compare the time of the data distribution, the local computation, and the result collection phases of these array operations based on the TMR and the EKMR schemes. The experimental results show that algorithms based on the EKMR scheme outperform those based on the TMR scheme for all test cases.

A parallel 3D semiconductor device simulator for gradual heterojunction bipolar transistors

AbstractIn this paper, we present a parallel three‐dimensional semiconductor device simulator for gradual heterojunction bipolar transistor. This simulator uses the drift‐diffusion transport model. The Poisson equation and continuity equations were discretized using a finite element method (FEM) on an unstructured tetrahedral mesh. Fermi–Dirac statistics is considered in our model and a compact formulation is used that makes it easy to take into account other effects such as the non‐parabolic nature of the bands or the presence of various subbands in the conduction process. Domain decomposition methods were tested to solve the linear systems. We have applied this simulator to a gradual heterojunction bipolar transistor (HBT), and we present some measures of the parallel execution time for several solvers and some electrical results. This code has been implemented for distributed memory multicomputers, making use of the MPI message passing standard library and a parallel solver library. Copyright © 2002 John Wiley & Sons, Ltd.

Skewed Data Partition and Alignment Techniques for Compiling Programs on Distributed Memory Multicomputers

Minimizing data communication over processors is the key to compile programs for distributed memory multicomputers. In this paper, we propose new data partition and alignment techniques for partitioning and aligning data arrays with a program in a way of minimizing communication over processors. We use skewed alignment instead of the dimension-ordered alignment techniques to align data arrays. By developing the skewed scheme, we can solve more complex programs with minimized data communication than that of the dimension-ordered scheme. Finally, we compare the proposed scheme with the dimension-ordered alignment one by experimental results. The experimental results show that our proposed scheme has more opportunities to align data arrays such that data communications over processors can be minimized.

Parallel Shear-Warp Factorization Volume Rendering Using Efficient 1-D and 2-D Partitioning Schemes for Distributed Memory Multicomputers

3-D data visualization is very useful for medical imaging and computational fluid dynamics. Volume rendering can be used to exhibit the shape and volumetric properties of 3-D objects. However, volume rendering requires a considerable amount of time to process the large volume of data. To deliver the necessary rendering rates, parallel hardware architectures such as distributed memory multicomputers offer viable solutions. The challenge is to design efficient parallel algorithms that utilize the hardware parallelism effectively. In this paper, we present two efficient parallel volume rendering algorithms, the 1D-partition and 2D-partition methods, based on the shear-warp factorization for distributed memory multicomputers. The 1D-partition method has a performance bound on the size of the volume data. If the number of processors is less than a threshold, the 1D-partition method can deliver a good rendering rate. If the number of processors is over a threshold, the 2D-partition method can be used. To evaluate the performance of these two algorithms, we implemented the proposed methods along with the slice data partitioning, volume data partitioning, and sheared volume data partitioning methods on an IBM SP2 parallel machine. Six volume data sets were used as the test samples. The experimental results show that the proposed methods outperform other compatible algorithms for all test samples. When the number of processors is over a threshold, the experimental results also demonstrate that the 2D-partition method is better than the 1D-partition method.

NUMERICAL SOLUTION OF DISCRETE STABLE LINEAR MATRIX EQUATIONS ON MULTICOMPUTERS

We investigate the parallel performance of numerical algorithms for solving discrete Sylvester and Stein equations as they appear for instance in discrete-time control problems, filtering, and image restoration. The methods used here are the squared Smith iteration and the sign function method on a Cayley transformation of the original equation. For Stein equations with semidefinite right-hand side these methods are modified such that the Cholesky factor of the solution can be computed directly without forming the solution matrix explicitly. We report experimental results of these algorithms on distributed-memory multicomputers

A Parallel Implementation of a Fast Multipole-Based 3-D Capacitance Extraction Program on Distributed Memory Multicomputers

Very fast and accurate 3-D capacitance extraction is essential for interconnect optimization in VLSI ultra-deep sub-micron designs (UDSM). Parallel processing provides an approach to reducing the simulation turn-around time. This paper examines the parallelization of the well-known fast multipole-based 3-D capacitance extraction program FASTCAP, which employs new adaptive and preconditioning techniques. To account for the complicated data dependencies in the unstructured problems, we propose a novel generalized cost function model, which can be used to accurately measure the workload associated with each cube in the hierarchy. We then present two adaptive partitioning schemes, combined with efficient communication mechanisms with bounded buffer size, to reduce the parallel processing overhead. The overall load balance is achieved through balancing the load at each level of the multipole computation. We report detailed performance results on a variety of distributed memory parallel platforms, using standard benchmarks on 3-D capacitance extraction.

A generalized processor mapping technique for array redistribution

In many scientific applications, array redistribution is usually required to enhance data locality and reduce remote memory access in many parallel programs on distributed memory multicomputers. Since the redistribution is performed at runtime, there is a performance trade-off between the efficiency of the new data decomposition for a subsequent phase of an algorithm and the cost of redistributing data among processors. In this paper, we present a generalized processor mapping technique to minimize the amount of data exchange for BLOCK-CYCLIC(kr) to BLOCK-CYCLIC(r) array redistribution and vice versa. The main idea of the generalized processor mapping technique is first to develop mapping functions for computing a new rank of each destination processor. Based on the mapping functions, a new logical sequence of destination processors can be derived. The new logical processor sequence is then used to minimize the amount of data exchange in a redistribution. The generalized processor mapping technique can handle array redistribution with arbitrary source and destination processor sets and can be applied to multidimensional array redistribution. We present a theoretical model to analyze the performance improvement of the generalized processor mapping technique. To evaluate the performance of the proposed technique, we have implemented the generalized processor mapping technique on an IBM SP2 parallel machine. The experimental results show that the generalized processor mapping technique can provide performance improvement over a wide range of redistribution problems.

An efficient adaptive scheduling scheme for distributed memory multicomputers

Traditional multiprocessor scheduling schemes have been one of either space-sharing or time-sharing. Space-sharing schemes perform better than time-sharing at low to moderate system loads. However, they have a disadvantage of wasting processing power within partitions at medium to high system loads. Time sharing schemes tend to perform better at medium to high system loads. Almost all the scheduling schemes proposed so far have been tested under ad hoc workload considerations. In light of recent knowledge about workloads, it is imperative to develop an integrated scheduling scheme that combines the advantages of space- and time-sharing while overcoming their individual drawbacks. We propose such a scheduling scheme, called Hierarchical Scheduling Policy, which is efficient as well as general enough to accommodate multiple workloads. Simulation results indicate that our scheme significantly outperforms the best space- and time-sharing mechanisms at medium to high system loads even in the absence knowledge regarding individual job characteristics.

A binomial tree based parallel load‐balancing method for solution‐adaptive finite element graphs on distributed memory multicomputers

In this paper, we propose a binomial tree based parallel load‐balancing method (BINOTPLB) to deal with the load imbalance of solution‐adaptive finite element application programs on distributed memory multicomputers. The main idea of the BINOTPLB method is first to construct a binomial tree based condensed processor graph. Based on the condensed processor graph, a prefix code tree is built. From the prefix code tree, a schedule for performing load transfer among processors can be determined by concurrently and recursively dividing the prefix code tree into two subtrees and finding a maximum matching for processors in the two subtrees until the leaves are reached. Since each leaf is a binomial tree and a binomial tree can also be divided into two equal halves of binomial trees, the approach used to determine the schedule of a prefix code tree could also be applied to the binomial trees. We have implemented the BINOTPLB method on an SP2 parallel machine and compared its performance with two load‐balancing methods, the directed diffusion method and the multilevel diffusion method, and three mapping methods, the JOSTLE‐MS method, the MLkP method, and the PARTY library method. Three criteria, the execution time of mapping/load‐balancing methods, the execution time of an application program under different mapping/load‐balancing methods, and the speedups achieved by mapping/load‐balancing methods for an application program, are used for the performance evaluation. The experimental results show that the BINOTPLB method outperforms other methods for most of test samples.

Efficient Compositing Methods for the Sort-Last-Sparse Parallel Volume Rendering System on Distributed Memory Multicomputers

In the sort-last-sparse parallel volume rendering system on distributed memory multicomputers, one can achieve a very good performance improvement in the rendering phase by increasing the number of...

A parallel iterative linear system solver with dynamic load balancing

This paper describes the design and implementation of a parallel iterative linear system solver for distributed memory multicomputers and workstation clusters. It is capable of applying heterogeneous data distribution and dynamic load balancing within an iterative solver routine at matrix level. Matrices as well as vectors are distributed heterogeneously according to the available performances of the processors, and redistributions are carried out at run time if the load of the processors changes. We present the concepts behind the chosen matrix data structures and load measurements, and discuss our dynamic load balancing algorithm. The results show the suitability of our approach.

Parallel finite element method to solve the 3D Poisson equation and its application to abrupt heterojunction bipolar transistors

In this work we present a parallel solver for the Poisson equation for 3D abrupt heterojunction bipolar transistors (HBT). Three-dimensional simulation is essential for studying devices of small geometry as in the case we have studied. We have used an unstructured tetrahedral mesh and we have applied the finite method element (FEM), making a specific formulation for the nodes located on the interface of the regions with different characteristics. For HBT devices, it is necessary to take into account that on both sides of the interface between the different regions exist materials with different properties. Our formulation implies situating pairs of nodes in the same physical positions of the interface, associating each nodes to a region of the HBT. This way, the effects due to thermionic emission and the tunnel effect may be simulated when the Poisson and the electron and hole equations are solved in an abrupt HBT. We have applied domain decomposition methods to solve the associate linear systems. This code has been implemented for distributed memory multicomputers, making use of a message passing standard library, MPI. Copyright © 2000 John Wiley & Sons, Ltd.

A Dynamic Diffusion Optimization Method for Irregular Finite Element Graph Partitioning

To efficiently execute a finite element application program on a distributed memory multicomputer, we need to distribute nodes of a finite element graph to processors of a distributed memory multicomputer as evenly as possible and minimize the communication cost of processors. This partitioning problem is known to be NP-complete. Therefore, many heuristics have been proposed to find satisfactory sub-optimal solutions. Based on these heuristics, many graph partitioners have been developed. Among them, Jostle, Metis, and Party are considered as the best graph partitioners available up-to-date. For these three graph partitioners, in order to minimize the total cut-edges, in general, they allow 3% to 5% load imbalance among processors. This is a tradeoff between the communication cost and the computation cost of the partitioning problem. In this paper, we propose an optimization method, the dynamic diffusion method (DDM), to balance the 3% to 5% load imbalance allowed by these three graph partitioners while minimizing the total cut-edges among partitioned modules. To evaluate the proposed method, we compare the performance of the dynamic diffusion method with the directed diffusion method and the multilevel diffusion method on an IBM SP2 parallel machine. Three 2D and two 3D irregular finite element graphs are used as test samples. For each test sample, 3% and 5% load imbalance situations are tested. From the experimental results, we have the following conclusions. (1) The dynamic diffusion method can improve the partition results of these three partitioners in terms of the total cut-edges and the execution time of a Laplace solver in most test cases while the directed diffusion method and the multilevel diffusion method may fail in many cases. (2) The optimization results of the dynamic diffusion method are better than those of the directed diffusion method and the multilevel diffusion method in terms of the total cut-edges and the execution time of a Laplace solver for most test cases. (3) The dynamic diffusion method can balance the load of processors for all test cases.

A Prefix Code Matching Parallel Load-Balancing Method for Solution-Adaptive Unstructured Finite Element Graphs on Distributed Memory Multicomputers

In this paper, we propose a prefix code matching parallel load-balancing method (PCMPLB) to efficiently deal with the load imbalance of solution-adaptive finite element application programs on distributed memory multicomputers. The main idea of the PCMPLB method is first to construct a prefix code tree for processors. Based on the prefix code tree, a schedule for performing load transfer among processors can be determined by concurrently and recursively dividing the tree into two subtrees and finding a maximum matching for processors in the two subtrees until the leaves of the prefix code tree are reached. We have implemented the PCMPLB method on an SP2 parallel machine and compared its performance with two load-balancing methods, the directed diffusion method and the multilevel diffusion method, and five mapping methods, the AE/ORB method, the AE/MC method, the MLkP method, the PARTY library method, and the JOSTLE-MS method. An unstructured finite element graph Truss was used as a test sample. During the execution, Truss was refined five times. Three criteria, the execution time of mapping/load-balancing methods, the execution time of an application program under different mapping/load-balancing methods, and the speedups achieved by mapping/load-balancing methods for an application program, are used for the performance evaluation. The experimental results show that (1) if a mapping method is used for the initial partitioning and this mapping method or a load-balancing method is used in each refinement, the execution time of an application program under a load-balancing method is less than that of the mapping method. (2) The execution time of an application program under the PCMPLB method is less than that of the directed diffusion method and the multilevel diffusion method.

Efficient Methods for Multi-Dimensional Array Redistribution

In many scientific applications, array redistribution is usually required to enhance data locality and reduce remote memory access on distributed memory multicomputers. Since the redistribution is performed at run-time, there is a performance tradeoff between the efficiency of the new data decomposition for a subsequent phase of an algorithm and the cost of redistributing data among processors. In this paper, we present efficient methods for multi-dimensional array redistribution. Based on the previous work, the basic-cycle calculation technique, we present a basic-block calculation (BBC) and a complete-dimension calculation (CDC) techniques. We also developed a theoretical model to analyze the computation costs of these two techniques. The theoretical model shows that the BBC method has smaller indexing costs and performs well for the redistribution with small array size. The CDC method has smaller packing/unpacking costs and performs well when array size is large. When implemented these two techniques on an IBM SP2 parallel machine along with the PITFALLS method and the Prylli's method, the experimental results show that the BBC method has the smallest execution time of these four algorithms when the array size is small. The CDC method has the smallest execution time of these four algorithms when the array size is large.

Parallel VLSI test in a shared-memory multiprocessor

This paper presents three parallel procedures implemented in a shared-memory multiprocessor to generate the patterns that allow the testing of digital circuits. The implementation of these procedures in a multiprocessor uses the system memory better than in a distributed-memory multicomputer, since it is not necessary to store the circuit structure in the local memory of each processor, besides other common structures. The parallel test generation procedures are based on a new sequential algorithm which mixes both the Boolean difference and digital spectral techniques. It is thus different from other methods proposed that deal with the parallelization of test generation algorithms that carry out an implicit enumeration of the input pattern space. The first procedure distributes the set of faults using a backtracking procedure starting from a primary output and allocating a similar number of lines to each processor. The second procedure distributes the set of faults among the processors taking into account the distance from each line to its nearest primary output; it then applies the algorithm to generate the test pattern with some modifications. The third procedure uses a circuit partitioning procedure which allows similar sized parts of the circuit to be assigned to each processor while communications between processors are minimized. The experimental results obtained when the procedures are applied to the usual benchmark circuits (the ISCAS set) show figures for speedup better than in a multicomputer, although fewer processors are used. Copyright © 2000 John Wiley & Sons, Ltd.

Compiler and run-time support for exploiting regularity within irregular applications

This paper starts from a well-known idea, that structure in irregular problems improves sequential performance, and tries to show that the same structure can also be exploited for parallelization of irregular problems on a distributed-memory multicomputer. In particular, we extend a well-known parallelization technique called run-time compilation to use structure information that is explicit on the array subscripts. This paper presents a number of internal representations suited to particular access patterns and shows how various preprocessing structures such as translation tables, trace arrays, and interprocessor communication schedules can be encoded in terms of one or more of these representations. We show how loop and index normalization are important for detection of irregularity in array references, as well as the presence of locality in such references. This paper presents methods for detection of irregularity, feasibility of inspection, and finally, placement of inspectors and interprocessor communication schedules. We show that this process can be automated through extensions to an HPF/Fortran-77 distributed-memory compiler (PARADIGM) and a new runtime support for irregular problems (PILAR) that uses a variety of internal representations of communication patterns. We devise performance measures which consider the relationship between the inspection cost, the execution cost, and the number of times the executor is invoked so that a comparison of the competing schemes can be performed independent of the number of iterations. Finally, we show experimental results on an IBM SP-2 that validate our approach. These results show that dramatic improvements in both memory requirements and execution time can be achieved by using these techniques.

Performance of Adaptive Space Sharing Processor Allocation Policies for Distributed-Memory Multicomputers

Several space sharing policies have been proposed for distributed-memory multicomputer systems. We consider adaptive space sharing policies, as these policies provide a better performance than fixed and static policies by taking system load and user requirements into account. In this paper we propose an improved space sharing policy by suggesting a simple modification to a previously proposed policy. We study performance sensitivity of the original and modified policies to job structure and various other system and workload parameters like variances in inter-arrival times and job service times. The results presented here demonstrate that the modified policy performs substantially better than the original policy.