IBM SP2 Research Articles

A two-level parallelization strategy for Genetic Algorithms applied to optimum shape design

This paper presents a two-level strategy for the parallelization of a Genetic Algorithm (GA) coupled to a compressible flow solver designed on unstructured triangular meshes. The parallel implementation is based on MPI and makes use of the process group features of this environment. The resulting algorithm is used for the optimum shape design of aerodynamic configurations. Numerical and performance results are presented for the optimization of two-dimensional airfoils for calculations performed on the following systems: an SGI Origin 2000 and an IBM SP-2 MIMD systems; an Pentium Pro (P6/200 MHz) cluster where the interconnection is realized through a FastEthernet (100 Mbits/s) switch.

Optimizing irregular HPF applications using halos

This paper presents extensions of High Performance Fortran (HPF) for specifying non-local access patterns of distributed arrays, called halos, and for controlling communication associated with these non-local accesses. Using these features, crucial optimization techniques required for an efficient parallelization of irregular applications may be applied. The information provided by halos is utilized by the compiler and the runtime system for optimizing the management of distributed arrays and the computation of communication schedules. High-level communication primitives for halos enable the programmer to avoid redundant communication, to reuse communication schedules, and to hide communication overheads by overlapping communication with computation. Performance results of a kernel from a crash simulation application on the NEC Cenju-4, the IBM SP2, and on the NEC SX-4 demonstrate that by using the proposed extensions, a performance close to handwritten message-passing codes can be achieved for irregular problems. Copyright © 2000 John Wiley & Sons, Ltd.

Parallelizing the Dual Simplex Method

We study the parallelization of the steepest-edge version of the dual simplex algorithm. Three different parallel implementations are examined, each of which is derived from the CPLEX dual simplex implementation. One alternative uses PVM, one general-purpose System V shared-memory constructs, and one the PowerC extension of C on a Silicon Graphics multi-processor. These versions were tested on different parallel platforms, including heterogeneous workstation clusters, Silicon Graphics multi-processors, and an IBM SP2. We report on our computational experience.

BSP clusters: High performance, reliable and very low cost

We describe a transport protocol suitable for BSPlib programs running on a cluster of PCs connected by a 100 Mbps Ethernet switch. The protocol provides a reliable packet-delivery mechanism that uses global knowledge of a program's communication pattern to maximise switch performance. The performance is comparable to previous low-latency protocols on similar hardware, but the addition of reliability means that this protocol can be directly used by application software. For a modest budget of $US 20 000 it is possible to build a machine that outperforms an IBM SP2 on all the NAS benchmarks (BT +80%, SP +70%, MG +9%, and LU +65% improvement), and an SGI Origin 2000 on half (BT +10, SP −24%, MG +10%, and LU −28%). The protocol has a CPU overhead of 1.5 μ s for packet download and 3.6 μ s for upload. Small packets can be communicated through the switch in a pipelined fashion every 21 μ s. Application-to-application one-way latency is 29 μ s plus the latency of the switch. A raw link bandwidth of 93 Mbps is achieved for 1400-byte packets, and 50 Mbps for 128-byte packets. This scales to eight processors communicating at 91 Mbps per link, to give a sustained global bandwidth of 728 Mbps.

A Case Study On The Importance Of Compiler And OtherOptimizations For Improving Super-scalar Processor Performance

The importance of properly optimizing code for execution on super-scalar processors was investigated. Access to the domain specialist was not available during the optimization investigation. For this study of an existing serial FORTRAN application, the use of compiler switches, manual coding techniques, a commercial preprocessor utility (KAP), and a commercial parallelization utility (FORGE) showed the potential to affect execution performance by more than an order of magnitude. The application for the case study was a three-dimensional boundary element code that modeled spherical particle transport phenomena in a particle suspension. Separate experiments were conducted using two different processor platforms: a four node IBM SP (160Mhz POWER2 CPU) and a single node DEC Alpha (667Mhz 21164 CPU). Execution times for the non-optimized, serial base case were 72 hours on a single IBM SP node and 66 hours on the DEC Alpha. Using a combination of compiler switches and manual optimizations, such as in-lining of inefficient subroutines, execution times were reduced to 7.5 hours on a single IBM SP node and 5.4 hours on the DEC Alpha. The use of the KAP pre-processor reduced execution time to 2.3 hours on the single IBM SP node. Using the parallelization software FORGE and four nodes on the IBM SP resulted in an execution time of 25.8 hours without compiler optimization and 3.0 hours using compiler switches for optimization. Applications of High Performance Computing in Engineering VI, C.A. Brebbia, M. Ingber & H. Power (Editors) © 2000 WIT Press, www.witpress.com, ISBN 1-85312-810-4

A Parallel Material-point Method With Application To 3DSolid Mechanics

This report describes the parallel implementation of the material-point method (MPM) for solving dynamic, solid mechanics problems in three dimensions. A data decomposition technique is employed for porting the serial version of the algorithm to a parallel platform, with the Message Passing Interface adopted for inter-processor communication. The algorithm has been tested on the IBM SP-2 at the Maui High Performance Computing Center (MHPCC) for some benchmark problems. The sample problems illustrate the capability to solve highspeed impact as well as lower speed manufacturing problems.

High-Performance Radix-2, 3 and 5 Parallel 1-D Complex FFT Algorithms for Distributed-Memory Parallel Computers

In this paper, we propose high-performance radix-2, 3 and 5 parallel 1-D complex FFT algorithms for distributed-memory parallel computers. We use the four-step or six-step FFT algorithms to implement the radix-2, 3 and 5 parallel 1-D complex FFT algorithms. In our parallel FFT algorithms, since we use cyclic distribution, all-to-all communication takes place only once. Moreover, the input data and output data are both in natural order. We also show that the suitability of a parallel FFT algorithm is machine-dependent because of the differences in the architecture of the processor elements in distributed-memory parallel computers. Experimental results of 2^p3^q5^r point FFTs on distributed-memory parallel computers, HITACHI SR2201 and IBM SP2 are reported. We succeeded to get performances of about 130 GFLOPS on a 1024PE HITACHI SR2201 and about 1.25 GFLOPS on a 32PE IBM SP2.

Optimizing irregular HPF applications using halos

This paper presents extensions of High Performance Fortran (HPF) for specifying non-local access patterns of distributed arrays, called halos, and for controlling communication associated with these non-local accesses. Using these features, crucial optimization techniques required for an efficient parallelization of irregular applications may be applied. The information provided by halos is utilized by the compiler and the runtime system for optimizing the management of distributed arrays and the computation of communication schedules. High-level communication primitives for halos enable the programmer to avoid redundant communication, to reuse communication schedules, and to hide communication overheads by overlapping communication with computation. Performance results of a kernel from a crash simulation application on the NEC Cenju-4, the IBM SP2, and on the NEC SX-4 demonstrate that by using the proposed extensions, a performance close to handwritten message-passing codes can be achieved for irregular problems. Copyright © 2000 John Wiley & Sons, Ltd.

Compiler and run-time support for exploiting regularity within irregular applications

This paper starts from a well-known idea, that structure in irregular problems improves sequential performance, and tries to show that the same structure can also be exploited for parallelization of irregular problems on a distributed-memory multicomputer. In particular, we extend a well-known parallelization technique called run-time compilation to use structure information that is explicit on the array subscripts. This paper presents a number of internal representations suited to particular access patterns and shows how various preprocessing structures such as translation tables, trace arrays, and interprocessor communication schedules can be encoded in terms of one or more of these representations. We show how loop and index normalization are important for detection of irregularity in array references, as well as the presence of locality in such references. This paper presents methods for detection of irregularity, feasibility of inspection, and finally, placement of inspectors and interprocessor communication schedules. We show that this process can be automated through extensions to an HPF/Fortran-77 distributed-memory compiler (PARADIGM) and a new runtime support for irregular problems (PILAR) that uses a variety of internal representations of communication patterns. We devise performance measures which consider the relationship between the inspection cost, the execution cost, and the number of times the executor is invoked so that a comparison of the competing schemes can be performed independent of the number of iterations. Finally, we show experimental results on an IBM SP-2 that validate our approach. These results show that dramatic improvements in both memory requirements and execution time can be achieved by using these techniques.

An Interleaving Transformation for Parallelizing Reductions for Distributed-Memory Parallel Machines

Reduction operations frequently appear in algorithms. Due to their mathematical invariance properties (assuming that round-off errorscan be tolerated), it is reasonable to ignore ordering constraints on the computation of reductions in order to take advantage of the computing power of parallel machines. One obvious and widely-used compilation approach for reductions is syntactic pattern recognition. Either the source language includes explicit reduction operators, or certain specific loops are recognized as equivalent to known reductions. Once such patterns are recognized, hand optimized code for the reductions are incorporated in the target program. The advantage of this approach is simplicity. However, it imposes restrictions on the reduction loops—no data dependence other than that caused by the reduction operation itself is allowed in the reduction loops. In this paper, we present a parallelizing technique, interleaving transformation, for distributed-memory parallel machines. This optimization exploits parallelism embodied in reduction loops through combination of data dependence analysis and region analysis. Data dependence analysis identifies the loop structures and the conditions that can trigger this optimization. Region analysis divides the iteration domain into a sequential region and an order-insensitive region. Parallelism is achieved by distributing the iterations in the order-insensitive region among multiple processors. We use a triangular solver as an example to illustrate the optimization. Experimental results on various distributed-memory parallel machines, including the Connection Machines CM-5, the nCUBE, the IBM SP-2, and a network of Sun Workstations are reported.

A distributed memory parallel element-by-element scheme for semiconductor device simulation

A domain decomposition and parallel element-by-element (EBE) scheme is developed for semiconductor device simulation modeled by the drift-diffusion (DD) equations. A classical Gummel iterative decoupling of the potential and carrier transport equations is applied on an unstructured triangulation. The distributed memory EBE scheme is formulated for a Galerkin finite element approximation of the nonlinear Poisson problem, and a modified Scharfetter–Gummel method is used for the carrier transport problem. The resulting sequences of symmetric and nonsymmetric linear systems are solved via preconditioned Krylov methods. Unstructured triangular grids are used to permit grading of the mesh, which is then partitioned to processor subdomains with appropriate data structures for message passing. Details of the parallel algorithm and data structure are provided. The scheme is implemented in Fortran90 with MPI and performance results are presented for a representative MOSFET on an IBM SP, a CRAY T3E, and an SGI/CRAY Origin2000.

Semicoarsening Multigrid on Distributed Memory Machines

This paper presents the results of a scalability study for a three-dimensional semicoarsening multigrid solver on a distributed memory computer. In particular, we are interested in the scalability of the solver---how the solution time varies as both problem size and number of processors are increased. For an iterative linear solver, scalability involves both algorithmic issues and implementation issues. We examine the scalability of the solver theoretically by constructing a simple parallel model and experimentally by results obtained on an IBM SP. The results are compared with those obtained for other solvers on the same computer.

Parallel sparse linear algebra and application to structural mechanics

The framework of this paper is the parallelization of a plasticity algorithm that uses an implicit method and an incremental approach. More precisely, we will focus on some specific parallel sparse linear algebra algorithms which are the most time-consuming steps to solve efficiently such an engineering application. First, we present a general algorithm which computes an efficient static scheduling of block computations for parallel sparse linear factorization. The associated solver, based on a supernodal fan-in approach, is fully driven by this scheduling. Second, we describe a scalable parallel assembly algorithm based on a distribution of elements induced by the previous distribution for the blocks of the sparse matrix. We give an overview of these algorithms and present performance results on an IBM SP2 for a collection of grid and irregular problems.

Interactive Scientific Visualization Using Parallel Redundant Prediction Method.

In this study, an interactive scientific visualization environment was constructed by directly connecting the five-screen immersive projection display CABIN with an IBM SP2 parallel super-computer through the FDDI network. In this environment, numerical simulation can be executed interactively according to the user's motion and the result of the calculation can be visualized in real-time. In particular, a parallel redundand prediction method using the parallel computer was developed to realize a real-time numerical simulation. This method was applied to the deformation analysis of the rubber using the non-linear finite element method, and the interactive visualization according to the user's push action was performed. By using this visualization, the performances of accuracy and calculation time were experimentally evaluated.

A Dynamic Diffusion Optimization Method for Irregular Finite Element Graph Partitioning

To efficiently execute a finite element application program on a distributed memory multicomputer, we need to distribute nodes of a finite element graph to processors of a distributed memory multicomputer as evenly as possible and minimize the communication cost of processors. This partitioning problem is known to be NP-complete. Therefore, many heuristics have been proposed to find satisfactory sub-optimal solutions. Based on these heuristics, many graph partitioners have been developed. Among them, Jostle, Metis, and Party are considered as the best graph partitioners available up-to-date. For these three graph partitioners, in order to minimize the total cut-edges, in general, they allow 3% to 5% load imbalance among processors. This is a tradeoff between the communication cost and the computation cost of the partitioning problem. In this paper, we propose an optimization method, the dynamic diffusion method (DDM), to balance the 3% to 5% load imbalance allowed by these three graph partitioners while minimizing the total cut-edges among partitioned modules. To evaluate the proposed method, we compare the performance of the dynamic diffusion method with the directed diffusion method and the multilevel diffusion method on an IBM SP2 parallel machine. Three 2D and two 3D irregular finite element graphs are used as test samples. For each test sample, 3% and 5% load imbalance situations are tested. From the experimental results, we have the following conclusions. (1) The dynamic diffusion method can improve the partition results of these three partitioners in terms of the total cut-edges and the execution time of a Laplace solver in most test cases while the directed diffusion method and the multilevel diffusion method may fail in many cases. (2) The optimization results of the dynamic diffusion method are better than those of the directed diffusion method and the multilevel diffusion method in terms of the total cut-edges and the execution time of a Laplace solver for most test cases. (3) The dynamic diffusion method can balance the load of processors for all test cases.

ParaPART: parallel mesh partitioning tool for distributed systems

In this paper, we present ParaPART, a parallel version of a mesh partitioning tool, called PART, for distributed systems. PART takes into consideration the heterogeneities in processor performance, network performance and application computational complexities to achieve a balanced estimate of execution time across the processors in the distributed system. Simulated annealing is used in PART to perform the backtracking search for desired partitions. ParaPART significantly improves performance of PART by using the asynchronous multiple Markov chain approach of parallel simulated annealing. ParaPART is used to partition six irregular meshes into 8, 16, and 100 subdomains using up to 64 client processors on an IBM SP2 machine. The results show superlinear speedup in most cases and nearly perfect speedup for the rest. Using the partitions from ParaPART, we ran an explicit, 2-D finite element code on two geographically distributed IBM SP machines. Results indicate that ParaPART produces results consistent with PART. The execution time was reduced by 12% as compared with partitions that consider only processor performance; this is significant given the theoretical upper bound of 15% reduction.

Parallel Krylov Methods for Econometric Model Simulation

This paper investigates parallel solution methods to simulate large-scale macroeconometric models with forward-looking variables. The method chosen is the Newton-Krylov algorithm, and we concentrate on a parallel solution to the sparse linear system arising in the Newton algorithm. We empirically analyze the scalability of the GMRES method, which belongs to the class of so-called Krylov subspace methods. The results obtained using an implementation of the PETSc 2.0 software library on an IBM SP2 show a near linear scalability for the problem tested.

A generalized basic-cycle calculation method for efficient array redistribution

In many scientific applications, dynamic array redistribution is usually required to enhance the performance of an algorithm. In this paper, we present a generalized basic-cycle calculation (GBCC) method to efficiently perform a BLOCK-CYCLIC(s) over P processors to BLOCK-CYCLIC(t) over Q processors array redistribution. In the GBCC method, a processor first computes the source/destination processor/data sets of array elements in the first generalized basic-cycle of the local array it owns. A generalized basic-cycle is defined as lcm(sP, tQ)/(gcd(s,t)/spl times/P) in the source distribution and lcm(sP, tQ)/(gcd(s,t)/spl times/Q) in the destination distribution. From the source/destination processor/data sets of array elements in the first generalized basic-cycle, we can construct packing/unpacking pattern tables to minimize the data-movement operations. Since each generalized basic-cycle has the same communication pattern, based on the packing/unpacking pattern tables, a processor can pack/unpack array elements efficiently. To evaluate the performance of the GBCC method, we have implemented this method on an IBM SP2 parallel machine, along with the PITFALLS method and the ScaLAPACK method. The cost models for these three methods are also presented. The experimental results show that the GBCC method outperforms the PITFALLS method and the ScaLAPACK method for all test samples. A brief description of the extension of the GBCC method to multidimensional array redistributions is also presented.

Efficient Methods for Multi-Dimensional Array Redistribution

In many scientific applications, array redistribution is usually required to enhance data locality and reduce remote memory access on distributed memory multicomputers. Since the redistribution is performed at run-time, there is a performance tradeoff between the efficiency of the new data decomposition for a subsequent phase of an algorithm and the cost of redistributing data among processors. In this paper, we present efficient methods for multi-dimensional array redistribution. Based on the previous work, the basic-cycle calculation technique, we present a basic-block calculation (BBC) and a complete-dimension calculation (CDC) techniques. We also developed a theoretical model to analyze the computation costs of these two techniques. The theoretical model shows that the BBC method has smaller indexing costs and performs well for the redistribution with small array size. The CDC method has smaller packing/unpacking costs and performs well when array size is large. When implemented these two techniques on an IBM SP2 parallel machine along with the PITFALLS method and the Prylli's method, the experimental results show that the BBC method has the smallest execution time of these four algorithms when the array size is small. The CDC method has the smallest execution time of these four algorithms when the array size is large.

Efficient Parallelization of a Three-Dimensional Navier-Stokes Solver on MIMD Multiprocessors

We present an efficient parallelization strategy for speeding up the computation of a high-accuracy 3-dimensional serial Navier-Stokes solver that treats turbulent transonic high-Reynolds flows. The code solves the full compressible Navier-Stokes equations and is applicable to realistic large size aerodynamic configurations and as such requires huge computational resources in terms of computer memory and execution time. The solver can resolve the flow properly on relatively coarse grids. Since the serial code contains a complex infrastructure typical for industrial code (which ensures its flexibility and applicability to complex configurations), then the parallelization task is not straightforward. We get scalable implementation on massively parallel machines by maintaining efficiency at a fixed value by simultaneously increasing the number of processors and the size of the problem. The 3-D Navier-Stokes solver was implemented on three MIMD message-passing multiprocessors (a 64-processors IBM SP2, a 20-processors MOSIX, and a 64-processors Origin 2000). The same code written with PVM and MPI software packages was executed on all the above distinct computational platforms. The examples in the paper demonstrate that we can achieve efficiency of about 60% for as many as 64 processors on Origin 2000 on a full-size 3-D aerodynamic problem which is solved on realistic computational grids.