Message-passing Parallel Computers Research Articles

Architecture and applications for an All-FPGA parallel computer

The Reconfigurable Computing Cluster (RCC) project has been investigating unconventional architectures for high end computing using a cluster of FPGA devices connected by a high-speed, custom network. Most applications use the FPGAs to realize an embedded System-on-a-Chip (SoC) design augmented with application-specific accelerators to form a message-passing parallel computer. Other applications take a single accelerator core and tessellate the core across all of the devices, treating them like a large virtual FPGA. The experimental hardware has also been used for basic computer research by emulating novel architectures. This article discusses the genesis of the over-arching project, summarizes results of individual investigations that have been completed, and how this approach may prove useful in the investigation of future Exascale systems.

ParCYCLIC: finite element modelling of earthquake liquefaction response on parallel computers

AbstractThis paper presents the computational procedures and solution strategy employed in ParCYCLIC, a parallel non‐linear finite element program developed based on an existing serial code CYCLIC for the analysis of cyclic seismically‐induced liquefaction problems. In ParCYCLIC, finite elements are employed within an incremental plasticity, coupled solid–fluid formulation. A constitutive model developed for simulating liquefaction‐induced deformations is a main component of this analysis framework. The elements of the computational strategy, designed for distributed‐memory message‐passing parallel computer systems, include: (a) an automatic domain decomposer to partition the finite element mesh; (b) nodal ordering strategies to minimize storage space for the matrix coefficients; (c) an efficient scheme for the allocation of sparse matrix coefficients among the processors; and (d) a parallel sparse direct solver. Application of ParCYCLIC to simulate 3‐D geotechnical experimental models is demonstrated. The computational results show excellent parallel performance and scalability of ParCYCLIC on parallel computers with a large number of processors. Copyright © 2004 John Wiley & Sons, Ltd.

Parallel finite element modeling of earthquake ground response and liquefaction

Parallel computing is a promising approach to alleviate the computational demand in conducting large-scale finite element analyses. This paper presents a numerical modeling approach for earthquake ground response and liquefaction using the parallel nonlinear finite element program, ParCYCLIC, designed for distributed-memory message-passing parallel computer systems. In ParCYCLIC, finite elements are employed within an incremental plasticity, coupled solid-fluid formulation. A constitutive model calibrated by physical tests represents the salient characteristics of sand liquefaction and associated accumulation of shear deformations. Key elements of the computational strategy employed in ParCYCLIC include the development of a parallel sparse direct solver, the deployment of an automatic domain decomposer, and the use of the Multilevel Nested Dissection algorithm for ordering of the finite element nodes. Simulation results of centrifuge test models using ParCYCLIC are presented. Performance results from grid models and geotechnical simulations show that ParCYCLIC is efficiently scalable to a large number of processors.

High-Speed Parallel Solution of the Neutron Diffusion Equation with the Hierarchical Domain Decomposition Boundary Element Method Incorporating Parallel Communications

A hierarchical domain decomposition boundary element method (HDD-BEM) for solving the multiregion neutron diffusion equation (NDE) has been fully parallelized, both for numerical computations and for data communications, to accomplish a high parallel efficiency on distributed memory message passing parallel computers. Data exchanges between node processors that are repeated during iteration processes of HDD-BEM are implemented, without any intervention of the host processor that was used to supervise parallel processing in the conventional parallelized HDD-BEM (P-HDD-BEM). Thus, the parallel processing can be executed with only cooperative operations of node processors. The communication overhead was even the dominant time consuming part in the conventional P-HDD-BEM, and the parallelization efficiency decreased steeply with the increase of the number of processors. With the parallel data communication, the efficiency is affected only by the number of boundary elements assigned to decomposed subregions, and the communication overhead can be drastically reduced. This feature can be particularly advantageous in the analysis of three-dimensional problems where a large number of processors are required. The proposed P-HDD-BEM offers a promising solution to the deterioration problem of parallel efficiency and opens a new path to parallel computations of NDEs on distributed memory message passing parallel computers.

Flash code: studying astrophysical thermonuclear flashes

The Center for Astrophysical Thermonuclear Flashes is constructing a new generation of codes designed to study runaway thermonuclear burning on the surface or in the interior of evolved compact stars. The center has completed the first version of Flash, Flash-1, which addresses various astrophysics problems. Flash-1 represents a major advance toward a fully flexible code for solving general astrophysical fluid dynamics problems. Flash-1 is modular and adaptive and operates in parallel computing environments. It was designed to let users configure initial and boundary conditions, change algorithms, and add new physical effects with minimal effort. It uses the Paramesh library to manage a block-structured adaptive grid, placing resolution elements only where needed most. It also uses the message passing interface (MPI) library to achieve portability and scalability on a variety of different message passing parallel computers.

High-Speed Parallel Solution of the Neutron Diffusion Equation with the Hierarchical Domain Decomposition Boundary Element Method Incorporating Parallel Communications.

A hierarchical domain decomposition boundary element method (HDD-BEM) for solving the multiregion neutron diffusion equation (NDE) has been fully parallelized, both for numerical computations and for data communications, to accomplish a high parallel efficiency on distributed memory message passing parallel computers. Data exchanges between node processors that are repeated during iteration processes of HDD-BEM are implemented, without any intervention of the host processor that was used to supervise parallel processing in the conventional parallelized HDD-BEM (P-HDD-BEM). Thus, the parallel processing can be executed with only cooperative operations of node processors. The communication overhead was even the dominant time consuming part in the conventional P-HDD-BEM, and the parallelization efficiency decreased steeply with the increase of the number of processors. With the parallel data communication, the efficiency is affected only by the number of boundary elements assigned to decomposed subregions, and the communication overhead can be drastically reduced. This feature can be particularly advantageous in the analysis of three-dimensional problems where a large number of processors are required. The proposed P-HDD-BEM offers a promising solution to the deterioration problem of parallel efficiency and opens a new path to parallel computations of NDEs on distributed memory message passing parallel computers.

Parallel Implementation of a Kalman Filter for Constituent Data Assimilation

Abstract A Kalman filter for the assimilation of long-lived atmospheric chemical constituents was developed for two-dimensional transport models on isentropic surfaces over the globe. Since the Kalman filter calculates the error covariances of the estimated constituent field, there are five dimensions to this problem, x1, x2, and time, where x1 and x2 are the positions of two points on an isentropic surface. Only computers with large memory capacity and high floating point speed can handle problems of this magnitude. This article describes an implementation of the Kalman filter for distributed-memory, message-passing parallel computers. To evolve the forecast error covariance matrix, an operator decomposition and a covariance decomposition were studied. The latter was found to be scalable and has the general property, of considerable practical advantage, that the dynamical model does not need to be parallelized. Tests of the Kalman filter code examined variance transport and observability properties. This...

Electrical defibrillation optimization: an automated, iterative parallel finite-element approach.

To date, optimization of electrode systems for electrical defibrillation has been limited to hand-selected electrode configurations. In this paper we present an automated approach which combines detailed, three-dimensional (3-D) finite-element torso models with optimization techniques to provide a flexible analysis and design tool for electrical defibrillation optimization. Specifically, a parallel direct search (PDS) optimization technique is used with a representative objective function to find an electrode configuration which corresponds to the satisfaction of a postulated defibrillation criterion with a minimum amount of power and a low possibility of myocardium damage. For adequate representation of the thoracic inhomogeneities, 3-D finite-element torso models are used in the objective function computations. The CPU-intensive finite-element calculations required for the objective function evaluation have been implemented on a message-passing parallel computer in order to complete the optimization calculations in a timely manner. To illustrate the optimization procedure, it has been applied to a representative electrode configuration for transmyocardial defibrillation, namely the subcutaneous patch-right ventricular catheter (SP-RVC) system. Sensitivity of the optimal solutions to various tissue conductivities has been studied. Results for the optimization of defibrillation systems are presented which demonstrate the feasibility of the approach.

Performance of a Fully Parallel Sparse Solver

The performance of a fully parallel direct solver for large sparse-symmetric positive definite systems of linear equations is demonstrated. The solver is designed for distributed-memory, message-passing parallel computer systems. All phases of the computation, including sym bolic processing as well as numeric factorization and triangular solution, are performed in parallel. A parallel Cartesian-nested dissection algorithm is used to compute a fill- reducing ordering for the matrix and an appropriate partitioning of the problem across the processors. The separator tree resulting from nested dissection is used to identify and exploit large-grain parallelism in the remaining steps of the computation. The parallel performance of the solver is reported for a series of test problems on the Thinking Machines CM-5 and the Intel Touchstone Delta. The parallel efficiency, scalability, and absolute perfor mance of the solver, as well as the relative importance of the various phases of the computation, are investigated empirically

An implementation framework for HPF distributed arrays on message-passing parallel computer systems

Data parallel languages, like High Performance Fortran (HPF), support the notion of distributed arrays. However, the implementation of such distributed array structures and their access on message passing computers is not straightforward. This holds especially for distributed arrays that are aligned to each other and given a block-cyclic distribution. In this paper, an implementation framework is presented for HPF distributed arrays on message passing computers. Methods are presented for efficient (in space and time) local index enumeration, local storage, and communication. Techniques for local set enumeration provide the basis for constructing local iteration sets and communication sets. It is shown that both local set enumeration and local storage schemes can be derived from the same equation. Local set enumeration and local storage schemes are shown to be orthogonal, i.e., they can be freely combined. Moreover, for linear access sequences generated by our enumeration methods, the local address calculations can be moved out of the enumeration loop, yielding efficient local memory address generation. The local set enumeration methods are implemented by using a relatively simple general transformation rule for absorbing ownership tests. This transformation rule can be repeatedly applied to absorb multiple ownership tests. Performance figures are presented for local iteration overhead, a simple communication pattern, and storage efficiency.

Nonlinear structural analysis on distributed-memory computers

A computational strategy is presented for the nonlinear static and postbuckling analyses of large complex structures on massively parallel computers. The strategy is designed for distributed-memory, message-passing parallel computer systems. The key elements of the proposed strategy are: (a) a multiple-parameter reduced basis technique; (b) a nested dissection (or multilevel substructuring) ordering scheme; (c) parallel assembly of global matrices; and (d) a parallel sparse equation solver. The effectiveness of the strategy is assessed by applying it to thermo-mechanical postbuckling analyses of stiffened composite panels with cutouts, and nonlinear large-deflection analyses of HSCT models on Intel Paragon XP/S computers. The numerical studies presented demonstrate the advantages of nested dissection-based solvers over traditional skyline-based solvers on distributed memory machines.

Numerical analysis of electrical defibrillation: The parallel approach

Numerical modeling offers a viable tool for studying electrical defibrilation, allowing the behavior of field quantities to be observed easily as the different system parameters are varied. One numerical technique, namely the finite-element method, has been found particularly effective for modeling complex thoracic anatomies. However, an accurate finite-element model of the thorax often requires a large number of elements and nodes, leading to a large set of equations that cannot be solved effectively with the computational power of conventional computers. This is especially true if many finite-element solutions need to be achieved within a reasonable time period (eg, electrode configuration optimization). In this study, the use of massively parallel computers to provide the memory and reduction in solution time for solving these large finite-element problems is discussed. Both the uniform and unstructured grid approaches are considered. Algorithms that allow efficient mapping of uniform and unstructured grids to data-parallel and message-passing parallel computers are discussed. An automatic iterative procedure for electrode configuration optimization is presented. The procedure is based on the minimization of an objective function using the parallel direct search technique. Computational performance results are presented together with simulation results.

Postbuckling and large-deflection nonlinear analyses on distributed-memory computers

A computational strategy is presented for postbuckling and nonlinear static analyses of large complex structures on distributed-memory parallel computers. The strategy is designed for message-passing parallel computer systems. The key elements of the proposed strategy are: (a) a multiple-parameter reduced basis technique, (b) a nested dissection (or multilevel substructuring) ordering scheme, (c) parallel assembly of global matrices, and (d) a parallel sparse equation solver. The effectiveness of the strategy is assessed by performing thermomechanical postbuckling analyses of stiffened composite panels with cutouts, and nonlinear large-deflection analyses of High Speed Civil Transport models on three distributed-memory computers. The numerical studies presented demonstrate the advantages of nested dissection-based solvers over traditional skyline-based solvers on distributed-memory machines.

Processor allocation policies for message-passing parallel computers

When multiple jobs compete for processing resources on a parallel computer, the operating system kernel's processor allocation policy determines how many and which processors to allocate to each. In this paper we investigate the issues involved in constructing a processor allocation policy for large scale, message-passing parallel computers supporting a scientific workload. We make four specific contributions: We define the concept of efficiency preservation as a characteristic of processor allocation policies. Efficiency preservation is the degree to which the decisions of the processor allocator degrade the processor efficiencies experienced by individual applications relative to their efficiencies when run alone. We identify the interplay between the kernel processor allocation policy and the application load distribution policy as a determinant of efficiency preservation. We specify the details of two families of processor allocation policies, called Equipartition and Folding. Within each family, different member policies cover a range of efficiency preservation values, from very high to very low. By comparing policies within each family as well as between families, we show that high efficiency preservation is essential to good performance, and that efficiency preservation is a more dominant factor in obtaining good performance than is equality of resource allocation.

Heuristic algorithms for task assignment and scheduling in a processor network

This paper addresses the problem of statically assigning and scheduling parallel executable tasks to processor networks in parallel or distributed computers to meet real-time computing constraints. Three procedures are carried out for the task assignment and scheduling. First, a task assignment algorithm is used to group M tasks into N task clusters and then initially to assign the N clusters onto N processors without considering the communication link constraints of the processors. Then a communication link number reduction algorithm is applied to remove the excess number of links according to the design limitations of the processors. A new structure of the processor network suitable for the N task clusters is then determined. Finally, the N assigned task clusters are scheduled by a task scheduling algorithm in order to minimize the idle time caused by interprocessor synchronized data communication and indirect data communication. The proposed algorithms are based on the family of heuristic approaches, and are particularly suitable for large grain parallel tasks and message passing parallel computers.

Data reduction and fast routing: A strategy for efficient algorithms for message-passing parallel computers

This paper presents several algorithms for solving problems using massively parallel SIMD hypercube and shuffle-exchange computers. The algorithms solve a wide variety of problems, but they are related because they all use a common strategy. Specifically, all of the algorithms use a divide-and-conquer approach to solve a problem withN inputs using a parallel computer withP processors. The structural properties of the problem are exploited to assure that fewer thanN data items are communicated during the division and combination steps of the divide-and-conquer algorithm. This reduction in the amount of data that must be communicated is central to the efficiency of the algorithm. This paper addresses four problems, namely the multiple-prefix, data-dependent parallel-prefix, image-component-labeling, and closest-pair problems. The algorithms presented for the data-dependent parallel-prefix and closest-pair problems are the fastest known whenN ≥P and the algorithms for the multiple-prefix and image-component-labeling problems are the fastest known whenN is sufficiently large with respect toP.

An Iterative Algorithm for Solving the Multidimensional Neutron Diffusion Nodal Method Equations on Parallel Computers

The one-group, steady-state neutron diffusion equation in two-dimensional Cartesian geometry is solved using the nodal integral method. The discrete variable equations comprise loosely coupled sets of equations representing the nodal balance of neutrons, as well as neutron current continuity along rows or columns of computational cells. An iterative algorithm that is more suitable for solving large problems concurrently is derived based on the decomposition of the spatial domain and is accelerated using successive overrelaxation. This algorithm is very well suited for parallel computers, especially since the spatial domain decomposition occurs naturally, so that the number of iterations required for convergence does not depend on the number of processors participating in the calculation. Implementation of our algorithm on the Intel iPSC/2 hypercube and Sequent Balance 8000 parallel computers is presented, and measured speedup and efficiency for test problems are reported. The results suggest that the efficiency of the hypercube quickly deteriorates when many processors are used, while the Sequent Balance retains very high efficiency for a comparable number of participating processors. This leads to the conjecture that message-passing parallel computers are not as well suited for this algorithm as shared-memory machines.

Translating from FP to Occam for systolic algorithms

Programming multiprocessor systems is not a simple task and is often error-prone. In particular, programming a message-passing parallel computer usually demands large amounts of work in development and debugging to avoid deadlock. It is, therefore, essential to have software to support the task of parallel programming, or more desirably to have automatic parallel program generators to greatly reduce the labour. In this paper, we present a novel method of automatically generating Occam programs; or more precisely, we present an automatic FP-to-Occam translator called AFO. Occam is a parallel, message-passing programming language executed on a network of processors called transputers, but is basically a sequential language extended with primitive parallel and communication constructs. In contrast, FP is a functional language that contains no communication details and supports parallel thinking naturally with higher-level parallel constructs. FP has been used to derive many systolic algorithms. Systolic algorithms can be implemented in Occam as soft-systolic algorithms. With this knowledge, we developed AFO to automatically generate Occam programs from FP programs that have corresponding systolic algorithms, in the hope of generating parallel programs without needing to know the details of low-level communications, and thus greatly improving the productivity of parallel programming.

Yet another polynomial preconditioner for the conjugate gradient algorithm

Polynomial preconditioning is a useful tool in the effective use of the conjugate gradient algorithm on special architectures such as message-passing parallel computers, machines with hierarchical memory, vector processors, and machines with very limited memory. In this work we investigate the use of a new adaptive algorithm which uses the polynomial preconditioner based on the residual polynomial from k steps of the conjugate gradient algorithm.

Parallel polynomial arithmetic over finite rings

Convolution algorithms for polynomial multiplication are well known, as is the use of Residue Number Systems and the Chinese Remainder Theorem. This paper discusses how these techniques may be used to perform polynomial arithmetic over very large rings or finite fields. The algorithm is practical and structured so that it can be implemented very efficiently on either a shared memory or a message passing parallel computer. We discuss the mathematical basis of the algorithm and contrast its performance on an Alliant FX/8 and a Symult S2010. These are, respectively, shared memory and message passing parallel computers.