Parallel Solution Methods Research Articles

Parallel Krylov Methods for Econometric Model Simulation

This paper investigates parallel solution methods to simulate large-scale macroeconometric models with forward-looking variables. The method chosen is the Newton-Krylov algorithm, and we concentrate on a parallel solution to the sparse linear system arising in the Newton algorithm. We empirically analyze the scalability of the GMRES method, which belongs to the class of so-called Krylov subspace methods. The results obtained using an implementation of the PETSc 2.0 software library on an IBM SP2 show a near linear scalability for the problem tested.

K-way neural network graph partitioning with separator vertices

A neural network for partitioning graphs into any number of subgraphs using a k-way procedure is presented. The resulting neural network optimises all subgraphs so that they contain approximately the same number of vertices with the exception of a `separator' subgraph. The latter separates all the other subgraphs and is optimised to contain a minimum number of vertices. Expressions for the neuron link weights for such a network are derived analytically, and the recall mechanism of the mean field theorem neural network is used to obtain the graph partitioning. Applications focus on partitioning graphs associated with finite element meshes which, together with parallel domain decomposition solution methods, provide the motivation for this work.

A neural network graph partitioning procedure for grid-based domain decomposition

This paper describes a neural network graph partitioning algorithm which partitions unstructured finite element/volume meshes as a precursor to a parallel domain decomposition solution method. The algorithm works by first constructing a coarse graph approximation using an automatic graph coarsening method. The coarse graph is partitioned and the results are interpolated onto the original graph to initialize an optimization of the graph partition problem. In practice, a hierarchy of (usually more than two) graphs are used to help obtain the final graph partition. A mean field theorem neural network is used to perform all partition optimization. The partitioning method is applied to graphs derived from unstructured finite element meshes and in this context it can be viewed as a multi-grid partitioning method. Copyright © 1999 John Wiley & Sons, Ltd.

Domain decomposition methods for parallel solution of shape sensitivity analysis problems

Domain decomposition methods for parallel solution of shape sensitivity analysis problems

A Variational Framework for Solution Method Developments in Structural Mechanics

We present a variational framework for the development of partitioned solution algorithms in structural mechanics. This framework is obtained by decomposing the discrete virtual work of an assembled structure into that of partitioned substructures in terms of partitioned substructural deformations, substructural rigid-body displacements and interface forces on substructural partition boundaries. New aspects of the formulation are: the explicit use of substructural rigid-body mode amplitudes as independent variables and direct construction of rank-sufficient interface compatibility conditions. The resulting discrete variational functional is shown to be variation-ally complete, thus yielding a full-rank solution matrix. Four specializations of the present framework are discussed. Two of them have been successfully applied to parallel solution methods and to system identification. The potential of the two untested specializations is briefly discussed.

Some P(EC)mE methods for the parallel solution of ODEs

Abstract Stability and efficiency are two important considerations in deriving good numerical methods for ODEs. The underlying challenge is to increase the stability region while maintaining or even improving efficiency. In this paper, we consider this problem within the context of parallel solution methods. A family of methods which we call parallel block predictor-corrector methods is proposed. Members of this family are characterized by iterated corrector evaluations carried out in two adjacent blocks. Stability properties of these methods are analyzed and compared with some existing block-based parallel predictor-corrector methods. Performance of two particular members of the family is compared with existing block-based parallel predictor-corrector methods, using solution formulas which extend over a range of integration orders and which use various numbers of processors. The results obtained from a stability analysis and from a collection of numerical experiments indicate that the proposed methods have the potential to balance stability properties and efficiency in parallel computer systems.

Optimized extrapolation methods for parallel solution of IVPs on different computer architectures

Extrapolation methods are known to be well suited for parallelism, but parallel execution of codes optimized for sequential computation cannot be optimal on parallel machines. Here we present parallel implementations on different architectures of the two most widely used extrapolation methods: The extrapolated mid-point rule for nonstiff systems and the extrapolated semi-implicit Euler method for stiff systems. The architectures include shared memory, distributed and virtually shared memory systems, and the number of processors can be fixed, unlimited, or dynamically limited throughout runtime. A speedup of more than a factor 3 is possible if the algorithms are optimized for the according architectures.

Variable Block CG Algorithms for Solving Large Sparse Symmetric Positive Definite Linear Systems on Parallel Computers, I: General Iterative Scheme

This paper considers a new approach to construction of efficient parallel solution methods of large sparse SPD linear systems. This approach is based on the so-called variable block CG method, a generalization of the standard block CG method, where it is possible to reduce the iteration block size adaptively (at any iteration) by construction of an $A$-orthogonal projector without restarts and without algebraic convergence of residual vectors. It enables one to find the constructive compromise between the required resource of parallelism, the resulting convergence rate, and the serial arithmetic costs of one block iteration to minimize the total parallel solution time. The orthogonality and minimization properties of the variable CG method are established and the theoretical analysis of the convergence rate is presented. The results of numerical experiments with large FE systems coming from $h$- and $p$-approximations of three-dimensional equilibrium equations for linear elastic orthotropic materials show that the convergence rate of the variable block CG method is comparable to that of the standard block CG method even when utilizing a large block size, while the total serial arithmetic costs of the variable block CG method are comparable or even smaller than those of the corresponding point CG method.

SOLUTION OF OPTIMAL CONTROL PROBLEMS ON A PARALLEL MACHINE USING THE EPSILON METHOD

SUMMARYUsing Balakrishnan's epsilon formulation and the Rayleigh‐Ritz method with a Walsh function basis, optimal control problems are transformed from the standard two‐point boundary value problem form to an unconstrained non‐linear programming problem. The resulting matrix‐vector equations describing the optimal solution can be very easily implemented on say an IBM PC running Matlab or MatrixX on the one hand and have standard parallel solution methods for implementation on parallel computers on the other. Some computational results performed on an Intel hypercube are presented to illustrate that considerable computational savings can be realized by using the proposed algorithm.

An efficient multicomputer algorithm for the solution of chemical process flowsheeting equations

This paper presents a parallel solution method of large sparse systems of linear equations arising in the context of a chemical process flowsheeting application on a message passing multicomputer. To maximize the performance, the algorithm uses a novel matrix decomposition and solution method, called parallel two-phased LU decomposition, which schedules the concurrent tasks in a maximally overlapping manner, and at the same time, tries to minimize the interprocessor data dependencies and obtain optimal load balancing. The forward elimination step is performed concurrently with the parallel two-phased LU decomposition step and backward substitution is parallelized in a piecewise manner. Implementation results on an Intel iPSC/2 multicomputer are presented. With 16 processors, speedups of up to 11.3 are observed for relatively large problem sizes and close to 71% processor utilization is achieved despite the high fraction of sequential operations in the process. Although this work was motivated by the need to reduce the computational bottleneck of the DZINE flowsheeting program, it should also be applicable to a wide range of scientific and engineering problems where fast and efficient solution of large sparse systems of linear equations is required.

Fast parallel solution of the Poisson equation on irregular domains

We present a fast parallel solution method for the Poisson equation on irregular domains. Due to a simple embedding method using harmonic polynomial approximation, a dominant part of the computation becomes solving one Poisson problem on a disk.

Stability of the Partitioned Inverse Method for Parallel Solution of Sparse Triangular Systems

Several authors have recently considered a parallel method for solving sparse triangular systems with many right-hand sides. The method employs a partition into sparse factors of the product form of the inverse of the coefficient matrix. It is shown here that while the method can be unstable, stability is guaranteed if a certain scalar that depends on the matrix and the partition is small and that this scalar is small when the matrix is well conditioned. Moreover, when the partition is chosen so that the factors have the same sparsity structure as the coefficient matrix, the backward error matrix can be taken to be sparse.

A direct incomplete factorization method for parallel solution of tridiagonal linear systems

For the direct solution of tridiagonal linear systems Ax = d, the best known serial algorithm is based on LU-factorization of the coefficient matrix A. In the present paper we consider extending the idea to partitioned tridiagonal matrices. Let A be partitioned: A = (A (i, j)) so that the diagonal blocks A( i,i )are tridiagonal. We seek a factorization of A into L = (L (i j) and U = (U (i j) ), partitioned conformally. For the diagonal blocks of A we require the classical factorization: A( ii ) = L (i,i) U (i,i) , L (i,i) unit lower bidiagonal and U (i,i) upper bidiagonal. But, because of the presence of a non-zero element in each of the off-diagonal blocks of A, it is necessary to have Lupper block bidiagonal and U lower block bidiagonal, with only last row of L(i,i+1) and last column of U(i,i-1) filled. To avoid any interlocking/updating during/after the factorization stage, each of these last row and column in each block are required to have their last elements as zeros. On completion of the determinat...

Application of clustering and factorisation tree techniques for parallel solution of sparse network equations

Through the use of diakoptical techniques it is possible to solve the network equations for a power system using a parallel computer architecture. Traditional parallel solution techniques give an unimpressive return on investment as more processors are applied to the problem owing to unacceptably large sequential sections of the algorithm and ill-balanced computational loads. The paper analyses the diakoptical method using the factorisation tree as a tool. New algorithms are proposed for network splitting, node ordering and load balancing between processors. This, in connection with a novel recursively parallel solution algorithm, gives improved speedup over the conventional parallel solution method. Simulation results are presented which confirm the usefulness of the method.

Epsilon-ritz method for solving optimal control problems: Useful parallel solution method

Using Balakrishnan's epsilon problem formulation (Ref. 1) and the Rayleigh-Ritz method with an orthogonal polynomial function basis, optimal control problems are transformed from the standard two-point boundary-value problem to a nonlinear programming problem. The resulting matrix-vector equations describing the optimal solution have standard parallel solution methods for implementation on parallel processor arrays. The method is modified to handle inequality constraints, and some results are presented under which specialized nonlinear functions, such as sines and cosines, can be handled directly. Some computational results performed on an Intel Sugarcube are presented to illustrate that considerable computational savings can be realized by using the proposed solution method.

A multi-level diffusion method for dynamic load balancing

We consider the problem of dynamic load balancing for multiprocessors, for which a typical application is a parallel finite element solution method using non-structured grids and adaptive grid refinement. This type of application requires communication between the subproblems which arises from the interdependencies in the data. A load balancing algorithm should ideally not make any assumptions about the physical topology of the parallel machine. Further requirements are that the procedure should be fast and accurate. An new multi-level algorithm is presented for solving the dynamic load balancing problem which has these properties and whose parallel complexity is logarithmic in the number of processors used in the computation.

Application of parallel processing techniques to the simulation of power system electromagnetic transients

The authors propose the use of parallel techniques for the computation of power system electromagnetic transients in a multiprocessor environment. System partitioning and parallel solution methods are described. Questions regarding computational load balancing and communication overheads are discussed and techniques are presented to improve the proposed method with respect to those matters. In order to demonstrate the feasibility and to assess the performance of the proposed techniques, a parallel electromagnetic transients program for a multiprocessor environment has been developed. Tests using real power networks of different sizes, executed in an eight-processor hypercube machine, have shown promising performance indices. >

Practical aspects of parallel scientific computing

In this paper we study the parallel solution of a very simple problem, namely the solution of a lower bidiagonal system. It will appear that this problem exhibits many aspects of parallel processing and it has the advantage that the scene is not obscured by complicated algorithmic aspects. The following aspects of the parallel solution methods will be regarded: numerical stability, complexity, data organization, implementation and performance.

Parallel processing techniques for FE analysis: System solution

In a companion paper (D. Zois, Parallel processing techniques for FE analysis: stiffnesses, loads and stresses evaluation, Comput. Struct. 28, 247–260, 1988) parallel processing techniques for element stiffness generations, load generations, the assembly process and the evaluation of the stresses after the system solution are presented for Finite Element (FE) analysis. In the present paper a new parallel system solution method called Asynchronous Systolic Multiple Gauss Jordan (ASMGJ), which can be used for the system solution on a message-passing MIMD parallel computer, is presented. The ASMGJ method consists of a computation algorithm, a communication algorithm and a storage handling algorithm. These algorithms are presented in the present paper, together with examples of how they can be used on a parallel computer of the above type. The ASMGJ method has been implemented as the system solution procedure in PARFES, a Parallel Finite Element System, which can be used on message-passing MIMD parallel computers for the linear static analysis of structures by the FE method. The current implementation of PARFES has been written in OCCAM, a new programming language that supports effectively parallel processing and executes on a VAX 8600 computer.

A Parallel Solution Method for Large Sparse Systems of Equations

This paper presents a new distributed multifrontal sparse matrix decomposition algorithm suitable for message passing parallel processors. The algorithm uses a nested dissection ordering and a multifrontal distribution of the matrix to minimize interprocessor data dependencies and overcome the communication bottleneck previously reported for sparse matrix decomposition [1]. Distributed multifrontal forward elimination and back substitution algorithms are also provided. Results of an implementation on the Intel iPSC are presented. Up to 16 processors are used to solve systems with as many as 7225 equations. With 16 processors, speedups of 10.2 are observed and the decomposition is shown to achieve 67 percent processor utilization. This work was motivated by the need to reduce the computational bottleneck in the Stanford PISCES [2] device simulator; however, it should be applicable to a wide range of scientific and engineering problems