Parallel Solution Of Linear Systems Research Articles

Analysis of a Splitting Approach for the Parallel Solution of Linear Systems on GPU Cards

We discuss an approach for solving sparse or dense banded linear systems $Ax = b$ on a graphics processing unit (GPU) card. The matrix ${A} \in {\mathbb{R}}^{N \times N}$ is possibly nonsymmetric and moderately large, i.e., ${10,000}{} \leq N \leq {500,000}{}$. The split and parallelize (SaP) approach seeks to partition the matrix $A$ into diagonal subblocks $A_i$, $i=1,\ldots,P$, which are independently factored in parallel. The solution may choose to consider or to ignore the matrices that couple the diagonal subblocks $A_i$. This approach, along with the Krylov-subspace-based iterative method that it preconditions, are implemented in a solver called SaP::GPU, which is compared in terms of efficiency with three commonly used sparse direct solvers: PARDISO, SuperLU, and MUMPS. SaP::GPU, which runs entirely on the GPU except for several stages involved in preliminary row and column permutations, is robust and compares well in terms of efficiency with the aforementioned direct solvers. In a comparison against Intel's MKL, SaP::GPU also fared well when used to solve dense banded systems that are close to being diagonally dominant. SaP::GPU is publicly available and distributed as open source under a permissive BSD-3 license.

Parallel Solution of Linear Systems

AbstractComputational scientists generally seek more accurate results in shorter times, and to achieve this a knowledge of evolving programming paradigms and hardware is important. In particular, optimising solvers for linear systems is a major challenge in scientific computation, and numerical algorithms must be modified or new ones created to fully use the parallel architecture of new computers. Parallel space discretisation solvers for Partial Differential Equations (PDE) such as Domain Decomposition Methods (DDM) are efficient and well documented. At first glance, parallelisation seems to be inconsistent with inherently sequential time evolution, but parallelisation is not limited to space directions. In this article, we present a new and simple method for time parallelisation, based on partial fraction decomposition of the inverse of some special matrices. We discuss its application to the heat equation and some limitations, in associated numerical experiments.

Improved parallel preconditioners for multidisciplinary topology optimisations

ABSTRACTTwo commonly used preconditioners were evaluated for parallel solution of linear systems of equations with high condition numbers. The test cases were derived from topology optimisation applications in multiple disciplines, where the material distribution finite element methods were used. Because in this optimisation method, the equations rapidly become ill-conditioned due to disappearance of large number of elements from the design space as the optimisations progresses, it is shown that the choice for a suitable preconditioner becomes very crucial. In an earlier work the conjugate gradient (CG) method with a Block-Jacobi preconditioner was used, in which the number of CG iterations increased rapidly with the increasing number processors. Consequently, the parallel scalability of the method deteriorated fast due to the increasing loss of interprocessor information among the increased number of processors. By replacing the Block-Jacobi preconditioner with a sparse approximate inverse preconditioner, it is shown that the number of iterations to converge became independent of the number of processors. Therefore, the parallel scalability is improved.

Finite Element Tearing and Interconnecting Method and its Algorithms for Parallel Solution of Magnetic Field Problems

Abstract Because of the exponential increase of computational resource requirement for numerical field simulations of more and more complex physical phenomena and more and more complex large problems in science and engineering practice, parallel processing appears to be an essential tool to handle the resulting large-scale numerical problems. One way of parallelization of sequential (singleprocessor) finite element simulations is the use of domain decomposition methods. Domain decomposition methods (DDMs) for parallel solution of linear systems of equations are based on the partitioning of the analyzed domain into sub-domains which are calculated in parallel while doing appropriate data exchange between those sub-domains. In this case, the non-overlapping domain decomposition method is the Lagrange multiplier based Finite Element Tearing and Interconnecting (FETI) method. This paper describes one direct solver and two parallel solution algorithms of FETI method. Finally, comparative numerical tests demonstrate the differences in the parallel running performance of the solvers of FETI method. We use a single-phase transformer and a three-phase induction motor as twodimensional static magnetic field test problems to compare the solvers

A hybrid GMRES/LS-arnoldi method to accelerate the parallel solution of linear systems

We present a parallel hybrid asynchronous method to solve large sparse linear systems by the use of a large parallel machine. This method combines a parallel GMRES( m) algorithm with the least squares method that needs some eigenvalues obtained from a parallel Arnoldi algorithm. All of the algorithms run on different processors of an IBM SP3 or IBM SP4 computer simultaneously. This implementation of this hybrid method allows us to take advantage of the parallelism available and to accelerate the convergence by decreasing considerably the number of iterations.

Parallel solution of linear system of equations using a column oriented approach

Abstract We present an efficient parallel algorithm for solving linear systems of equations. The algorithm runs with nine processors on Transputer array. We give the experimental, the theoretical timings and the analysis of calculation the optimum number of processors.

Hybrid scheduling for the parallel solution of linear systems

We consider the problem of designing a dynamic scheduling strategy that takes into account both workload and memory information in the context of the parallel multifrontal factorization. The originality of our approach is that we base our estimations (work and memory) on a static optimistic scenario during the analysis phase. This scenario is then used during the factorization phase to constrain the dynamic decisions that compute fully irregular partitions in order to better balance the workload. We show that our new scheduling algorithm significantly improves both the memory behaviour and the factorization time. We give experimental results for large challenging real-life 3D problems on 64 and 128 processors.

Processor Efficient Parallel Solution of Linear Systems of Equations

We present a deterministic parallel algorithm that solves a n-dimensional system Ax=b of linear equations over an ordered field or over a subfield of the complex numbers. This algorithm uses O(log2n) parallel time and O(max{M(n),n2(loglogn)/logn}) arithmetic processors if M(n) is the processor complexity of fast parallel matrix multiplication.

Developments and trends in the parallel solution of linear systems

In this review paper, we consider some important developments and trends in algorithm design for the solution of linear systems concentrating on aspects that involve the exploitation of parallelism. We briefly discuss the solution of dense linear systems, before studying the solution of sparse equations by direct and iterative methods. We consider preconditioning techniques for iterative solvers and discuss some of the present research issues in this field.

A Parallel Matrix Class Library in C++ for Computational Mechanics Applications

The object–oriented design and implementation of a matrix class library in C++ for parallel computing is described. The main goal of this library is to support the management of matrix data in a distributed environment and the parallel solution of linear systems of algebraic equations typical of computational mechanics applications. By using the object–oriented paradigm, a clean and consistent interface to the library is provided, and the implementation details are hidden from the user. These features improve the clarity and expressiveness of the client code and consequently ease the task of parallel programming. The library was developed and tested originally using a network of Sun Sparc 10 workstations. Test examples subsequently were run on the Intel Paragon XP/S. In these examples, the portability of the library among different distributed environments was demonstrated, and some preliminary performance studies of the library were carried out. Finally, the utilization of the class library in parallel finite–element computations is discussed. It is shown that the present library greatly simplifies the implementation of matrix operations intrinsic to parallel finite–element applications.

A parallel solver for the hp-version of finite element methods

A prototype software system has been implemented for the parallel solution of linear systems associated with the hp-version, using the master/slave model and the Parallel Virtual Machine (PVM) message passing library. In a series of performance tests, significant speed-up was achieved in those typical cases for the hp-version where there was sufficient computational granularity to justify use of the parallel method. These tests indicate that the algorithms devised for element ordering, load distribution and load balancing are sufficiently robust. These tests also indicate that, even though communication overhead in a network environment is relatively high, there is significant potential for scaling the method to larger processor ensembles. The solution method developed was implemented as a four step procedure. In step 1, a dynamic load distribution algorithm selects an element for each idle process based on the mesh topology. In step 2, element descriptions are communicated to each process where local matrix computations are performed in parallel. In step 3, regional assembly and elimination operations are carried out in parallel to produce multiple ‘frontal matrices’ which are eventually returned to a host process for final solution. In step 4, the solution to the global stiffness matrix is back-substituted into each previous frontal matrix to determine the values of all unknowns eliminated during the parallel assembly procedure.

Object-oriented parallel programming tools for structural engineering applications

The principal objective of this work is to develop portable and extensible programming tools for the development of object-oriented parallel finite element codes for structural engineering applications. An object-oriented parallel portability interface for message-passing operations has been designed and implemented. An existing object-oriented matrix library is currently being extended to support the management of distributed matrix data and parallel solution of linear systems of algebraic equations. By taking advantage of C++ object-oriented programming, both the class libraries provide clean and consistent user interfaces, which not only help to improve the clarity and expressiveness of the client parallel codes, but also hide implementation details and complexity from the user to ease parallel programming tasks. In this paper, the object-oriented design and implementation of the class libraries are discussed. The libraries were first developed and tested using a network of Sun SPARC 10 workstations. Application examples were then studied on two commercial parallel computers: the IBM SP1 and the Intel Paragon XP/S 10, for evaluation of the portability and efficiency of the present class libraries.

Two-Stage and Multisplitting Methods for the Parallel Solution of Linear Systems

Two-stage and multisplitting methods for the parallel solution of linear systems are studied. A two-stage multisplitting method is presented that reduces to each of the others in particular cases. Conditions for its convergence are given. In the particular case of a multisplitting method related to block Jacobi, it is shown that it is equivalent to a two-stage method with only one inner iteration per outer iteration. A fixed number of iterations of this method, say, p, is compared with a two-stage method with p inner iterations. The asymptotic rate of convergence of the first method is faster, but, depending on the structure of the matrix and the parallel architecture, it takes more time to converge. This is illustrated with numerical experiments.

Parallel solution of linear systems by repeated squaring

We present an efficient parallel implementation of iterative methods for the solution of linear systems. The performance attained can be favourably compared with the one of known algorithms.

A Monte Carlo method for the parallel solution of linear systems

A new parallel algorithm for the solution of linear systems, based upon the Monte Carlo approach, is shown. The method allows one to obtain the solution of a linear system with parallel cost growing as the logarithm of the size of the coefficient matrix, and with “probabilistic” error bounded in terms of the Chebyshev inequality.

Parallel solution of linear systems with striped sparse matrices

The multiplication of a vector by a matrix and the solution of triangular linear systems are the most demanding operations in the majority of iterative techniques for the solution of linear systems. Data-driven VLSI networks which perform these two operations, efficiently, for certain sparse matrices are introduced. In order to avoid computations that involve zero operands, the non-zero elements in a sparse matrix are organized in the form of non-overlapping stripes, and only the elements within the stripe structure of the matrix are manipulated. Detailed analysis of the networks proves that both operations may be completed in n global cycles with minimal communication overhead, where n is the order of the linear system. The number of cells in each network as well as the communication overhead, are determined by the stripe structure of the matrix. Different stripe structures for the class of sparse matrices generated in Finite Element Analysis are examined in a separate paper.

Iterative methods for the parallel solution of linear systems

The purpose of this paper is to introduce a new technique for the parallel solution of linear systems by iterative methods, attaining the bound O(log 2 n), and consisting of the acceleration of given convergent iterative methods.

Multi-Splittings of Matrices and Parallel Solution of Linear Systems

We present two classes of matrix splittings and give applications to the parallel iterative solution of systems of linear equations. These splittings generalize regular splittings and P-regular splittings, resulting in algorithms which can be implemented efficiently on parallel computing systems. Convergence is established, rate of convergence is discussed, and numerical examples are given.

Parallel solution of linear systems by quadrant interlocking factorisation methods

This paper presents a parallel algorithm for the solution of linear systems and determinant evaluation suitable for use on the proposed parallel computers of the future. The new method can be considered as extending the novel matrix factorisation strategies introduced by Evans and Hatzopoulos [1] and Evans and Hadjidimos [2] in which quadrant interlocking factors are considered instead of the more usual LU factors of triangular decomposition.