Large Sparse Nonsymmetric Linear Systems Research Articles

GPBi‐CGstab(L): A Lanczos‐type product method unifying Bi‐CGstab(L) and GPBi‐CG

SummaryLanczos‐type product methods (LTPMs), in which the residuals are defined by the product of stabilizing polynomials and the Bi‐CG residuals, are effective iterative solvers for large sparse nonsymmetric linear systems. Bi‐CGstab(L) and GPBi‐CG are popular LTPMs and can be viewed as two different generalizations of other typical methods, such as CGS, Bi‐CGSTAB, and Bi‐CGStab2. Bi‐CGstab(L) uses stabilizing polynomials of degree L, while GPBi‐CG uses polynomials given by a three‐term recurrence (or equivalently, a coupled two‐term recurrence) modeled after the Lanczos residual polynomials. Therefore, Bi‐CGstab(L) and GPBi‐CG have different aspects of generalization as a framework of LTPMs. In the present paper, we propose novel stabilizing polynomials, which combine the above two types of polynomials. The resulting method is referred to as GPBi‐CGstab(L). Numerical experiments demonstrate that our presented method is more effective than conventional LTPMs.

A transformation approach that makes SPAI, PSAI and RSAI procedures efficient for large double irregular nonsymmetric sparse linear systems

A sparse matrix is called double irregular sparse if it has at least one relatively dense column and row, and it is double regular sparse if all the columns and rows of it are sparse. The sparse approximate inverse preconditioning procedures SPAI, PSAI( t o l ) and RSAI( t o l ) are costly and even impractical to construct preconditioners for a large sparse nonsymmetric linear system with the coefficient matrix being double irregular sparse, but they are efficient for double regular sparse problems. Double irregular sparse linear systems have a wide range of applications, and 24.4% of the nonsymmetric matrices in the Florida University collection are double irregular sparse. For this class of problems, we propose a transformation approach, which consists of four steps: (i) transform a given double irregular sparse problem into a small number of double regular sparse ones with the same coefficient matrix A ˆ , (ii) use SPAI, PSAI( t o l ) and RSAI( t o l ) to construct sparse approximate inverses M of A ˆ , (iii) solve the preconditioned double regular sparse linear systems by Krylov solvers, and (iv) recover an approximate solution of the original problem with a prescribed accuracy from those of the double regular sparse ones. A number of theoretical and practical issues are considered on the transformation approach. Numerical experiments on a number of real-world problems confirm the very sharp superiority of the transformation approach to the standard approach that preconditions the original double irregular sparse problem by SPAI, PSAI( t o l ) or RSAI( t o l ) and solves the resulting preconditioned system by Krylov solvers.

Special iterative methods for solution of the steady Convection-Diffusion-Reaction equation with dominant convection

Iterative methods based on skew-symmetric splitting of initial matrix, arising from central finite-difference approximation of steady convection-diffusion-reaction (CDR) equation in 2-D domain are considered. The property of obtained large sparse nonsymmetric linear system Au = f is investigated. A new class of triangular and product triangular skew-symmetric iterative methods is presented. Sufficient conditions of convergence for iterative methods of solution CDR with variable coefficient of reaction and dominant convection are obtained. The results of numerical experiments for the solution of a two-dimensional CDR equation are presented. The uniform grid, central differences for the first derivatives, natural ordering of points, Peclet numbers Pe = 103, 104, 105, variable coefficient of reaction and different velocity coefficients have been used.

Numerical Solution of the Steady Convection-diffusion Equation with Dominant Convection

Steady convection-diffusion equation in 2-D domain is considered. Central finite-difference approximation has been taken to obtain a large sparse nonsymmetric linear system with positive real matrix. New class of product triangular skew-symmetric iterative methods for solution of such system is presented and considered. Using this method as preconditioner for GMRES and BiCG has been made. Results of numerical experiments for two-dimensional convection-diffusion equation for different big Peclet numbers and velocity coefficients have been presented.

A parallel sparse algorithm targeting arterial fluid mechanics computations

Iterative solution of large sparse nonsymmetric linear equation systems is one of the numerical challenges in arterial fluid---structure interaction computations. This is because the fluid mechanics parts of the fluid + structure block of the equation system that needs to be solved at every nonlinear iteration of each time step corresponds to incompressible flow, the computational domains include slender parts, and accurate wall shear stress calculations require boundary layer mesh refinement near the arterial walls. We propose a hybrid parallel sparse algorithm, domain-decomposing parallel solver (DDPS), to address this challenge. As the test case, we use a fluid mechanics equation system generated by starting with an arterial shape and flow field coming from an FSI computation and performing two time steps of fluid mechanics computation with a prescribed arterial shape change, also coming from the FSI computation. We show how the DDPS algorithm performs in solving the equation system and demonstrate the scalability of the algorithm.

COMPARISONS OF THE PARALLEL PRECONDITIONERS FOR LARGE NONSYMMETRIC SPARSE LINEAR SYSTEMS ON A PARALLEL COMPUTER

In this paper we compare various parallel preconditioners for solving large sparse nonsymmetric linear systems. They are Block Jacobi, Point-SSOR, ILU(0) in the wavefront order, ILU(0) in the multi-color order, SPAI(SParse Approximate Inverse), and Multi-Color Block SOR. The Block Jacobi and Point-SSOR are well-known, and ILU(0) is one of the most popular preconditioners, but it is inherently serial. ILU(0) in the wavefront order maximizes the parallelism, and ILU(0) in the multi-color order achieves the parallelism of order (N), where N is the order of the matrix. The SPAI tries to capture the approximate inverse in sparse form, which, then, is expected to be a scalable preconditioner. Finally, we implemented the Multi-Color Block SOR preconditioner combined with direct sparse matrix solver. For the Laplacian matrix the SOR method is known to have a non-deteriorating rate of convergence when used with Multi-Color ordering. Since most of the time is spent on the diagonal inversion, which is done on each processor, we expect it to be a good scalable preconditioner. Finally, due to the blocking effect, it will be effective for ill-conditioned problems. Experiments were conducted for the Finite Difference discretizations of two problems with various meshsizes varying up to 1024×1024, and for an ill-conditioned matrix from the shell problem from the Harwell–Boeing collection. CRAY-T3E with 128 nodes was used. MPI library was used for interprocess communications. The results show that Multi-Color Block SOR and ILU(0) with Multi-Color ordering give the best performances for the finite difference matrices and for the shell problem only the Multi-Color Block SOR and Block Jacobi converges. Based on this we recommend that the Multi-Color Block SOR is the most robust preconditioner out of the preconditioners considered.

An efficient method for constructing an ILU preconditioner for solving large sparse nonsymmetric linear systems by the GMRES method

The main idea of this paper is in determination of the pattern of nonzero elements of the LU factors of a given matrix A. The idea is based on taking the powers of the Boolean matrix derived from A. This powers of a Boolean matrix strategy (PBS) is an efficient, effective, and inexpensive approach. Construction of an ILU preconditioner using PBS is described and used in solving large nonsymmetric sparse linear systems. Effectiveness of the proposed ILU preconditioner in solving large nonsymmetric sparse linear systems by the GMRES method is also shown. Numerical experiments are performed which show that it is possible to considerably reduce the number of GMRES iterations when the ILU preconditioner constructed here is used. In numerical examples, the influence of k, the dimension of the Krylov subspace, on the performance of the GMRES method using an ILU preconditioner is tested. For all the tests carried out, the best value for k is found to be 10.

SCALABILITY ANALYSIS OF PARALLEL GMRES IMPLEMENTATIONS

Applications involving large sparse nonsymmetric linear systems encourage parallel implementations of robust iterative solution methods, such as GMRES(k). Two parallel versions of GMRES(k) based on different data distributions and using Householder reflections in the orthogonalization phase are analyzed with respect to scalability (their ability to maintain fixed efficiency with an increase in problem size and number of processors). A theoretical algorithm-machine model for scalability of GMRES(k) with fixed k is derived and validated by experiments on three parallel computers, each with different machine characteristics. The analysis for an adaptive version of GMRES(k), in which the restart value k is adapted to the problem, is also presented and scalability results for this case are briefly discussed.

LU-decomposition and numerical structure for solving large sparse nonsymmetric linear systems

In this work, the solution of a large sparse linear system of equations with an arbitrary sparsity pattern is obtained by using LU-decomposition method as well as numerical structure approach. The LU-decomposition method is based on Doolittle's method while the numerical structure approach is based on Cramer's rule. The numerical structure approach produces direct solution without facing fill-in problems as encountered in LU-decomposition. In order to reduce the ‘fill-ins’ in the decomposition, the powers of a Boolean matrix, obtained from the coefficient matrix A are taken so that the ‘fill-ins’ in the structure of A can be known in advance. The position of fill-ins in A are thus determined in the best choice manner, that is, it is very effective and memory-wise cheap. We also outline a method by using numerical structure with reduced computation efforts. Finally, experiments are performed on eight examples to compare the efficiency of the proposed methods. The results obtained are reported in a table. It is found that the LU-decomposition method is much better than numerical structure. The usefulness of numerical structure approach is also discussed.

The global Hessenberg and CMRH methods for linear systems with multiple right-hand sides

In this paper, we introduce two new methods for solving large sparse nonsymmetric linear systems with several right-hand sides. These methods are the global Hessenberg and global CMRH methods. Using the global Hessenberg process, these methods are less expensive than the global FOM and global GMRES methods [9]. Theoretical results about the new methods are given, and experimental results that show good performances of these new methods are presented.

COMPARISONS OF THE PARALLEL PRECONDITIONERS ON THE CRAY-T3E FOR LARGE NONSYMMETRIC LINEAR SYSTEMS

In this paper we consider five types of parallel preconditioners for solving large sparse nonsymmetric linear systems on the CRAY-T3E. They are ILU(0) in the wavefront ordering, ILU(0) in the multi-coloring ordering, SSOR in the wavefront ordering, the SPAI(SParse Approximate Inverse) preconditioner, and finally Multi-color Block SOR preconditioner. The ILU(0) is known to be robust and the wavefront ordering naturally exploits the parallelism but has a limited speedup due to the nonuniform lengths of the wavefronts. Multi-coloring is an efficient way of introducing the parallelism of order(N), where N is the order of the matrix but the convergence rate often deteriorates. The SPAI type preconditioner is inherently parallel and is gaining popularity. Finally, for the 5-point Laplacian matrix SOR method is known to have a nondeteriorating rate of convergence when the multi-coloring order is adopted. Also, Block SOR is expected to incur less communication overheads in a message-passing machine. Hence, Multi-Color Block SOR method is expected to have a good performance. Experiments were conducted for the Finite Difference discretizations of two problems with various meshsizes varying up to 1024×1024. MPI library was used for interprocess communications. The results show that ILU(0) in the multi-coloring ordering gives the best performance.

Parallel Adaptive GMRES Implementations for Homotopy Methods

The success of probability-one homotopy methods in solving large-scale optimization problems and nonlinear systems of equations on parallel architectures may be significantly enhanced by the accurate parallel solution of large sparse nonsymmetric linear systems. Iterative solution techniques, such as GMRES(k), favor parallel implementations. However, their straightforward parallelization usually leads to a poor parallel performance because of global communication incurred by processors. One variation of GMRES(k) considered here is to adapt the restart value k for any given problem and use Householder reflections in the orthogonalization phase, coupled with graph-based matrix partitioning, to achieve high accuracy and reduce the communication overhead. This particular GMRES implementation is tailored to the uniquely stringent requirements imposed on a linear system solver by probability-one homotopy algorithms: occasionally unusually high accuracy, ability to adapt to problems of widely varying difficulty, and parallelism.

Reduced Systems for Three-Dimensional Elliptic Equations with Variable Coefficients

We consider large sparse nonsymmetric linear systems arising from finite difference discretization of three-dimensional (3D) convection-diffusion equations with variable coefficients. We show that performing one step of cyclic reduction yields a system of equations which is well conditioned and for which fast convergence can be obtained. A certain block ordering strategy is applied, and analytical results concerning symmetrizability conditions and bounds on convergence rates are given. The analysis is accompanied by numerical examples.

SOR as a preconditioner

Preconditioning is usually necessary for CG-type iterative algorithms for the solution of large sparse nonsymmetric linear systems. However, many good preconditioners have only marginal intrinsic parallelism--ILU and SSOR in the natural ordering are essentially sequential algorithms. Reordering can yield very good parallelism, but can cause (severe) degradation in the rate of convergence. We consider multi-step SOR in the red-black ordering as a preconditioner. It can have very good parallel properties, but it does not cause degradation in the overall rate of convergence, as does ILU or SSOR. We present one-processor results using multi-step SOR to precondition GMRES which substantiate this claim. There has been some confusion in the literature as to whether this approach should be effective. We deal with the objections, and discuss the relationship between the spectral properties of the preconditioned system and the convergence of GMRES. We also present results from the Intel Paragon and the IBM SP2 showing that as a preconditioner, multistep red-black SOR can give good scaled speedup.

Porting a three-dimensional semiconductor device modelling program to the Intel iPSC/860 hypercube

EVEREST is a three-dimensional device modelling package used to study the electrical behaviour of semiconductor devices. A set of partial differential equations describing the current flow within the device is solved using a mixed mesh of tetrahedral and hexahedral elements. The highly non-linear nature of the equations requires that a flexible solution strategy be used to make the software robust and efficient. Within the solution process large sparse non-symmetric linear systems are solved using iterative solvers such as CGS with preconditioning. Run times of many hours or even days are required for realistic devices on current workstations. EVEREST consists of over 100 000 lines of Fortran and was developed for scalar architectures. The main aim of this work was to investigate how such an application could be adapted to run on the iPSC/860. A review of some of the possible parallelisation techniques is made. Most techniques for the parallel implementation of such mesh based calculations on MIMD machines involve a partitioning of the mesh. An extension of an algorithm due to Farhat has been implemented in a pre-processor to partition the device meshes used by EVEREST. Aspects of load balancing and the need to minimise the number of interface nodes in the decomposition are discussed. Some initial results using the partitioned mesh to perform the system matrix assembly are presented. Methods that could be used to implement a parallel preconditioned CGS solver on the iPSC/860 are reviewed and some performance estimates made using the results of certain BBS tests on this machine.

A Newton basis GMRES implementation

The GMRES method by Saad and Schultz is one of the most popular iterative methods for the solution of large sparse non-symmetric linear systems of equations. The implementation proposed by Saad and Schultz uses the Arnoldi process and the modified Gram-Schmidt (MGS) method to compute orthonormal bases of certain Krylov subspaces. The MGS method requires many vector-vector operations, which can be difficult to implement efficiently on vector and parallel computers due to the low granularity of these operations. We present a new implementation of the GMRES method in which, for each Krylov subspace used, we first determine a Newton basis, and then orthogonalize it by computing a QR factorization of the matrix whose columns are the vectors of the Newton basis. In this way we replace the vector-vector operations of the MGS method by the task of computing a QR factorization of a dense matrix. This makes the implementa tion more flexible, and provides a possibility to adapt the computations to the computer at hand in order to achieve better performance.

A comparison of GMRES and CGSTAB accelerations for incompressible Navier-Stokes problems

Chin, P. and P.A. Forsyth, A comparison of GMRES and CGSTAB accelerations for incompressible Navier-Stokes problems, Journal of Computational and Applied Mathematics 46 (1993) 415–426. Recently, incomplete LU preconditioned conjugate-gradient-type matrix solvers have been used in conjunction with Newton or Newton-like methods to solve the fully-coupled incompressible Navier-Stokes equations. A number of acceleration methods can be used for the large sparse nonsymmetric linear systems that arise from such situations. In this work, we compare GMRES and CGSTAB, a “stabilized” variant of CGS. By observing the effect of various factors (e.g., grid size, preconditioning), we attempt to determine the classes of problems to which each acceleration method is best suited. Numerical results obtained with several two-dimensional geometries are provided.

Experiments with Krylov subspace methods on a massively parallel computer

In this note, we compare some Krylov subspace iterative methods on the MASPAR, a massively parallel computer with 16K processors. In particular, we apply these methods to solve large sparse nonsymmetric linear systems arising from elliptic partial differential equations. The methods under consideration include conjugate gradient type methods, semiiterative methods, and a hybrid variant. Our numerical results show that, on the MASPAR, one should compare iterative methods rather on the basis of total computing time than on the basis of number of iterations required to achieve a given accuracy. Our limited numerical experiments here suggest that, in terms of total computing time, semiiterative and hybrid methods are very attractive for such MASPAR implementations.

Implementation of Iterative Methods for Large Sparse Nonsymmetric Linear Systems On a Parallel Vector Machine

We restructure three outstanding iterative methods for large sparse nonsymmetric linear systems. These methods are CGS (conjugate gradient squared), CRS (conjugate residual squared), and Orthomin(k). The re structured methods are more suitable for vector and parallel processing. We implemented these methods on a parallel vector system. The linear systems for the nu merical tests are obtained from discretizing four two- dimensional elliptic partial differential equations by finite difference and finite element methods. A vectorizable and parallelizable version of incomplete LU precondi tioning is used. We restructured the subroutines to en hance the data locality in vector machines with storage hierarchy. Speedup was measured for multitasking by four processors.

A Hybrid Chebyshev Krylov Subspace Algorithm for Solving Nonsymmetric Systems of Linear Equations

We present an iterative method for solving large sparse nonsymmetric linear systems of equations that enhances Manteuflel’s adaptive Chebyshev method with a conjugate gradient-like method. The new method replaces the modified power method for computing needed eigenvalue estimates with Arnoldi’s method, which can be used to simultaneously compute eigenvalues and to improve the approximate solution. Convergence analysis and numerical experiments suggest that the method is more efficient than the original adaptive Chebyshev algorithm.