Sparse Approximate Inverse Research Articles

An efficient sparse approximate inverse preconditioning for FMM implementation

AbstractFor efficiently solving large dense complex linear systems that arise in electric field integral equations (EFIE) of electromagnetic scattering problems, the fast multipole method (FMM) is used to accelerate the matrix‐vector product operations, and the sparse approximate inverse (SAI) preconditioning technique is employed to speed up the convergence rate of the Krylov iterations. A good quality SAI preconditioner in the FMM context is constructed based on the near‐field matrix of the EFIE. The main purpose of this paper is to show that the quality of the SAI preconditioner can be greatly improved by use of information from FMM implementation. Numerical experiments indicate that the resulted SAI preconditioner is very effective with FMM and can reduce the overall simulation time significantly. © 2007 Wiley Periodicals, Inc. Microwave Opt Technol Lett 49: 1746–1750, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.22538

Parallel imaging reconstruction for arbitrary trajectories usingk-space sparse matrices (kSPA)

Although the concept of receiving MR signal using multiple coils simultaneously has been known for over two decades, the technique has only recently become clinically available as a result of the development of several effective parallel imaging reconstruction algorithms. Despite the success of these algorithms, it remains a challenge in many applications to rapidly and reliably reconstruct an image from partially-acquired general non-Cartesian k-space data. Such applications include, for example, three-dimensional (3D) imaging, functional MRI (fMRI), perfusion-weighted imaging, and diffusion tensor imaging (DTI), in which a large number of images have to be reconstructed. In this work, a systematic k-space-based reconstruction algorithm based on k-space sparse matrices (kSPA) is introduced. This algorithm formulates the image reconstruction problem as a system of sparse linear equations in k-space. The inversion of this system of equations is achieved by computing a sparse approximate inverse matrix. The algorithm is demonstrated using both simulated and in vivo data, and the resulting image quality is comparable to that of the iterative sensitivity encoding (SENSE) algorithm. The kSPA algorithm is noniterative and the computed sparse approximate inverse can be applied repetitively to reconstruct all subsequent images. This algorithm, therefore, is particularly suitable for the aforementioned applications.

A numerical experimental study of inverse preconditioning for the parallel iterative solution to 3D finite element flow equations

Integration of the subsurface flow equation by finite elements (FE) in space and finite differences (FD) in time requires the repeated solution to sparse symmetric positive definite systems of linear equations. Iterative techniques based on preconditioned conjugate gradients (PCG) are one of the most attractive tool to solve the problem on sequential computers. A present challenge is to make PCG attractive in a parallel computing environment as well. To this aim a key factor is the development of an efficient parallel preconditioner. FSAI (factorized sparse approximate inverse) and enlarged FSAI relying on the approximate inverse of the coefficient matrix appears to be a most promising parallel preconditioner. In the present paper PCG using FSAI, diagonal and pARMS (parallel algebraic recursive multilevel solvers) preconditioners is implemented on the IBM SP4/512 and CLX/768 supercomputers with up to 32 processors to solve underground flow problems of a large size. The results show that FSAI may allow for a parallel relative efficiency E p * larger than 50% on the largest problems with p = 32 processors. Moreover, FSAI turns out to be significantly less expensive and more robust than pARMS. Finally, it is shown that E p * for p in the upper range may be much improved if PCG–FSAI is implemented on CLX.

Factored sparse approximate inverse of block tridiagonal and block pentadiagonal matricies

This paper is concerned with approaches to compute a factored sparse approximate inverse for block tridiagonal and block pentadiagonal matrices. Recurrence formulas are developed for computing sparse approximate inverse factors of these matrices using bordering technique. Resulting factored sparse approximate inverse is used as a preconditioner for the conjugate gradient method (PCG). As an application these formulas are simplified for computing the preconditioner for solving Lyapanuv matrix equations by PCG method. Numerical experiments on linear system, arising from discretization of partial differential equations are presented.

Notes on convergence of an algebraic multigrid method

The convergence theory for algebraic multigrid (AMG) algorithms proposed in Chang and Huang [Q.S. Chang, Z.H. Huang, Efficient algebraic multigrid algorithms and their convergence, SIAM J. Sci. Comput. 24 (2002) 597–618] is further discussed and a smaller and elegant upper bound is obtained. On the basis of element-free AMGe [V.E. Henson, P.S. Vassilevski, Element-free AMGe: General algorithms for computing interpolation weights in AMG, SIAM J. Sci. Comput. 23(2) (2001) 629–650] we rewrite the interpolation operator for the classical AMG (cAMG), present a uniform expression and then, by introducing a sparse approximate inverse in the Frobenius norm, give a general convergence theorem which is suited for not only cAMG but also AMG for finite elements and element-free AMGe.

On the performance of SPAI and ADI-like preconditioners for core collapse supernova simulations in one spatial dimension

The simulation of core collapse supernovæ calls for the time accurate solution of the (Euler) equations for inviscid hydrodynamics coupled with the equations for neutrino transport. The time evolution is carried out by evolving the Euler equations explicitly and the neutrino transport equations implicitly. Neutrino transport is modeled by the multi-group Boltzmann transport (MGBT) and the multi-group flux limited diffusion (MGFLD) equations. An implicit time stepping scheme for the MGBT and MGFLD equations yields Jacobian systems that necessitate scaling and preconditioning. Two types of preconditioners, namely, a sparse approximate inverse (SPAI) preconditioner and a preconditioner based on the alternating direction implicit iteration (ADI-like) have been found to be effective for the MGFLD and MGBT formulations. This paper compares these two preconditioners. The ADI-like preconditioner performs well with both MGBT and MGFLD systems. For the MGBT system tested, the SPAI preconditioner did not give competitive results. However, since the MGBT system in our experiments had a high condition number before scaling and since we used a sequential platform, care must be taken in evaluating these results.

Sparse and explicit FETD via approximate inverse Hodge (mass) matrix

Finite-element time-domain (FETD) simulations of Maxwell equations in irregular simplicial grids require the solution of a sparse linear system involving the Hodge (mass) matrix at each time step. This can be avoided by mass lumping techniques that approximate the mass matrix by a diagonal matrix, but not without shortcomings. In this communication, we propose an alternative approach to yield a conditionally stable, fully explicit, and sparse FETD by using a sparse approximate inverse of the mass matrix.

A Matrix-free Two-grid Preconditioner for Solving Boundary Integral Equations in Electromagnetism

In this paper, we describe a matrix-free iterative algorithm based on the GMRES method for solving electromagnetic scattering problems expressed in an integral formulation. Integral methods are an interesting alternative to differential equation solvers for this problem class since they do not require absorbing boundary conditions and they mesh only the surface of the radiating object giving rise to dense and smaller linear systems of equations. However, in realistic applications the discretized systems can be very large and for some integral formulations, like the popular Electric Field Integral Equation, they become ill-conditioned when the frequency increases. This means that iterative Krylov solvers have to be combined with fast methods for the matrix-vector products and robust preconditioning to be affordable in terms of CPU time. In this work we describe a matrix-free two-grid preconditioner for the GMRES solver combined with the Fast Multipole Method. The preconditioner is an algebraic two-grid cycle built on top of a sparse approximate inverse that is used as smoother, while the grid transfer operators are defined using spectral information of the preconditioned matrix. Experiments on a set of linear systems arising from real radar cross section calculation in industry illustrate the potential of the proposed approach for solving large-scale problems in electromagnetism.

ON THE PARALLEL SOLUTION OF LARGE INDUSTRIAL WAVE PROPAGATION PROBLEMS

The use of Fast Multipole Methods (FMM) combined with embedded Krylov solvers preconditioned by a sparse approximate inverse is investigated for the solution of large linear systems arising in industrial acoustic and electromagnetic simulations. We use a boundary element integral equation method to solve the Helmholtz and the Maxwell equations in the frequency domain. The resulting linear systems are solved by iterative solvers using FMM to accelerate the matrix-vector products. The simulation code is developed in a distributed memory environment using message passing and it has out-of-core capabilities to handle very large calculations. When the calculation involves one incident wave, one linear system has to be solved. In this situation, embedded solvers can be combined with an approximate inverse preconditioner to design extremely robust algorithms. For radar cross section calculations, several linear systems have to be solved. They involve the same coefficient matrix but different right-hand sides. In this case, we propose a block variant of the single right-hand side scheme. The efficiency, robustness and parallel scalability of our approach are illustrated on a set of large academic and industrial test problems.

A two-level sparse approximate inverse preconditioner for unsymmetric matrices

Sparse approximate inverse (SPAI) preconditioners are effective in accelerating iterative solutions of a large class of unsymmetric linear systems and their inherent parallelism has been widely explored. The effectiveness of SPAI relies on the assumption of the unknown true inverse admitting a sparse approximation. Furthermore, for the usual right SPAI, one must restrict the number of non-zeros in each column to control the overall construction cost and this restriction can reduce the effectiveness of such preconditioners. To extend the applicability of SPAI, this paper proposes to use two-level preconditioning: possible dense columns of the true inverse, skipped by right SPAI (column-wise), will be better approximated by left SPAI (row-wise). Essentially, we approximate the true inverse by sparse matrices via a Gauss-Jordan like decomposition. Numerical experiments on a class of benchmark test matrices show that our new idea of two-level preconditioning can lead to a major enhancement to the standard SPAI method.

Application of a two-step preconditioning strategy to the finite element analysis for electromagnetic problems

A two-step preconditioning strategy is presented for the conjugate gradient (CG) iterative method to solve a large system of linear equations resulting from the use of edge-based finite-element discretizations of Helmholtz equations. The key idea is to combine both the factorized sparse approximate inverse (FSAI) and the symmetric successive overrelaxation (SSOR) preconditioning techniques in two successive steps in order to obtain a better preconditioner for the original matrix equations. The newly constructed preconditioner combines the advantages of both the FSAI and SSOR preconditioners with less computational complexity without the breakdowns of incomplete factorization technique. Numerical simulations show that this jointly preconditioned CG iterative method has a faster convergence speed than both FSAI and SSOR preconditioned CG methods. © 2006 Wiley Periodicals, Inc. Microwave Opt Technol Lett 48: 1623–1627, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.21683

Block Preconditioners Based on Approximate Commutators

This paper introduces a strategy for automatically generating a block for solving the incompressible Navier--Stokes equations. We consider the convection--diffusion preconditioners proposed by Kay, Loghin, and Wathen [SIAM J. Sci. Comput., 24 (2002), pp. 237-256] and Silvester, Elman, Kay, and Wathen [J. Comput. Appl. Math., 128 (2001), pp. 261-279]. Numerous theoretical and numerical studies have demonstrated mesh independent convergence on several problems and the overall efficacy of this methodology. A drawback, however, is that it requires the construction of a convection--diffusion operator (denoted $F_p$) projected onto the discrete pressure space. This means that integration of this idea into a code that models incompressible flow requires a sophisticated understanding of the discretization and other implementation issues, something often held only by the developers of the model. As an alternative, we consider automatic ways of computing $F_p$ based on purely algebraic considerations. The new methods are closely related to the BFBt preconditioner of Elman [SIAM J. Sci. Comput., 20 (1999), pp. 1299-1316]. We use the fact that the is derived from considerations of commutativity between the gradient and convection--diffusion operators, together with methods for computing sparse approximate inverses, to generate the required matrix $F_p$ automatically. We demonstrate that with this strategy the favorable convergence properties of the preconditioning methodology are retained.

On the preconditioners for fast multipole boundary element methods for 2D multi-domain elastostatics

Fast multipole method (FMM) has been successfully applied to accelerate the numerical solvers of boundary element method (BEM). However, the coefficient matrix implicitly formed by using FMM is sometimes ill-conditioned in cases when mixed boundary conditions exist, resulting in poor rate of convergence for iteration. So preconditioning is a critical part in the development of efficient FMM solver for BEM. In this paper, preconditioners based on sparse approximate inverse type are used for fast multipole BEM to deal with 2D elastostatics. Several sparsity patterns of the preconditioner are considered for single- and multi-domain problems, especially for 2D elastic body with large number of inclusions or cracks. Algorithms and cost analysis of preconditioning under different prescribed sparsity patterns are discussed. GMRES is used as the iterative solver. Numerical results show this type of preconditioner achieves satisfactory rate of convergence for fast multipole BEM and performs well for problems of fairly large sizes.

Diagonally-striped matrices and approximate inverse preconditioners

The inverse of a banded matrix is, in general, dense. If the structure of the original banded matrix is “striped”, that is, the non-zero diagonals are separated by one or more zero diagonals, the inverse may exhibit a similar striped structure. The motivation for studying inverses of striped matrices is to obtain efficient preconditioners for systems arising from radiation transport equations, whose matrices include dominant values along diagonal stripes. Linear systems whose system matrix has a striped inverse lend themselves to the use of a sparse approximate inverse (SPAI) preconditioner whose structure is derived from that of the actual inverse.

Sparse Approximate Inverses and Target Matrices

If P has a prescribed sparsity and minimizes the Frobenius norm |I-PA|F, it is called a sparse approximate inverse of A. It is well known that the computation of such a matrix P is via the solution of independent linear least squares problems for the rows separately (and therefore in parallel). In this paper we consider the choice of other norms and introduce the idea of "target" matrices. A target matrix, T, is readily inverted and thus forms part of a preconditioner when |T-PA| is minimized over some appropriate sparse matrices P. The use of alternatives to the Frobenius norm which maintain parallelizability, while discussed in early literature, does not appear to have been exploited. (A correction to this article has been appended to the pdf file.)

Fast multipole method applied to elastostatic BEM–FEM coupling

BEM–FEM coupling is desirable for three-dimensional problems involving specific features such as (i) large or unbounded media with linear constitutive properties, (ii) cracks, (iii) critical parts of complex geometry requiring accurate stress analyses. However, for cases with a BEM discretization involving a large number N BEM of degrees of freedom, setting up the BEM contribution to the coupled problem using conventional techniques is an expensive O ( N BEM 2 ) task. Moreover, the fully-populated BEM block entails a O ( N BEM 2 ) storage requirement and a O ( N BEM 3 ) contribution to the solution time via usual direct solvers. To overcome these pitfalls, the BEM contribution is formulated using the fast multipole method (FMM) and the coupled equations are solved by means of an iterative GMRES solver. Both the storage requirements and the solution times are found to be close to O( N BEM). A preconditioner based on the sparse approximate inverse of the BEM block is shown to improve the convergence of the GMRES solver. Numerical examples involving N BEM = O(10 5 − 10 6) unknowns, run on a PC computer, are presented; they include the Eshelby inclusion (as a validation example), a many-inclusion configuration, and a dam structure.

Sparse Inverse Preconditioning of Multilevel Fast Multipole Algorithm for Hybrid Integral Equations in Electromagnetics

In computational electromagnetics, the multilevel fast multipole algorithm (MLFMA) is used to reduce the computational complexity of the matrix vector product operations. In iteratively solving the dense linear systems arising from discretized hybrid integral equations, the sparse approximate inverse (SAI) preconditioning technique is employed to accelerate the convergence rate of the Krylov iterations. We show that a good quality SAI preconditioner can be constructed by using the near part matrix numerically generated in the MLFMA. The main purpose of this study is to show that this class of the SAI preconditioners are effective with the MLFMA and can reduce the number of Krylov iterations substantially. Our experimental results indicate that the SAI preconditioned MLFMA maintains the computational complexity of the MLFMA, but converges a lot faster, thus effectively reduces the overall simulation time.

Ainv and bilum preconditioning techniques

It was proposed that a robust and efficient parallelizable preconditioner for solving general sparse linear systems of equations, in which the use of sparse approximate inverse (AINV) techniques in a multi-level block ILU (BILUM) preconditioner were investigated. The resulting preconditioner retains robustness of BILUM preconditioner and has two advantages over the standard BILUM preconditioner: the ability to control sparsity and increased parallelism. Numerical experiments are used to show the effectiveness and efficiency of the new preconditioner.

Parallel simulation of anisotropic diffusion with human brain DT-MRI Data

We conduct simulations for the 3D unsteady state anisotropic diffusion process with DT-MRI data in the human brain by discretizing the governing diffusion equation on Cartesian grid and adopting a high performance differential–algebraic equation (DAE) solver, the parallel version of implicit differential–algebraic (IDA) solver, to tackle the resulting large scale system of DAEs. Parallel preconditioning techniques including sparse approximate inverse and banded-block-diagonal preconditioners are used with the GMRES method to accelerate the convergence rate of the iterative solution. We then investigate and compare the efficiency and effectiveness of the two parallel preconditioners. The experimental results of the diffusion simulations on a parallel supercomputer show that the sparse approximate inverse preconditioning strategy, which is robust and efficient with good scalability, gives a much better overall performance than the banded-block-diagonal preconditioner.

COMPARISONS OF THE PARALLEL PRECONDITIONERS FOR LARGE NONSYMMETRIC SPARSE LINEAR SYSTEMS ON A PARALLEL COMPUTER

In this paper we compare various parallel preconditioners for solving large sparse nonsymmetric linear systems. They are Block Jacobi, Point-SSOR, ILU(0) in the wavefront order, ILU(0) in the multi-color order, SPAI(SParse Approximate Inverse), and Multi-Color Block SOR. The Block Jacobi and Point-SSOR are well-known, and ILU(0) is one of the most popular preconditioners, but it is inherently serial. ILU(0) in the wavefront order maximizes the parallelism, and ILU(0) in the multi-color order achieves the parallelism of order (N), where N is the order of the matrix. The SPAI tries to capture the approximate inverse in sparse form, which, then, is expected to be a scalable preconditioner. Finally, we implemented the Multi-Color Block SOR preconditioner combined with direct sparse matrix solver. For the Laplacian matrix the SOR method is known to have a non-deteriorating rate of convergence when used with Multi-Color ordering. Since most of the time is spent on the diagonal inversion, which is done on each processor, we expect it to be a good scalable preconditioner. Finally, due to the blocking effect, it will be effective for ill-conditioned problems. Experiments were conducted for the Finite Difference discretizations of two problems with various meshsizes varying up to 1024×1024, and for an ill-conditioned matrix from the shell problem from the Harwell–Boeing collection. CRAY-T3E with 128 nodes was used. MPI library was used for interprocess communications. The results show that Multi-Color Block SOR and ILU(0) with Multi-Color ordering give the best performances for the finite difference matrices and for the shell problem only the Multi-Color Block SOR and Block Jacobi converges. Based on this we recommend that the Multi-Color Block SOR is the most robust preconditioner out of the preconditioners considered.