Class Of Preconditioners Research Articles

Analysis of optimal preconditioners for CutFEM

AbstractIn this article, we consider a class of unfitted finite element methods for scalar elliptic problems. These so‐called CutFEM methods use standard finite element spaces on a fixed unfitted triangulation combined with the Nitsche technique and a ghost penalty stabilization. As a model problem we consider the application of such a method to the Poisson interface problem. We introduce and analyze a new class of preconditioners that is based on a subspace decomposition approach. The unfitted finite element space is split into two subspaces, where one subspace is the standard finite element space associated to the background mesh and the second subspace is spanned by all cut basis functions corresponding to nodes on the cut elements. We will show that this splitting is stable, uniformly in the discretization parameter and in the location of the interface in the triangulation. Based on this we introduce an efficient preconditioner that is uniformly spectrally equivalent to the stiffness matrix. Using a similar splitting, it is shown that the same preconditioning approach can also be applied to a fictitious domain CutFEM discretization of the Poisson equation. Results of numerical experiments are included that illustrate optimality of such preconditioners for the Poisson interface problem and the Poisson fictitious domain problem.

A Class of Iterative Solvers for the Helmholtz Equation: Factorizations, Sweeping Preconditioners, Source Transfer, Single Layer Potentials, Polarized Traces, and Optimized Schwarz Methods

Solving time-harmonic wave propagation problems by iterative methods is a difficult task, and over the last two decades an important research effort has gone into developing preconditioners for the simplest representative of such wave propagation problems, the Helmholtz equation. A specific class of these new preconditioners is considered here. They were developed by researchers with various backgrounds using formulations and notations that are very different, and all are among the most promising preconditioners for the Helmholtz equation. \indent The goal of the present article is to show that this class of preconditioners is based on a common mathematical principle, and that they can all be formulated in the context of domain decomposition methods known as optimized Schwarz methods. This common formulation allows us to explain in detail how and why all these methods work. The domain decomposition formulation also allows us to avoid technicalities in the implementation description we give of these recent methods. \indent The equivalence of these methods with optimized Schwarz methods translates at the discrete level into equivalence with approximate block LU decomposition preconditioners, and in each case we give the algebraic version, including a detailed description of the approximations used. While we choose to use the Helmholtz equation for which these methods were developed, our notation is completely general and the algorithms we give are written for an arbitrary second-order elliptic operator. The algebraic versions are even more general, assuming only a connectivity pattern in the discretization matrix. \indent All the new methods studied here are based on sequential decomposition of the problem in space into a sequence of subproblems, and they have in their optimal form the property to lead to nilpotent iterations, like an exact block LU factorization. Using our domain decomposition formulation, we finally present an algorithm for two-dimensional decompositions, i.e., decompositions that contain cross points, which is still nilpotent in its optimal form. Its approximation is currently an active area of research, and it would have been difficult to discover such an algorithm without the domain decomposition framework.

A comparison of preconditioned Krylov subspace methods for large‐scale nonsymmetric linear systems

SummaryPreconditioned Krylov subspace (KSP) methods are widely used for solving large‐scale sparse linear systems arising from numerical solutions of partial differential equations (PDEs). These linear systems are often nonsymmetric due to the nature of the PDEs, boundary or jump conditions, or discretization methods. While implementations of preconditioned KSP methods are usually readily available, it is unclear to users which methods are the best for different classes of problems. In this work, we present a comparison of some KSP methods, including GMRES, TFQMR, BiCGSTAB, and QMRCGSTAB, coupled with three classes of preconditioners, namely, Gauss–Seidel, incomplete LU factorization (including ILUT, ILUTP, and multilevel ILU), and algebraic multigrid (including BoomerAMG and ML). Theoretically, we compare the mathematical formulations and operation counts of these methods. Empirically, we compare the convergence and serial performance for a range of benchmark problems from numerical PDEs in two and three dimensions with up to millions of unknowns and also assess the asymptotic complexity of the methods as the number of unknowns increases. Our results show that GMRES tends to deliver better performance when coupled with an effective multigrid preconditioner, but it is less competitive with an ineffective preconditioner due to restarts. BoomerAMG with a proper choice of coarsening and interpolation techniques typically converges faster than ML, but both may fail for ill‐conditioned or saddle‐point problems, whereas multilevel ILU tends to succeed. We also show that right preconditioning is more desirable. This study helps establish some practical guidelines for choosing preconditioned KSP methods and motivates the development of more effective preconditioners.

Preconditioners for the geometry optimisation and saddle point search of molecular systems

A class of preconditioners is introduced to enhance geometry optimisation and transition state search of molecular systems. We start from the Hessian of molecular mechanical terms, decompose it and retain only its positive definite part to construct a sparse preconditioner matrix. The construction requires only the computation of the gradient of the corresponding molecular mechanical terms that are already available in popular force field software packages. For molecular crystals, the preconditioner can be combined straightforwardly with the exponential preconditioner recently introduced for periodic systems. The efficiency is demonstrated on several systems using empirical, semiempirical and ab initio potential energy surfaces.

New preconditioning techniques for the steady and unsteady buoyancy driven flow problems

In this paper we study the performances of generalized minimal residual method (GMRES) preconditioned with geometric multigrid (GMG), applied to steady and unsteady buoyancy driven flow problems, discretized with the finite element method. For the unsteady case, the second order Crank–Nicolson method is used for the temporal discretization. At each geometric multigrid level, we use Richardson iterative solvers preconditioned with different combinations of physics-based and domain decomposition preconditioners. Three different preconditioners are considered: incomplete LU decomposition (ILU), overlapping Vanka-type domain decomposition for additive Schwarz method (ASM), and field split (FS) physics-based decomposition. We also analyze the effect on the smoother of how the variables are ordered, and in particular whether the leading variable is the velocity or the temperature, resulting in six classes of preconditioners: ILU_VT, ILU_TV, ASM_VT, ASM_TV, FS_VT and FS_TV. The eigenvalue analysis for the six preconditioners is conducted to study the rate of GMRES convergence under several Prandtl numbers. The numerical performances of nested combinations of the above preconditioners are compared. Numerical results show that the pair of FS_VT and FS_TV preconditioners works better than the other two pairs, and that the FS_TV preconditioner always performs the best in terms of the computational time for all the steady and unsteady cases.

Preconditioning of GMRES by skew-Hermitian iterations

A class of preconditioners for solving non-Hermitian positive definite systems of linear algebraic equations is proposed and investigated. It is based on Hermitian and skew-Hermitian splitting of the initial matrix. A generalization for saddle point systems having semidefinite or singular (1, 1) blocks is given. Our approach is based on an augmented Lagrangian formulation. It is shown that such preconditioners can be efficiently used for the iterative solution of systems of linear algebraic equations by the GMRES method.

A novel class of approximate inverse preconditioners for large positive definite linear systems in optimization

We propose a class of preconditioners for large positive definite linear systems, arising in nonlinear optimization frameworks. These preconditioners can be computed as by-product of Krylov-subspace solvers. Preconditioners in our class are chosen by setting the values of some user-dependent parameters. We first provide some basic spectral properties which motivate a theoretical interest for the proposed class of preconditioners. Then, we report the results of a comparative numerical experience, among some preconditioners in our class, the unpreconditioned case and the preconditioner in Fasano and Roma (Comput Optim Appl 56:253---290, 2013). The experience was carried on first considering some relevant linear systems proposed in the literature. Then, we embedded our preconditioners within a linesearch-based Truncated Newton method, where sequences of linear systems (namely Newton's equations), are required to be solved. We performed an extensive numerical testing over the entire medium-large scale convex unconstrained optimization test set of CUTEst collection (Gould et al. Comput Optim Appl 60:545---557, 2015), confirming the efficiency of our proposal and the improvement with respect to the preconditioner in Fasano and Roma (Comput Optim Appl 56:253---290, 2013).

Inexact linear solvers for control volume discretizations in porous media

We discuss the construction of multi-level inexact linear solvers for control volume discretizations for porous media. The methodology forms a contrast to standard iterative solvers by utilizing an algebraic hierarchy of approximations which preserve the conservative structure of the underlying control volume. Our main result is the generalization of multiscale control volume methods as multi-level inexact linear solvers for conservative discretizations through the design of a particular class of preconditioners. This construction thereby bridges the gap between multiscale approximation and linear solvers. The resulting approximation sequence is referred to as inexact solvers. We seek a conservative solution, in the sense of control-volume discretizations, within a prescribed accuracy. To this end, we give an abstract guaranteed a posteriori error bound relating the accuracy of the linear solver to the underlying discretization. These error bounds are explicitly computable for the grids considered herein. The afore-mentioned hierarchy of conservative approximations can also be considered in the context of multi-level upscaling, and this perspective is highlighted in the text as appropriate. The new construction is supported by numerical examples highlighting the performance of the inexact linear solver realized in both a multi- and two-level context for two- and three-dimensional heterogeneous problems defined on structured and unstructured grids. The numerical examples assess the performance of the approach both as an inexact solver, as well in comparison to standard algebraic multigrid methods.

Overlapping for preconditioners based on incomplete factorizations and nested arrow form

In this paper, we discuss the usage of overlapping techniques for improving the convergence of preconditioners based on incomplete factorizations. To enable parallelism, these preconditioners are usually applied after the input matrix is permuted into a nested arrow form using k-way nested dissection. This graph partitioning technique uses k-way partitionning by vertex separator to recursively partition the graph of the input matrix into k subgraphs using a subset of its vertices called a separator. The overlapping technique is then based on algebraically extending the associated subdomains of these subgraphs and their corresponding separators obtained from k-way nested dissection by their direct neighbours. A similar approach is known to accelerate the convergence of domain decomposition methods, where the input matrix is partitioned into a number of independent subdomains using k-way vertex partitioning of a graph by edge separators, a different graph decomposition technique. We discuss the effect of the overlapping technique on the convergence of two classes of preconditioners, on the basis of nested factorization and block incomplete LDU factorization.

Preconditioned Krylov Subspace Methods for Sampling Multivariate Gaussian Distributions

A common problem in statistics is to compute sample vectors from a multivariate Gaussian distribution with zero mean and a given covariance matrix $A$. A canonical approach to the problem is to compute vectors of the form $y = S z$, where $S$ is the Cholesky factor or square root of $A$, and $z$ is a standard normal vector. When $A$ is large, such an approach becomes computationally expensive. This paper considers preconditioned Krylov subspace methods to perform this task. The Lanczos process provides a means to approximate $A^{1/2} z$ for any vector $z$ from an $m$-dimensional Krylov subspace. The main contribution of this paper is to show how to enhance the convergence of the process via preconditioning. Both incomplete Cholesky preconditioners and approximate inverse preconditioners are discussed. It is argued that the latter class of preconditioners has an advantage in the context of sampling. Numerical tests, performed with stationary covariance matrices used to model Gaussian processes, illustrate the dramatic improvement in computation time that can result from preconditioning.

Modification of iterative methods for solving linear complementarity problems

Purpose – The purpose of this paper is to present the efficient iterative methods for solving linear complementarity problems (LCP), using a class of pre-conditioners. Design/methodology/approach – By using the concept of solving the fixed-point system of equations associated to the LCP, pre-conditioning techniques and Krylov subspace methods the authors design some projected methods to solve LCP. Furthermore, within the computational framework, some models of pre-conditioners candidates are investigated and evaluated. Findings – The proposed algorithms have a simple and graceful structure and can be applied to other complementarity problems. Asymptotic convergence of the sequence generated by the method to the unique solution of LCP is proved, along with a result regarding the convergence rate of the pre-conditioned methods. Finally, a computational comparison of the standard methods against pre-conditioned methods based on Example 1 is presented which illustrate the merits of simplicity, power and effectiveness of the proposed algorithms. Research limitations/implications – Comparison between the authors' methods and other similar methods for the studied problem shows a remarkable agreement and reveals that their models are superior in point of view rate of convergence and computing efficiency. Originality/value – For solving LCP more attention has recently been paid on a class of iterative methods called the matrix-splitting such as AOR, MAOR, GAOR, SSOR, etc. But up to now, no paper has discussed the effect of pre-conditioning technique for matrix-splitting methods in LCP. So, this paper is planning to fill in this gap and the authors use a class of pre-conditioners with iterative methods and analyze the convergence of these methods for LCP.

Preconditioning for modal discontinuous Galerkin methods for unsteady 3D Navier–Stokes equations

We compare different block preconditioners in the context of parallel time adaptive higher order implicit time integration using Jacobian-free Newton–Krylov (JFNK) solvers for discontinuous Galerkin (DG) discretizations of the three dimensional time dependent Navier–Stokes equations. A special emphasis of this work is the performance for a relative high number of processors, i.e. with a low number of elements on the processor. For high order DG discretizations, a particular problem that needs to be addressed is the size of the blocks in the Jacobian. Thus, we propose a new class of preconditioners that exploits the hierarchy of modal basis functions and introduces a flexible order of the off-diagonal Jacobian blocks. While the standard preconditioners ‘block Jacobi’ (no off-blocks) and full symmetric Gauss–Seidel (full off-blocks) are included as special cases, the reduction of the off-block order results in the new scheme ROBO-SGS. This allows us to investigate the impact of the preconditioner’s sparsity pattern with respect to the computational performance. Since the number of iterations is not well suited to judge the efficiency of a preconditioner, we additionally consider CPU time for the comparisons. We found that both block Jacobi and ROBO-SGS have good overall performance and good strong parallel scaling behavior.

Iterative methods with analytical preconditioning technique to linear complementarity problems: application to obstacle problems

For solving linear complementarity problems LCP more attention has recently been paid on a class of iterative methods called the matrix-splitting. But up to now, no paper has discussed the effect of preconditioning technique for matrix-splitting methods in LCP . So, this paper is planning to fill in this gap and we use a class of preconditioners with generalized Accelerated Overrelaxation (GAOR ) methods and analyze the convergence of these methods for LCP . Furthermore, Comparison between our methods and other non-preconditioned methods for the studied problem shows a remarkable agreement and reveals that our models are superior in point of view of convergence rate and computing efficiency. Besides, by choosing the appropriate parameters of these methods, we derive same results as the other iterative methods such as AOR , JOR , SOR etc. Finally the method is tested by some numerical experiments.

Block Preconditioners for Coupled Physics Problems

Finite element discretizations of multiphysics problems frequently give rise to block-structured linear algebra problems that require effective preconditioners. We build two classes of preconditioners in the spirit of well-known block factorizations [M. F. Murphy, G. H. Golub, and A. J. Wathen, SIAM J. Sci. Comput., 21 (2000), pp. 1969--1972; I. C. F. Ipsen, SIAM J. Sci. Comput., 23 (2001), pp. 1050--1051] and apply these to the diffusive portion of the bidomain equations and the Bénard convection problem. An abstract generalized eigenvalue problem allows us to give application-specific bounds for the real parts of eigenvalues for these two problems. This analysis is accompanied by numerical calculations with several interesting features. One of our preconditioners for the bidomain equations converges in five iterations for a range of problem sizes. For Bénard convection, we observe mesh-independent convergence with reasonable robustness with respect to physical parameters, and offer some preliminary parallel scaling results on a multicore processor via message passing interface (MPI).

Preconditioning Large Indefinite Linear Systems

After briefly recalling some relevant approaches for preconditioning large symmetric linear systems, we describe a novel class of preconditioners. Our proposal is tailored for large indefinite linear systems, which arise very frequently in many different contexts of numerical analysis and nonlinear optimization. Our preconditioners are built as a byproduct of the Krylov subspace method used to solve the system. We describe theoretical properties of the proposed class of preconditioners, namely their capability of both shifting some eigenvalues of the system’s matrix to controlled values, and reducing the modulus of the other ones. The results of a numerical experimentation give evidence of the good performance of our proposal.

A Class of Preconditioners for Large Indefinite Linear Systems, as By-Product of Krylov Subspace Methods: Part II

We propose a class of preconditioners, which are also tailored for symmetric linear systems from linear algebra and nonconvex optimization. Our preconditioners are specifically suited for large linear systems and may be obtained as by-product of Krylov subspace solvers. Each preconditioner in our class is identified by setting the values of a pair of parameters and a scaling matrix, which are user-dependent, and may be chosen according with the structure of the problem in hand. We provide theoretical properties for our preconditioners. In particular, we show that our preconditioners both shift some eigenvalues of the system matrix to controlled values, and they tend to reduce the modulus of most of the other eigenvalues. In a companion paper we study some structural properties of our class of preconditioners, and report the results on a significant numerical experience.

A class of approximate inverse preconditioners for solving linear systems

Some preconditioners for accelerating the classical iterative methods are given in Zhang et al. [Y. Zhang and T.Z. Huang, A class of optimal preconditioners and their applications, Proceedings of the Seventh International Conference on Matrix Theory and Its Applications in China, 2006. Y. Zhang, T.Z. Huang, and X.P. Liu, Modified iterative methods for nonnegative matrices and M-matrices linear systems, Comput. Math. Appl. 50 (2005), pp. 1587–1602. Y. Zhang, T.Z. Huang, X.P. Liu, A class of preconditioners based on the (I+S(α))-type preconditioning matrices for solving linear systems, Appl. Math. Comp. 189 (2007), pp. 1737–1748]. Another kind of preconditioners approximating the inverse of a symmetric positive definite matrix was given in Simons and Yao [G. Simons, Y. Yao, Approximating the inverse of a symmetric positive definite matrix, Linear Algebra Appl. 281 (1998), pp. 97–103]. Zhang et al. ’s preconditioners and Simons and Yao's are generalized in this paper. These preconditioners are all of low construction cost, which all could be taken as approximate inverse of M-matrices. Numerical experiments of these preconditioners applied with Krylov subspace methods show the effectiveness and performance, which also show that the preconditioners proposed in this paper are better approximate inverse for M-matrices than Simons’.

An iterative solver-based long-step infeasible primal-dual path-following algorithm for convex QP based on a class of preconditioners

In this paper, we present a long-step infeasible primal-dual path-following algorithm for convex quadratic programming (CQP) whose search directions are computed by means of a preconditioned iterative linear solver. In contrast to the authors’ previous paper [Z. Lu, R.D.C. Monteiro, and J.W. O'Neal. An iterative solver-based infeasible primal-dual path-following algorithm for convex quadratic programming, SIAM J. Optim. 17(1) (2006), pp. 287–310], we propose a new linear system, which we refer to as the hybrid augmented normal equation (HANE), to determine the primal-dual search directions. Since the iterative linear solver can only generate an approximate solution to the HANE, this solution does not yield a primal-dual search direction satisfying all equations of the primal-dual Newton system. We propose a recipe to compute an inexact primal-dual search direction, based on a suitable approximate solution to the HANE. The second difference between this paper and [Z. Lu, R.D.C. Monteiro, and J.W. O'Neal. An iterative solver-based infeasible primal-dual path-following algorithm for convex quadratic programming, SIAM J. Optim. 17(1)(2006), pp. 287–310] is that, instead of using the maximum weight basis (MWB) preconditioner in the aforesaid recipe for constructing the inexact search direction, this paper proposes the use of any member of a whole class of preconditioners, of which the MWB preconditioner is just a special case. The proposed recipe allows us to: (i) establish a polynomial bound on the number of iterations performed by our path-following algorithm and (ii) establish a uniform bound, depending on the quality of the preconditioner, on the number of iterations performed by the iterative solver.

Frobenius norm minimization and probing for preconditioning

In this paper we introduce a new method for defining preconditioners for the iterative solution of a system of linear equations. By generalizing the class of modified preconditioners (e.g. MILU), the interface probing, and the class of preconditioners related to the Frobenius norm minimization (e.g. FSAI, SPAI) we develop a toolbox for computing preconditioners that are improved relative to a given small probing subspace. Furthermore, by this MSPAI (modified SPAI) probing approach we can improve any given preconditioner with respect to this probing subspace. All the computations are embarrassingly parallel. Additionally, for symmetric linear system we introduce new techniques for symmetrizing preconditioners. Many numerical examples, e.g. from PDE applications such as domain decomposition and Stokes problem, show that these new preconditioners often lead to faster convergence and smaller condition numbers.

Support-Graph Preconditioners

We present a preconditioning technique, called support-graph preconditioning, and use it to analyze two classes of preconditioners. The technique was first described in a talk by Pravin Vaidya, who did not formally publish his results. Vaidya used the technique to devise and analyze a class of novel preconditioners. The technique was later extended by Gremban and Miller, who used it in the development and analysis of yet another class of new preconditioners. This paper extends the technique further and uses it to analyze a class of existing preconditioners, modified incomplete Cholesky. The paper also contains a presentation of Vaidya's preconditioners, which was previously missing from the literature.