Optimal Preconditioner Research Articles

Exploiting high‐contrast Stokes preconditioners to efficiently solve incompressible fluid–structure interaction problems

AbstractIn this work, we develop a new algorithm to solve large‐scale incompressible time‐dependent fluid–structure interaction problems using a matrix‐free finite element method in arbitrary Lagrangian–Eulerian frame of reference. We derive a semi‐implicit time integration scheme which improves the geometry‐convective explicit scheme for problems involving the interaction between incompressible hyperelastic solids and incompressible fluids. The proposed algorithm relies on the reformulation of the time‐discrete problem as a generalized Stokes problem with strongly variable coefficients, for which optimal preconditioners have recently been developed. The resulting algorithm is scalable, optimal, and robust: we test our implementation on model problems that mimic classical Turek–Hron benchmarks in two and three dimensions, and investigate timing and scalability results.

Non-Convex Optimization: Using Preconditioning Matrices for Optimally Improving Variable Bounds in Linear Relaxations

The performance of branch-and-bound algorithms for solving non-convex optimization problems greatly depends on convex relaxation techniques. They generate convex regions which are used for improving the bounds of variable domains. In particular, convex polyhedral regions can be represented by a linear system A.x=b. Then, bounds of variable domains can be improved by minimizing and maximizing variables in the linear system. Reducing or contracting optimally variable domains in linear systems, however, is an expensive task. It requires solving up to two linear programs for each variable (one for each variable bound). Suboptimal strategies, such as preconditioning, may offer satisfactory approximations of the optimal reduction at a lower cost. In non-square linear systems, a preconditioner P can be chosen such that P.A is close to a diagonal matrix. Thus, the projection of the equivalent system P.A.x=P.b over x, by using an iterative method such as Gauss–Seidel, can significantly improve the contraction. In this paper, we show how to generate an optimal preconditioner, i.e., a preconditioner that helps the Gauss–Seidel method to optimally reduce the variable domains. Despite the cost of generating the preconditioner, it can be re-used in sub-regions of the search space without losing too much effectiveness. Experimental results show that, when used for reducing domains in non-square linear systems, the approach is significantly more effective than Gauss-based elimination techniques. Finally, the approach also shows promising results when used as a component of a solver for non-convex optimization problems.

Horizon-Independent Preconditioner Design for Linear Predictive Control

First-order optimizationsolvers, such as the fast gradient method (FGM), are increasingly being used to solve model predictive control problems in resource-constrained environments. Unfortunately, the convergence rate of these solvers is significantly affected by the conditioning of the problem data, with ill-conditioned problems requiring a large number of iterations. To reduce the number of iterations required, we present a simple method for computing a horizon-independent preconditioning matrix for the Hessian of the condensed problem. The preconditioner is based on the block Toeplitz structure of the Hessian. Horizon independence allows one to use only the predicted system and cost matrices to compute the preconditioner, instead of the full Hessian. The proposed preconditioner has equivalent performance to an optimal preconditioner in numerical examples, producing speedups between 2x and 9x for the FGM. Additionally, we derive horizon-independent spectral bounds for the Hessian in terms of the transfer function of the predicted system, and show how these can be used to compute a novel horizon-independent bound on the condition number for the preconditioned Hessian.

Analysis of optimal preconditioners for CutFEM

AbstractIn this article, we consider a class of unfitted finite element methods for scalar elliptic problems. These so‐called CutFEM methods use standard finite element spaces on a fixed unfitted triangulation combined with the Nitsche technique and a ghost penalty stabilization. As a model problem we consider the application of such a method to the Poisson interface problem. We introduce and analyze a new class of preconditioners that is based on a subspace decomposition approach. The unfitted finite element space is split into two subspaces, where one subspace is the standard finite element space associated to the background mesh and the second subspace is spanned by all cut basis functions corresponding to nodes on the cut elements. We will show that this splitting is stable, uniformly in the discretization parameter and in the location of the interface in the triangulation. Based on this we introduce an efficient preconditioner that is uniformly spectrally equivalent to the stiffness matrix. Using a similar splitting, it is shown that the same preconditioning approach can also be applied to a fictitious domain CutFEM discretization of the Poisson equation. Results of numerical experiments are included that illustrate optimality of such preconditioners for the Poisson interface problem and the Poisson fictitious domain problem.

Uniform block-diagonal preconditioners for divergence-conforming HDG Methods for the generalized Stokes equations and the linear elasticity equations

Abstract We propose a uniform block-diagonal preconditioner for condensed $H$(div)-conforming hybridizable discontinuous Galerkin schemes for parameter-dependent saddle point problems, including the generalized Stokes equations and the linear elasticity equations. An optimal preconditioner is obtained for the stiffness matrix on the global velocity/displacement space via the auxiliary space preconditioning technique (Xu (1994) The Auxiliary Space Method and Optimal Multigrid Preconditioning Techniques for Unstructured Grids, vol. 56. International GAMM-Workshop on Multi-level Methods (Meisdorf), pp. 215–235). A spectrally equivalent approximation to the Schur complement on the element-wise constant pressure space is also constructed, and an explicit computable exact inverse is obtained via the Woodbury matrix identity. Finally, the numerical results verify the robustness of our proposed preconditioner with respect to model parameters and mesh size.

Preconditioning for a Pressure-Robust HDG Discretization of the Stokes Equations

We introduce a new preconditioner for a recently developed pressure-robust hybridized discontinuous Galerkin (HDG) finite element discretization of the Stokes equations. A feature of HDG methods is the straightforward elimination of degrees-of-freedom defined on the interior of an element. In our previous work (J. Sci. Comput., 77(3):1936--1952, 2018) we introduced a preconditioner for the case in which only the degrees-of-freedom associated with the element velocity were eliminated via static condensation. In this work we introduce a preconditioner for the statically condensed system in which the element pressure degrees-of-freedom are also eliminated. In doing so the number of globally coupled degrees-of-freedom are reduced, but at the expense of a more difficult problem to analyse. We will show, however, that the Schur complement of the statically condensed system is spectrally equivalent to a simple trace pressure mass matrix. This result is used to formulate a new, provably optimal preconditioner. Through numerical examples in two- and three-dimensions we show that the new preconditioned iterative method converges in fewer iterations, has superior conservation properties for inexact solves, and is faster in CPU time when compared to our previous preconditioner.

Fast iterative solver for the optimal control of time‐dependent PDEs with Crank–Nicolson discretization in time

AbstractIn this article, we derive a new, fast, and robust preconditioned iterative solution strategy for the all‐at‐once solution of optimal control problems with time‐dependent PDEs as constraints, including the heat equation and the non‐steady convection–diffusion equation. After applying an optimize‐then‐discretize approach, one is faced with continuous first‐order optimality conditions consisting of a coupled system of PDEs. As opposed to most work in preconditioning the resulting discretized systems, where a (first‐order accurate) backward Euler method is used for the discretization of the time derivative, we employ a (second‐order accurate) Crank–Nicolson method in time. We apply a carefully tailored invertible transformation for symmetrizing the matrix, and then derive an optimal preconditioner for the saddle‐point system obtained. The key components of this preconditioner are an accurate mass matrix approximation, a good approximation of the Schur complement, and an appropriate multigrid process to apply this latter approximation—these are constructed using our work in transforming the matrix system. We prove the optimality of the approximation of the Schur complement through bounds on the eigenvalues, and test our solver against a widely‐used preconditioner for the linear system arising from a backward Euler discretization. These demonstrate the effectiveness and robustness of our solver with respect to mesh‐sizes, regularization parameter, and diffusion coefficient.

Direct optimization of BPX preconditioners

We consider an automatic construction of locally optimal preconditioners for positive definite linear systems. To achieve this goal, we introduce a differentiable loss function that does not explicitly include the estimation of minimal eigenvalue. Nevertheless, the resulting optimization problem is equivalent to a direct minimization of the condition number. To demonstrate our approach, we construct a parametric family of modified BPX preconditioners. Namely, we define a set of empirical basis functions for coarse finite element spaces and tune them to achieve better condition number. For considered model equations (that includes Poisson, Helmholtz, Convection–diffusion, Biharmonic, and others), we achieve from two to twenty times smaller condition numbers for symmetric positive definite linear systems.

Optimal preconditioners on solving the Poisson equation with Neumann boundary conditions

MILU preconditioner is well known [16,3] to be the optimal choice among all the ILU-type preconditioners in solving the Poisson equation with Dirichlet boundary conditions. However, it is less known which is an optimal preconditioner in solving the Poisson equation with Neumann boundary conditions. The condition number of an unpreconditioned matrix is as large as O(h−2), where h is the step size of grid. Only the optimal preconditioner results in condition number O(h−1), while the others such as Jacobi and ILU result in O(h−2).We review Relaxed ILU and Perturbed MILU preconditioners in the case of Neumann boundary conditions, and present empirical results which indicate that the former is optimal in two dimensions and the latter is optimal in two and three dimensions. To the best of our knowledge, these empirical results have not been rigorously verified yet. We present a formal proof for the optimality of Relaxed ILU in rectangular domains, and discuss its possible extension to general smooth domains and Perturbed MILU.

Spectral analysis of saddle-point matrices from optimization problems with elliptic PDE constraints

The main focus of this paper is the characterization and exploitation of the asymptotic spectrum of the saddle--point matrix sequences arising from the discretization of optimization problems constrained by elliptic partial differential equations. They uncover the existence of an hidden structure in these matrix sequences, namely, they show that these are indeed an example of Generalized Locally Toeplitz (GLT) sequences. They show that this enables a sharper characterization of the spectral properties of such sequences than the one that is available by using only the fact that they deal with saddle--point matrices. Finally, they exploit it to propose an optimal preconditioner strategy for the GMRES, and Flexible--GMRES methods.

A short note on plain convergence of adaptive least-squares finite element methods

We show that adaptive least-squares finite element methods driven by the canonical least-squares functional converge under weak conditions on PDE operator, mesh-refinement, and marking strategy. Contrary to prior works, our plain convergence does neither rely on sufficiently fine initial meshes nor on severe restrictions on marking parameters. Finally, we prove that convergence is still valid if a contractive iterative solver is used to obtain the approximate solutions (e.g., the preconditioned conjugate gradient method with optimal preconditioner). The results apply within a fairly abstract framework which covers a variety of model problems.

Preconditioning the EFIE on screens

We consider the electric field integral equation (EFIE) modeling the scattering of time-harmonic electromagnetic waves at a perfectly conducting screen. When discretizing the EFIE by means of low-order Galerkin boundary methods (BEM), one obtains linear systems that are ill-conditioned on fine meshes and for low wave numbers [Formula: see text]. This makes iterative solvers perform poorly and entails the use of preconditioning. In order to construct optimal preconditioners for the EFIE on screens, the authors recently derived compact equivalent inverses of the EFIE operator on simple Lipschitz screens in [R. Hiptmair and C. Urzúa-Torres, Compact equivalent inverse of the electric field integral operator on screens, Integral Equations Operator Theory 92 (2020) 9]. This paper elaborates how to use this result to build an optimal operator preconditioner for the EFIE on screens that can be discretized in a stable fashion. Furthermore, the stability of the preconditioner relies only on the stability of the discrete [Formula: see text] duality pairing for scalar functions, instead of the vectorial one. Therefore, this novel approach not only offers [Formula: see text]-independent and [Formula: see text]-robust condition numbers, but it is also easier to implement and accommodates non-uniform meshes without additional computational effort.

Speeding up SimRank computations by polynomial preconditioners

SimRank is popularly used for evaluating similarities between nodes in a graph. Though some recent work has transformed the SimRank into a linear system which can be solved by using conjugate gradient (CG) algorithm. However, the diagonal preconditioner used in the algorithm still left some room for improvement. There is no work showing what should be an optimal preconditioner for the linear system. For such a sake, in this paper, we theoretically study the preconditioner problem and propose a polynomial preconditioner which can improve the computation speed significantly. In our investigation, we also modify the original linear system to adapt to all situations of the graph. We further prove that the condition number of a polynomial preconditioned matrix is bounded in a narrow interval. It implies that the new preconditioner can effectively accelerate the convergence rate of the CG algorithm. The experimental results show the proposed algorithm outperforms the state-of-the-art algorithms for the all-pairs SimRank computation.

Adaptive BEM with inexact PCG solver yields almost optimal computational costs

We consider the preconditioned conjugate gradient method (PCG) with optimal preconditioner in the frame of the boundary element method for elliptic first-kind integral equations. Our adaptive algorithm steers the termination of PCG as well as the local mesh-refinement. Besides convergence with optimal algebraic rates, we also prove almost optimal computational complexity. In particular, we provide an additive Schwarz preconditioner which can be computed in linear complexity and which is optimal in the sense that the condition numbers of the preconditioned systems are uniformly bounded. As model problem serves the 2D or 3D Laplace operator and the associated weakly-singular integral equation with energy space widetilde{H}^{-1/2}(Gamma ). The main results also hold for the hyper-singular integral equation with energy space H^{1/2}(Gamma ).

An optimal preconditioner for tensor equations involving Einstein product

ABSTRACT An optimal preconditioner for tensor equations involving the Einstein product is considered. This preconditioner actually is an approximation to any given tensor in some tensor subspaces such as the subspace of all circulant tensors. Numerical examples show that the optimal preconditioner is efficient for some Toeplitz tensor equations.

On approximate inverse of Hermite and Laguerre collocation differentiation matrices and new collocation schemes in unbounded domains

In this paper, we provide an explicit, stable and fast means to compute the approximate inverse of Hermite/Laguerre collocation differentiation matrices, and also the approximate inverse of the Hermite/Laguerre collocation matrices of a second-order differential operator. The latter offers optimal preconditioners for developing well-conditioned Hermite/Laguerre collocation schemes. We apply the new approaches to various partial differential equations in unbounded domains and demonstrate the advantages over the usual collocation methods.

Preconditioning of a Hybridized Discontinuous Galerkin Finite Element Method for the Stokes Equations

We present optimal preconditioners for a recently introduced hybridized discontinuous Galerkin finite element discretization of the Stokes equations. Typical of hybridized discontinuous Galerkin methods, the method has degrees-of-freedom that can be eliminated locally (cell-wise), thereby significantly reducing the size of the global problem. Although the linear system becomes more complex to analyze after static condensation of these element degrees-of-freedom, the pressure Schur complement of the original and reduced problem are the same. Using this fact, we prove spectral equivalence of this Schur complement to two simple matrices, which is then used to formulate optimal preconditioners for the statically condensed problem. Numerical simulations in two and three spatial dimensions demonstrate the good performance of the proposed preconditioners.

Optimal preconditioners for systems defined by functions of Toeplitz matrices

We propose several circulant preconditioners for systems defined by some functions g of Toeplitz matrices An. In this paper we are interested in solving g(An)x=b by the preconditioned conjugate method or the preconditioned minimal residual method, namely in the cases when g(z) are the functions ez, sin⁡z and cos⁡z. Numerical results are given to show the effectiveness of the proposed preconditioners.

A Korovkin-type theory for non-self-adjoint Toeplitz operators

We consider the Frobenius optimal preconditioners for Toeplitz operators with complex-valued symbols. We prove Korovkin-type theorems for C⁎-algebra generated by complex-valued continuous periodic symbols corresponding to Toeplitz operators under various modes of convergence in eigenvalue cluster sense. This generalizes the existing results for C⁎-algebra generated by real-valued symbols. In some special cases, the optimal preconditioners can be chosen from matrix algebras with faster convergence rate for the corresponding preconditioned linear systems. We will also consider the linear positive operators associated with the eigenvalues of these preconditioners and obtain a Korovkin-type theorem for such operators as an application of our results.

共轭梯度法中预条件子的优化

为了降低方程组求解中共轭梯度法系数矩阵的条件数，提高收敛速度，常用预处理方法将原方程进行等价转化，同时预条件子既要接近原系数矩阵，又要容易求其逆矩阵。本文从寻求对角预条件子出发，用矩阵的特征值分解方法解出了预处理后系数矩阵特征值矩阵的显式表达，得到对角预条件子矩阵的最优选择，并予以证明。给出了三个p-范数预条件子，将之与常用的预条件子进行对比，实例检验表明三个p-范数预条件子的作用更优越，且使算法收敛更快。 For decreasing the conditional number of the coefficient matrix in solving the linear equations with conjugate gradient methods and accelerating the convergence, it is common to use preconditioned methods to find the equivalent equations, whose conditional numbers are smaller. It is required that the preconditioners should be as close as possible to the original coefficient matrix and their inverse matrices can be easily computed. Starting from diagonal preconditioners, we first compute the eigenvalue decomposition of the coefficient matrix, and obtain the optimal preconditioner. However, it is of high computational complexity to do the eigenvalue decomposition. In this paper, we introduce three p-norm preconditioners to approximate the optimal preconditioner. Comparing with the existing preconditioners, the experimental results show that the proposed three diagonal p-norm preconditioners converge much faster, which demonstrates the advantages of the proposed family of preconditioners.