Block Diagonal Preconditioner Research Articles

A new approximation of the Schur complement in preconditioners for PDE‐constrained optimization

SUMMARYSaddle point systems arise widely in optimization problems with constraints. The utility of Schur complement approximation is now broadly appreciated in the context of solving such saddle point systems by iteration. In this short manuscript, we present a new Schur complement approximation for PDE‐constrained optimization, an important class of these problems. Block diagonal and block triangular preconditioners have previously been designed to be used to solve such problems along with MINRES and non‐standard Conjugate Gradients, respectively; with appropriate approximation blocks, these can be optimal in the sense that the time required for solution scales linearly with the problem size, however small the mesh size we use. In this paper, we extend this work to designing such preconditioners for which this optimality property holds independently of both the mesh size and the Tikhonov regularization parameterβthat is used. This also leads to an effective symmetric indefinite preconditioner that exhibits mesh andβindependence. We motivate the choice of these preconditioners based on observations about approximating the Schur complement obtained from the matrix system, derive eigenvalue bounds that verify the effectiveness of the approximation and present numerical results that show that these new preconditioners work well in practice. Copyright © 2011 John Wiley & Sons, Ltd.

Fast multipole accelerated boundary element method for the Helmholtz equation in acoustic scattering problems

We apply the fast multipole method (FMM) accelerated boundary element method (BEM) for the three-dimensional (3D) Helmholtz equation, and as a result, large-scale acoustic scattering problems involving 400000 elements are solved efficiently. This is an extension of the fast multipole BEM for two-dimensional (2D) acoustic problems developed by authors recently. Some new improvements are obtained. In this new technique, the improved Burton-Miller formulation is employed to overcome non-uniqueness difficulties in the conventional BEM for exterior acoustic problems. The computational efficiency is further improved by adopting the FMM and the block diagonal preconditioner used in the generalized minimum residual method (GMRES) iterative solver to solve the system matrix equation. Numerical results clearly demonstrate the complete reliability and efficiency of the proposed algorithm. It is potentially useful for solving large-scale engineering acoustic scattering problems.

Parallel numerical solution of the time‐harmonic Maxwell equations in mixed form

SUMMARYWe develop a fully scalable parallel implementation of an iterative solver for the time‐harmonic Maxwell equations with vanishing wave numbers. We use a mixed finite element discretization on tetrahedral meshes, based on the lowest order Nédélec finite element pair of the first kind. We apply the block diagonal preconditioning approach of Greif and Schötzau (Numer. Linear Algebra Appl. 2007; 14(4):281–297), and use the nodal auxiliary space preconditioning technique of Hiptmair and Xu (SIAM J. Numer. Anal. 2007; 45(6):2483–2509) as the inner iteration for the shifted curl–curl operator. Algebraic multigrid is employed to solve the resulting sequence of discrete elliptic problems. We demonstrate the performance of our parallel solver on problems with constant and variable coefficients. Our numerical results indicate good scalability with the mesh size on uniform, unstructured, and locally refined meshes. Copyright © 2011 John Wiley & Sons, Ltd.

Stability and preconditioning for a hybrid approximation on the sphere

This paper proposes a new preconditioning scheme for a linear system with a saddle-point structure arising from a hybrid approximation scheme on the sphere, an approximation scheme that combines (local) spherical radial basis functions and (global) spherical polynomials. In principle the resulting linear system can be preconditioned by the block-diagonal preconditioner of Murphy, Golub and Wathen. Making use of a recently derived inf–sup condition and the Brezzi stability and convergence theorem for this approximation scheme, we show that in this context the Schur complement in the above preconditioner is spectrally equivalent to a certain non-constant diagonal matrix. Numerical experiments with a non-uniform distribution of data points support the theoretically proved quality of the new preconditioner.

A fast multipole boundary element method based on the improved Burton–Miller formulation for three-dimensional acoustic problems

A fast multipole boundary element method (FMBEM) based on the improved Burton–Miller formulation is presented in this paper for solving large-scale three-dimensional (3D) acoustic problems. Some improvements can be made for the developed FMBEM. In order to overcome the non-unique problems of the conventional BEM, the FMBEM employs the improved Burton–Miller formulation developed by the authors recently to solve the exterior acoustic problems for all wave numbers. The improved Burton–Miller formulation contains only weakly singular integrals, and avoids the numerical difficulties associated to the evaluation of the hypersingular integral, it leads to the numerical implementations more efficient and straightforward. In this study, the fast multipole method (FMM) and the preconditioned generalized minimum residual method (GMRES) iterative solver are applied to solve system matrix equation. The block diagonal preconditioner needs no extra memory and no extra CPU time in each matrix–vector product. Thus, the overall computational efficiency of the developed FMBEM is further improved. Numerical examples clearly demonstrate the accuracy, efficiency and applicability of the FMBEM based on improved Burton–Miller formulation for large-scale acoustic problems.

Preconditioning Iterative Methods for the Optimal Control of the Stokes Equations

Solving problems regarding the optimal control of partial differential equations (PDEs)—also known as PDE-constrained optimization—is a frontier area of numerical analysis. Of particular interest is the problem of flow control, where one would like to effect some desired flow by exerting, for example, an external force. The bottleneck in many current algorithms is the solution of the optimality system—a system of equations in saddle point form that is usually very large and ill conditioned. In this paper we describe two preconditioners—a block diagonal preconditioner for the minimal residual method and a block lower-triangular preconditioner for a nonstandard conjugate gradient method—which can be effective when applied to such problems where the PDEs are the Stokes equations. We consider only distributed control here, although we believe other problems could be treated in the same way. We give numerical results, and we compare these with those obtained by solving the equivalent forward problem using similar techniques.

Constructing efficient substructure-based preconditioners for BEM systems of equations

In this work, a generic substructuring algorithm is employed to construct global block-diagonal preconditioners for BEM systems of equations. In this strategy, the allowable fill-in positions are those on-diagonal block matrices corresponding to each BE subregion. As these subsystems are independently assembled, the preconditioner for a particular BE model, after the LU decomposition of all subsystem matrices, is easily formed. So as to highlight the efficiency of the preconditioning proposed, the Bi-CG solver, which presents a quite erratic convergence behavior, is considered. In the particular applications of this paper, 3D representative volume elements (RVEs) of carbon-nanotube (CNT) composites are analyzed. The models contain up to several tens of thousands of degrees of freedom. The efficiency and relevance of the preconditioning technique is also discussed in the context of developing general (parallel) BE codes.

Stabilizing block diagonal preconditioners for complex dense matrices in electromagnetics

Preconditioning techniques are widely used to speed up the convergence of iterative methods for solving large linear systems with sparse or dense coefficient matrices. For certain application problems, however, the standard block diagonal preconditioner makes the Krylov iterative methods converge more slowly or even diverge. To handle this problem, we apply diagonal shifting and stabilized singular value decomposition (SVD) to each diagonal block, which is generated from the multilevel fast multiple algorithm (MLFMA), to improve the stability and efficiency of the block diagonal preconditioner. Our experimental results show that the improved block diagonal preconditioner maintains the computational complexity of MLFMA, converges faster and also reduces the CPU cost.

Preconditioning discretizations of systems of partial differential equations

This survey paper is based on three talks given by the second author at the London Mathematical Society Durham Symposium on Computational Linear Algebra for Partial Differential Equations in the summer of 2008. The main focus will be on an abstract approach to the construction of preconditioners for symmetric linear systems in a Hilbert space setting. Typical examples that are covered by this theory are systems of partial differential equations which correspond to saddle point problems. We will argue that the mapping properties of the coefficient operators suggest that block diagonal preconditioners are natural choices for these systems. To illustrate our approach a number of examples will be considered. In particular, parameter-dependent systems arising in areas like incompressible flow, linear elasticity, and optimal control theory will be studied. The paper contains analysis of several models which have previously been discussed in the literature. However, here each example is discussed with reference to a more unified abstract approach. Copyright 2010 John Wiley & Sons, Ltd.

Order optimal preconditioners for fully implicit Runge-Kutta schemes applied to the bidomain equations

The partial differential equation part of the bidomain equations is discretized in time with fully implicit Runge–Kutta methods, and the resulting block systems are preconditioned with a block diagonal preconditioner. By studying the time-stepping operator in the proper Sobolev spaces, we show that the preconditioned systems have bounded condition numbers given that the Runge–Kutta scheme is A-stable and irreducible with an invertible coefficient matrix. A new proof of order optimality of the preconditioners for the one-leg discretization in time of the bidomain equations is also presented. The theoretical results are verified by numerical experiments. Additionally, the concept of weakly positive-definite matrices is introduced and analyzed. © 2010 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq Eq 27: 1290–1312, 2011

On block preconditioners for saddle point problems with singular or indefinite (1, 1) block

We discuss a class of preconditioning methods for the iterative solution of symmetric algebraic saddle point problems, where the (1, 1) block matrix may be indefinite or singular. Such problems may arise, e.g. from discrete approximations of certain partial differential equations, such as the Maxwell time harmonic equations. We prove that, under mild assumptions on the underlying problem, a class of block preconditioners (including block diagonal, triangular and symmetric indefinite preconditioners) can be chosen in a way which guarantees that the convergence rate of the preconditioned conjugate residuals method is independent of the discretization mesh parameter. We provide examples of such preconditioners that do not require additional scaling. Copyright © 2010 John Wiley & Sons, Ltd.

Acquired Clustering Properties and Solution of Certain Saddle Point Systems

Many mathematical models involve flow equations characterized by nonconstant viscosity, and a Stokes-type problem with variable viscosity coefficient arises. Appropriate block diagonal preconditioners for the resulting algebraic saddle point linear system produce well-clustered spectra, except for a few interior isolated eigenvalues which may tend to approach zero. These outliers affect the convergence of Krylov subspace system solvers, causing a possibly long stagnation phase. In this paper we characterize the influence of the spectral properties of the preconditioner on the final spectrum of the saddle point matrix by providing accurate spectral intervals depending on the involved operators. Moreover, we suggest that the stagnation phase may be completely eliminated by means of an augmentation procedure, where approximate spectral eigenspace information can be injected. We show that the modifications to the original code are minimal and can be easily implemented. Numerical experiments confirm our findings.

Preconditioning Stochastic Galerkin Saddle Point Systems

Mixed finite element discretizations of deterministic second-order elliptic PDEs lead to saddle point systems for which the study of iterative solvers and preconditioners is mature. Galerkin approximation of solutions of stochastic second-order elliptic PDEs, which couple standard mixed finite element discretizations in physical space with global polynomial approximation on a probability space, also give rise to linear systems with familiar saddle point structure. For stochastically nonlinear problems, the solution of such systems presents a serious computational challenge. The blocks are sums of Kronecker products of pairs of matrices associated with two distinct discretizations, and the systems are large, reflecting the curse of dimensionality inherent in most stochastic approximation schemes. Moreover, for the problems considered herein, the leading blocks of the saddle point matrices are block-dense, and the cost of a matrix vector product is nontrivial. We implement a stochastic Galerkin discretization for the steady-state diffusion problem written as a mixed first-order system. The diffusion coefficient is assumed to be a lognormal random field, approximated via a nonlinear function of a finite number of Gaussian random variables. We study the resulting saddle point systems and investigate the efficiency of block-diagonal preconditioners of Schur complement and augmented type for use with the minimal residual method (MINRES). By introducing so-called Kronecker product preconditioners, we improve the robustness of cheap, mean-based preconditioners with respect to the statistical properties of the stochastically nonlinear diffusion coefficients.

Spectral Analysis of Saddle Point Matrices with Indefinite Leading Blocks

We provide eigenvalue intervals for symmetric saddle point and regularized saddle point matrices in the case where the (1,1) block may be indefinite. These generalize known results for the definite (1,1) case. We also study the spectral properties of the equivalent augmented formulation, which is an alternative to explicitly dealing with the indefinite (1,1) block. Such an analysis may be used to assess the convergence of suitable Krylov subspace methods. We conclude with spectral analyses of the effects of common block-diagonal preconditioners.

Efficient monolithic simulation techniques for the stationary Lattice Boltzmann equation on general meshes

In this paper, we present special discretization and solution techniques for the numerical simulation of the Lattice Boltzmann equation (LBE). In Hübner and Turek (Computing, 81:281–296, 2007), the concept of the generalized mean intensity had been proposed for radiative transfer equations which we adapt here to the LBE, treating it as an analogous (semi-discretized) integro-differential equation with constant characteristics. Thus, we combine an efficient finite difference-like discretization based on short-characteristic upwinding techniques on unstructured, locally adapted grids with fast iterative solvers. The fully implicit treatment of the LBE leads to nonlinear systems which can be efficiently solved with the Newton method, even for a direct solution of the stationary LBE. With special exact preconditioning by the transport part due to the short-characteristic upwinding, we obtain an efficient linear solver for transport dominated configurations (macroscopic Stokes regime), while collision dominated cases (Navier-Stokes regime for larger Re numbers) are treated with a special block-diagonal preconditioning. Due to the new generalized equilibrium formulation (GEF) we can combine the advantages of both preconditioners, i.e. independence of the number of unknowns for convection-dominated cases with robustness for stiff configurations. We further improve the GEF approach by using hierarchical multigrid algorithms to obtain grid-independent convergence rates for a wide range of problem parameters, and provide representative results for various benchmark problems. Finally, we present quantitative comparisons between a highly optimized CFD-solver based on the Navier-Stokes equation (FeatFlow) and our new LBE solver (FeatLBE).

On parameterized block triangular preconditioners for generalized saddle point problems

In this paper, we discuss two classes of parameterized block triangular preconditioners for the generalized saddle point problems. These preconditioners generalize the common block diagonal and triangular preconditioners. We will give distributions of the eigenvalues of the preconditioned matrix and provide estimates for the interval containing the real eigenvalues. Numerical experiments of a model Stokes problem are presented.

$H( {div})$ preconditioning for a mixed finite element formulation of the diffusion problem with random data

We study $H(\mathrm {div})$ preconditioning for the saddle-point systems that arise in a stochastic Galerkin mixed formulation of the steady-state diffusion problem with random data. The key ingredient is a multigrid V-cycle for an $H(\mathrm {div})$ operator with random weight function acting on a certain tensor product space of random fields with finite variance. We build on the Arnold-Falk-Winther multigrid algorithm presented in 1997 by varying the spatial discretization from grid to grid whilst keeping the stochastic discretization fixed. We extend the deterministic analysis to accommodate the modified $H(\mathrm {div})$ operator and establish spectral equivalence bounds with a new multigrid V-cycle operator that are independent of the spatial and stochastic discretization parameters. We implement multigrid within a block-diagonal preconditioner for the full saddle-point problem, derive eigenvalue bounds for the preconditioned system matrices and investigate the impact of all the discretization parameters on the convergence rate of preconditioned minres.

A parallel computational model for sensitivity analysis in optimization for robustness

We present an efficient method for computing, in a parallel distributed environment, gradients of functionals depending on the solution sensitivities of dynamical systems described by ordinary differential equations. This work was motivated by the need to compute cost function gradients for dynamically constrained optimization for robustness, i.e. the problem of finding the model parameters of a given dynamical system that lead to minimum solution sensitivity. The proposed approach for this particular second-order sensi-tivity problem falls into the class of the so-called ‘adjoint-over-forward’ methods and is based on solving the continuous forward sensitivity equations (for evaluating the cost functional) and their continuous adjoint systems (for evaluation of the gradient of the cost functional). Efficiency is ensured by: (i) a suitable distribution over processors of the individual sensitivity systems, both forward and adjoint so that inter-process communication is minimized; and (ii) use of an iterative solver combined with a block-diagonal preconditioner for the solution of the linear systems arising in the implicit integration of the resulting ordinary differential equations (ODE) systems. The proposed algorithm was implemented as an extension of the cvodes solver in sundials [P. Brown, K. Grant, A. Hindmarsh, S. Lee, R.S.D. Shumaker, and C. Woodward. SUNDIALS: SUite of Nonlinear and DIfferential/ALgebraic equation Solvers, ACM Trans. Math. Software, 31(3) (2005), pp. 363–396], but it can be used in conjunction with any sensitivity-enabled ODE integrator that provides adjoint sensitivity capabilities and (if based on implicit integration methods) support for iterative linear algebra.

Efficient preconditioning for the discontinuous Galerkin finite element method by low-order elements

We derive and analyze a block diagonal preconditioner for the linear problems arising from a discontinuous Galerkin finite element discretization. The method can be applied to second-order self-adjoint elliptic boundary value problems and exploits the natural decomposition of the discrete function space into a global low-order subsystem and components of higher polynomial degree. Similar to results for the p-version of the conforming FEM, we prove for the interior penalty discontinuous Galerkin discretization that the condition number of the preconditioned system is uniformly bounded with respect to the mesh size of the triangulation. Numerical experiments demonstrate the performance of the method.

Guidance for Choosing Multigrid Preconditioners for Systems of Elliptic Partial Differential Equations

This paper presents some guidance for choosing multigrid preconditioners for systems of partial differential equations (PDEs). Rather than developing a black-box algorithm for arbitrary systems of PDEs, the strategy in this paper is to classify systems with multigrid preconditioners. From this classification, a new preconditioner is developed, and it is shown to be suitable for a particular class of strongly cross-coupled systems, solving the whole cross-coupled system almost as effectively as solving just the diagonal system. Also, this paper gives an explanation of why block diagonal preconditioners are somewhat effective for certain strongly cross-coupled systems that at first glance appear to be intractable for multigrid preconditioning.