Block Jacobi Iteration Research Articles

An Adaptive Two-Grid Solver for DPG Formulation of Compressible Navier–Stokes Equations in 3D

Abstract We present an overlapping domain decomposition iterative solver for linear systems resulting from the discretization of compressible viscous flows with the Discontinuous Petrov–Galerkin (DPG) method in three dimensions. It is a two-grid solver utilizing the solution on the auxiliary coarse grid and the standard block-Jacobi iteration on patches of elements defined by supports of the coarse mesh base shape functions. The simple iteration defined in this way is used as a preconditioner for the conjugate gradient procedure. Theoretical analysis indicates that the condition number of the preconditioned system should be independent of the actual finite element mesh and the auxiliary coarse mesh, provided that they are quasiuniform. Numerical tests confirm this result. Moreover, they show that presence of strongly flattened or elongated elements does not slow the convergence. The finite element mesh is subject to adaptivity, i.e. dividing the elements with large errors until a required accuracy is reached. The auxiliary coarse mesh is adjusting to the nonuniform actual mesh.

AN IMPLICIT BLOCK JACOBI APPROACH FOR HIGH-ORDER FLUX RECONSTRUCTION METHOD

Recently, the flux reconstruction (FR) method has attracted more and more attentions for its simplicity and generality. However, it is still computationally expensive and time consuming when simulating the complex flow problems by FR method. There is a huge demand for developing appropriate efficient implicit solvers and parallel computing techniques for FR. This paper proposes an implicit high-order flux reconstruction solver on GPU platform based on the block Jacobi iteration method. As it is inefficient to solve the large global linear system resulting from spatial and implicit temporal discretization of FR directly. A block Jacobi approach is used to change the characteristics of the lift-hand matrix of the global linear system and this avoids the dependence of neighboring elements. Therefore, only the diagonal blocks of global matrix need to be stored and calculated. Then, the problem of solving the huge global linear system is transformed into solving a series of local linear equations simultaneously. Finally, these small local linear equations would be solved by the LU decomposition method in parallel on GPU platforms. Two typical cases, including subsonic flows over a bump and a NACA0012 airfoil, were simulated and compared with the multi-grid explicit Runge-Kutta scheme. The numerical results demonstrated that the present implicit method can reduce the iterations significantly. Meanwhile, the implicit solver has shown at least 10x speedup over the multi-grid Runge-Kutta scheme in all cases.

Partition-based multi-agent optimization in the presence of lossy and asynchronous communication

We address the problem of multi-agent partition-based convex optimization which arises, for example, in robot localization problems and in regional state estimation in smart grids. More specifically, the global cost function is the sum of locally coupled cost functions that depend only on each agent variables and their neighbors’ variables. Inspired by a generalized gradient descent strategy, namely the Block Jacobi iteration, we propose an algorithm amenable to a scalable distributed implementation, i.e., each agent eventually computes only the optimal values for its own variables via local communication with its neighbors. In particular, we provide sufficient conditions for global and semi-global exponential stability for the proposed algorithms even in the presence of lossy communications and asynchronous updates. The theoretical analysis relies on novel tools on Lyapunov theory based on separation of time scales and averaging theory for discrete-time systems. Finally, the proposed algorithm is numerically tested on the IEEE 123 nodes distribution feeder in the context of multi-area robust state estimation of smart grids in the presence of measurement outliers.

An implicit block ILU smoother for preconditioning of Newton–Krylov solvers with application in high-order stabilized finite-element methods

This paper presents an efficient and highly-parallelizable preconditioning technique for Newton–Krylov solvers. The proposed method can be viewed as a generalization of the implicit line smoothing technique by extending the groups of implicitly-solved unknowns from lines to blocks. The blocks are formed by partitioning the computational domain such that the strong connections between unknowns are not broken by the partition boundaries. The ILU algorithm is used to obtain an approximate (or exact) factorization for each block. Then, a block-Jacobi iteration is formulated in which the degrees of freedom within the blocks are solved implicitly. To stabilize the iterations for high-CFL systems, a dual-CFL strategy, with a lower CFL in the preconditioner matrix, is developed. The performance of the proposed method as a linear preconditioner is demonstrated for second- and third-order steady-state solutions of Reynolds-Averaged Navier–Stokes (RANS) equations on the NASA Common Research Model (CRM), including the high-lift configuration. For the studied test cases, it is shown that in comparison with the traditional ILU(k) method, the proposed preconditioner requires significantly less memory and it can result in notably faster solutions.

An introduction to multi-trace formulations and associated domain decomposition solvers

Multi-trace formulations (MTFs) are based on a decomposition of the problem domain into subdomains, and thus domain decomposition solvers are of interest. The fully rigorous mathematical MTF can however be daunting for the non-specialist. The first aim of the present contribution is to provide a gentle introduction to MTFs. We introduce these formulations on a simple model problem using concepts familiar to researchers in domain decomposition. This allows us to get a new understanding of MTFs and a natural block Jacobi iteration, for which we determine optimal relaxation parameters. We then show how iterative multi-trace formulation solvers are related to a well known domain decomposition method called optimal Schwarz method: a method which used Dirichlet to Neumann maps in the transmission condition. We finally show that the insight gained from the simple model problem leads to remarkable identities for Calderón projectors and related operators, and the convergence results and optimal choice of the relaxation parameter we obtained is independent of the geometry, the space dimension of the problem, and the precise form of the spatial elliptic operator, like for optimal Schwarz methods. We illustrate our analysis with numerical experiments.

The Parallel Algorithm for Solving Toeplitz Systems

This paper studies a parallel algorithm for real Toeplitz systems, which is proposed based on the block Jacobi iteration and GMRES method. The algorithm has the advantage of less float operations, fast convergence speed and especially suitable for parallel computating. In this paper, we first use the block Jacobi iterative method to obtain the iterative process, and then the GMRES method is nested to obtain the iterative sequences {xk}. Therefore, the parallel algorithm for solving symmetric positive definite Toeplitz systems is constructed. The convergence of the algorithm is also discussed simply in the paper. At the end, we give some numerical examples to illustrate the effectiveness of the parallel algorithm.

Generalized polynomial chaos decomposition and spectral methods for the stochastic Stokes equations

In this paper we propose and analyze a high order numerical method to solve the Stokes equations with random coefficients. A stochastic Galerkin approach, based on a finite dimensional Karhunen–Loève decomposition technique for the stochastic inputs, is used to reduce the original stochastic Stokes equations into a set of deterministic equations for the expansion coefficients. Then a PN×PN-2 spectral method, together with a block Jacobi iteration is applied to solve the resulting problem. We establish the well-posedness of the weak formulation and its discrete counterpart. Moreover, we provide a rigorous convergence analysis and demonstrate exponential convergence with respect to the degrees of the polynomials chaos expansion used for the approximation in the random direction. Finally, a series of numerical tests is presented to support the theoretical results.

Optimal Parameters for 2-cyclic AOR

Assume A ∈ C n × n is a 2-cyclic consistently ordered matrix and J is denoted as its associated block Jacobi iteration matrix. We consider the 2-cyclic AOR method for solving the consistent linear systems Ax = b . In the case that σ ( J 2 ) is either nonnegative or nonpositive, we give detailed discussion and derive definite expressions on optimal parameters and spectral radius by efficient method. Moreover, we give some numerical examples.

On successive‐overrelaxation acceleration of the Hermitian and skew‐Hermitian splitting iterations

AbstractWe further generalize the technique for constructing the Hermitian/skew‐Hermitian splitting (HSS) iteration method for solving large sparse non‐Hermitian positive definite system of linear equations to the normal/skew‐Hermitian (NS) splitting obtaining a class of normal/skew‐Hermitian splitting (NSS) iteration methods. Theoretical analyses show that the NSS method converges unconditionally to the exact solution of the system of linear equations. Moreover, we derive an upper bound of the contraction factor of the NSS iteration which is dependent solely on the spectrum of the normal splitting matrix, and is independent of the eigenvectors of the matrices involved. We present a successive‐overrelaxation (SOR) acceleration scheme for the NSS iteration, which specifically results in an acceleration scheme for the HSS iteration. Convergence conditions for this SOR scheme are derived under the assumption that the eigenvalues of the corresponding block Jacobi iteration matrix lie in certain regions in the complex plane. A numerical example is used to show that the SOR technique can significantly accelerate the convergence rate of the NSS or the HSS iteration method. Copyright © 2006 John Wiley & Sons, Ltd.

Robust Distributed Estimation Using the Embedded Subgraphs Algorithm

We propose a new iterative, distributed approach for linear minimum mean-square-error (LMMSE) estimation in graphical models with cycles. The embedded subgraphs algorithm (ESA) decomposes a loopy graphical model into a number of linked embedded subgraphs and applies the classical parallel block Jacobi iteration comprising local LMMSE estimation in each subgraph (involving inversion of a small matrix) followed by an information exchange between neighboring nodes and subgraphs. Our primary application is sensor networks, where the model encodes the correlation structure of the sensor measurements, which are assumed to be Gaussian. The resulting LMMSE estimation problem involves a large matrix inverse, which must be solved in-network with distributed computation and minimal intersensor communication. By invoking the theory of asynchronous iterations, we prove that ESA is robust to temporary communication faults such as failing links and sleeping nodes, and enjoys guaranteed convergence under relatively mild conditions. Simulation studies demonstrate that ESA compares favorably with other recently proposed algorithms for distributed estimation. Simulations also indicate that energy consumption for iterative estimation increases substantially as more links fail or nodes sleep. Thus, somewhat surprisingly, sensor network energy conservation strategies such as low-powered transmission and aggressive sleep schedules could actually prove counterproductive. Our results can be replicated using MATLAB code from www.dsp.rice.edu/software

Analysis of a preconditioned GMRES solver for a nonsymmetric SUPG discretization of the compressible Navier-Stokes equations

A preconditioner for a system of linear equations resulting from finite element discretization of the compressible Navier-Stokes equations in conservative variables is presented. This system is nonsymmetric and the construction of a preconditioner is based on a symmetric preconditioner for a symmetric formulation of flows in entropy variables. The algorithm is a modification of a block Jacobi iteration. It is combined with a GMRES procedure resulting in an effective equation solver. The solver is tested on a number of numerical examples.

On the modified preconditioned simultaneous displacement (m-psd) method

In this paper, we derive a new functional equation which relates the eigenvalues of a weakly cyclic matrix of index p block Jacobi iteration matrix to the eigenvalues of its associated Modified Preconditioned Simultaneous Displacement (M-PSD) iteration matrix. The convergence bound is established for the M-PSD method when coefficient matrix A is a p-cyclic.

A generalized Schwarz splitting method based on Hermite collocation for elliptic boundary value problems

The Schwarz alternating method (SAM) coupled with various numerical discretization schemes has already been established as an efficient alternative for solving differential equations on various parallel machines. In this paper we consider an extension of SAM (generalized Schwarz splitting—GSS) for solving elliptic boundary value problems with generalized interface conditions that depend on a parameter that might differ in each overlapping region (Tang, 1992). The GSS considered in this paper is coupled with the cubic Hermite collocation discretization scheme (Mitchell et al., 1985) to solve the corresponding boundary value problem in each subdomain. The main objective of this study is the mathematical analysis of the iterative solution of the so-called enhanced GSS collocation discrete matrix equation corresponding to a model elliptic boundary value problem defined on a rectangle. This analysis is based on the spectral properties of the associated enhanced block Jacobi iteration matrix which are explicitly derived. We were able to determine analytically the domain of convergence of the one-parameter GSS scheme for both one-dimensional and two-dimensional problems. In addition sets of optimal multi-parameter GSS schemes have been determined in the case of one-dimensional problems. The analyzed GSS scheme is applied to a number of model elliptic boundary value problems to verify the theoretical results and compare the convergence rates of the SAM and GSS schemes with minimum and maximum overlap. Finally, the same GSS scheme was applied to general elliptic boundary value problems utilizing the optimal interface parameters derived for a model problem. The numerical data obtained indicate that the computational behavior of the optimal GSS schemes determined holds for general elliptic operators.

On the convergence domains of the p-cyclic SOR

For the solution of the linear system Ax = b, where A is block p- cyclic, the block SOR iterative method is to be considered. Suppose that the block Jacobi iteration matrix B, associated with A, has eigenvalues whose pth powers are all real of the same sign. The problem of the determination of the precise convergence domains of the SOR method in case A is also consistently ordered was solved by Hadjidimos, Li and Varga by using the Schur-Cohn algorithm. The same convergence domains were later recovered by other approaches too; specifically, Wild and Niethammer and also Noutsos, independently, used hypocycloidal curves. In this manuscript it is assumed that A is not consistently ordered but A T is. By using the Schur-Cohn algorithm we successfully determine, not only: (i) the precise SOR convergence domains, but also (ii) intervals for ϱ( B), the spectral radius of B, that directly imply that the optimal value of the SOR relaxation factor ω is equal to 1. In this work new results are obtained, some well-known ones are recovered or confirmed and a number of theoretical examples are investigated further. It is worth noting that among the new results, we derived something not quite expected; specifically, in many cases there exist pairs ( ϱ( B), ω) for which the SOR method associated with the matrix A we consider converges while the corresponding SOR for the p-cyclic consistently ordered matrix A T does not!

Block iterative methods for the nine-point approximation to the convection-diffusion equation

We study the numerical solution of a steady convection-diffusion equation in which the convection term is dominant. A nine-point fourth order difference scheme is examined and the block successive overrelaxation (BSOR) and the block alternating group explicit (BLAGE) iterative methods are considered for solving the sparse unsymmetric linear system. When the diffusion coefficient is small, the eigenvalues of the block Jacobi iteration matrix almost lie on the imaginary axis and using bounds for their eigenspectrum, we show that the line|ar system can be effectively solved by the block Gauss-Seidel method. The size of the linear system is reduced by the Strides of 3 algorithm and the BSOR and BLAGE methods are applied to the reduced system of linear equations. This leads to greater convergence for both methods. In addition, a great deal of parallelism can be achieved by using BLAGE.

Exact SOR convergence regions for a general class ofp-cyclic matrices

Linear systems whose associated block Jacobi iteration matrixB is weakly cyclic generated by the cyclic permutation σ = (σ1,σ2,..., σp) in the spirit of Li and Varga are considered. Regions of convergence for the corresponding blockp-cyclic SOR method are derived and the exact convergence domains for real spectra, σ(Bp), of the same sign are obtained. Moreover, analytical expressions for two special cases forp = 5 are given and numerical results are presented confirming the theory developed. The tools used for this work are mainly from complex analysis and extensive use of (asteroidal) hypocycloids in the complex plane is made to produce our results.

Optimum modified extrapolated jacobi method for some class of matrices

In this paper we study a two parametric linear stationary iterative method, called Modified Extrapolated Jacobi (MEJ) method, for solving linear systems Ax = b, where A is nonsingular consistently ordered 2-cyclic matrix. We give sufficient and necessary conditions for strong convergence of the MEJ method and we determine the optimum extrapolation parameters and the optimum spectral radius of it, in the case where all the eigenvalues of the block Jacobi iteration matrix associated with A are purely imaginary. In the last section, we compare the MEJ with other known methods.

On the Iterative Solution of Hermite Collocation Equations

Collocation methods based on bicubic Hermite piecewise polynomials have been proven to be effective techniques for solving general second order linear elliptic partial differential equations (PDEs) with mixed boundary conditions [ACM Trans. Math. Software, 11 (1985), pp. 379–412]. The corresponding system of discrete collocation equations is generally nonsymmetric and nondiagonally dominant. Methods for their iterative solution are not known and are currently solved using Gauss elimination with scaling and partial pivoting. Point iterative methods like those in ITPACK [Tech. Report CNA-216, Center for Numerical Analysis, Univ. of Texas at Austin, April 1988] do not converge even for the collocation equations obtained from the discretization of model PDE problems. The development of efficient iterative solvers for these collocation equations is necessary for the case of three-dimensional PDE problems and their parallel solution, since direct solvers tend to be space bound and their parallelization is difficult. In this paper block iterative methods are developed and analyzed for the collocation equations corresponding to elliptic PDEs defined on a rectangle and subject to uncoupled mixed boundary conditions. For these types of PDE problems certain boundary degrees of freedom of the collocation approximation can be predetermined symbolically [Houstis, Mitchell, and Rice]. The remaining equations are called “interior” collocation equations. The system of all discrete equations is referred to as “general” collocation equations. Papatheodorou [Math. Comp., 41 (1983), pp. 511-525] was first to determine the exact parameters of accelerated overrelaxation (AOR)-type iterative methods for the case of “interior” collocation equations associated with a model problem. This paper generalizes the results of Papatheodorou for the “interior” collocation equations and presents new results for a particular class of “general” collocation equations. Specifically, in the case of a model elliptic PDE problem with uncoupled mixed boundary conditions, analytic expressions are derived for the eigenvalues of the block Jacobi iteration matrix based on a new partitioning of the interior collocation matrix, and the optimal overrelaxation factors are determined for the block successive overrelaxation (SOR) iterative method. A number of numerical results are presented to verify the theoretical analysis of the block SOR method and to compare its convergence behavior with those of the block Jacobi, Gauss–Seidel and the optimal AOR of Papatheodorou. Furthermore, the authors compare the time and memory complexity of the block SOR, UNPACK Band GE, and generalized minimal residual (GMRES) mathematical software for solving the Hermite collocation equations obtained from the discretization of several PDE problems. The numerical results indicate that the block SOR is an efficient method for solving these equations.

On domains of superior convergence of the SSOR method to that of the SOR method

Let v denote the spectral radius of J A B , the block Jacobi iteration matrix. For the classes of (1) nonsingular M-matrices and (2) p-cyclic, p ⩾ 3, consistently ordered matrices, we study domains in the ( v, ω) plane when v < 1, where the block SSOR iteration method has at least as favorable an asymptotic rate of convergence as the block SOR method. Let L A ω and S A ω denote, respectively, the block SOR and SSOR iteration matrices. For the class of nonsingular M-matrices A, we determine conditions when the spectral radii satisfy ρ( S A ω) ⩽ ρ( L A ω) ∀0 < ω ⩽ 2 1 + v and ∀0 ⩽ v < 1 . Under these conditions we also show that the optimal SOR iteration parameter is ω b = 1. For the class of p-cyclic, p ⩾ 3, consistently ordered matrices A we determine for which ω's and v's, ρ( S A ω) < |ω − 1| [ ⩽ ρ( L A ω)]. Our investigations make use of the equality case in Wielandt's inequality between the spectral radii of a complex matrix and its nonnegative and irreducible majorizers and of Rouché's theorem for the location of zeros of complex functions.

Iterative Line Cubic Spline Collocation Methods for Elliptic Partial Differential Equations in Several Dimensions

This paper presents a new class of second- and fourth-order line cubic spline collocation methods (the LCSC methods) for multidimensional linear elliptic partial differential equations with no cross derivative terms. The LCSC methods approximate the differential operator along lines in each dimension independently and then combine the results into one large linear system. Expressed in terms of discretization stencils for the operator, these methods have nonzero entries only in the coordinate directions. The advantage of this approach is that the discretization is much simpler to derive and analyze. Further, iterative methods are easily applied to the resulting linear systems, especially on parallel computers. The disadvantage is that the resulting linear system is k times larger in k dimensions. Using the simplicity of the methods, iterative schemes are analyzed and formulated in order to solve the resulting LCSC linear systems in the case of Helmholtz problems. Block Jacobi, extrapolated Jacobi (EJ), and successive overrelaxation (SOR) iteration methods are analyzed with the rates of convergence and the optimum relaxation parameters determined. The simple structure of the linear system makes these methods particularly suitable for parallel computation. It is shown that the overall efficiency of the method is attractive in spite of involving such a large linear system. Experimental results presented here confirm the convergence results for both the discretization and iterative methods, and indicate that the convergence results hold for problems more general than Helmholtz problems.