CG Iteration Research Articles

Second‐order accurate hierarchical approximate factorizations for solving sparse linear systems

AbstractWe describe a second‐order accurate approach to sparsifying the off‐diagonal matrix blocks in the hierarchical approximate factorization methods for solving sparse linear systems. These methods repeatedly sparsify the fill‐in matrix blocks that arise in block Gaussian elimination, to compute an approximate factorization of the given matrix, assuming that the fill‐in blocks are low‐rank. The factorization is then used for preconditioning in a Krylov subspace method, such as the conjugate gradient method (CG), BiCGSTAB or GMRES. However, to achieve fast convergence on ill‐conditioned systems, sparsifications can only introduce a small error, in which case they may inefficiently restore sparsity, and consequently the factorization can take a lot of time to compute. In the novel approach to sparsification, the 2‐norm of the incurred error in the sparsification of a matrix block is squared compared to previous approaches, with no additional computations. We incorporate the new approach into the recent sparsified nested dissection algorithm and test it on a wide range of symmetric positive definite problems. The new approach halves the number of CG iterations needed for convergence, significantly improving the overall performance of the algorithm. Our approach can be incorporated into other solvers that exploit the low‐rank property of matrix blocks.

On the generalized AOR and CG iteration methods for a class of block two-by-two linear systems

On the generalized AOR and CG iteration methods for a class of block two-by-two linear systems

Tensor method for optimal control problems constrained by fractional three‐dimensional elliptic operator with variable coefficients

AbstractWe introduce the tensor numerical method for solving optimal control problems that are constrained by fractional two‐ (2D) and three‐dimensional (3D) elliptic operators with variable coefficients. We solve the governing equation for the control function which includes a sum of the fractional operator and its inverse, both discretized over large 3D spacial grids. Using the diagonalization of the arising matrix‐valued functions in the eigenbasis of the one‐dimensional Sturm–Liouville operators, we construct the rank‐structured tensor approximation with controllable precision for the discretized fractional elliptic operators and the respective preconditioner. The right‐hand side in the constraining equation (the optimal design function) is supposed to be represented in a form of a low‐rank canonical tensor. Then the equation for the control function is solved in a tensor structured format by using preconditioned CG iteration with the adaptive rank truncation procedure that also ensures the accuracy of calculations, given an ‐threshold. This method reduces the numerical cost for solving the control problem to (plus the quadratic term with a small weight), which outperforms traditional approaches with complexity in the 3D case. The storage for the representation of all 3D nonlocal operators and functions involved is also estimated by . This essentially outperforms the traditional methods operating with fully populated matrices and vectors in . Numerical tests for 2D/3D control problems indicate the almost linear complexity scaling of the rank truncated preconditioned conjugate gradient iteration in the univariate grid size n.

Accurate error estimation in CG

In practical computations, the (preconditioned) conjugate gradient (P)CG method is the iterative method of choice for solving systems of linear algebraic equations Ax = b with a real symmetric positive definite matrix A. During the iterations, it is important to monitor the quality of the approximate solution xk so that the process could be stopped whenever xk is accurate enough. One of the most relevant quantities for monitoring the quality of xk is the squared A-norm of the error vector x − xk. This quantity cannot be easily evaluated; however, it can be estimated. Many of the existing estimation techniques are inspired by the view of CG as a procedure for approximating a certain Riemann–Stieltjes integral. The most natural technique is based on the Gauss quadrature approximation and provides a lower bound on the quantity of interest. The bound can be cheaply evaluated using terms that have to be computed anyway in the forthcoming CG iterations. If the squared A-norm of the error vector decreases rapidly, then the lower bound represents a tight estimate. In this paper, we suggest a heuristic strategy aiming to answer the question of how many forthcoming CG iterations are needed to get an estimate with the prescribed accuracy. Numerical experiments demonstrate that the suggested strategy is efficient and robust.

A note on multigrid preconditioning for fractional PDE-constrained optimization problems

In this note we present a multigrid preconditioning method for solving quadratic optimization problems constrained by a fractional diffusion equation. Multigrid methods within the all-at-once approach to solve the first order optimality Karush–Kuhn–Tucker (KKT) systems are widely popular, but their development have relied on the underlying systems being sparse. On the other hand, for most discretizations, the matrix representation of fractional operators is expected to be dense. We develop a preconditioning strategy for our problem based on a reduced approach, namely we eliminate the state constraint using the control-to-state map. Our multigrid preconditioning approach shows a dramatic reduction in the number of CG iterations. We assess the quality of preconditioner in terms of the spectral distance. Finally, we provide a partial theoretical analysis for this preconditioner, and we formulate a conjecture which is clearly supported by our numerical experiments.

Adjustable robust optimization in enabling optimal day-ahead economic dispatch of CCHP-MG considering uncertainties of wind-solar power and electric vehicle

At present, electric vehicles (EVs), small-scale wind power, and solar power have been increasingly integrated into modern power system via the combined cooling heating and power based microgrid (CCHP-MG). However, inside the microgrid the uncertainties of EVs charging, wind power, and solar power significantly impact the economy of CCHP-MG operation. Therefore to improve the economy deteriorated by the uncertainties, this paper presents a two-stage adjustable robust optimization to achieve the minimal operational cost for CCHP-MG. Before the realizations of the uncertainties, the day-ahead stage as the first stage decides an operational strategy that can withstand the worst-case uncertainties. As long as the uncertainties are observed, the real-time stage as the second stage adjusts the operational units to compensate the errors caused by the day-ahead operational strategy. Due to the difficulties of the model solution, this paper further adopts the duality theory, Big-M method, and column-and-constraint generation (C & CG) decomposition to convert the model into two tractable mixed integer linear programming (MILP) problems. Further, C & CG iteration algorithm is also employed to solve the MILPs, which can ultimately provide an optimal economic day-ahead dispatch strategy capable of handling uncertainties. The experimental results demonstrate the effectiveness of the presented approach.

An Algebraic Sparsified Nested Dissection Algorithm Using Low-Rank Approximations

We propose a new algorithm for the fast solution of large, sparse, symmetric positive-definite linear systems, spaND (sparsified Nested Dissection). It is based on nested dissection, sparsification, and low-rank compression. After eliminating all interiors at a given level of the elimination tree, the algorithm sparsifies all separators corresponding to the interiors. This operation reduces the size of the separators by eliminating some degrees of freedom but without introducing any fill-in. This is done at the expense of a small and controllable approximation error. The result is an approximate factorization that can be used as an efficient preconditioner. We then perform several numerical experiments to evaluate this algorithm. We demonstrate that a version using orthogonal factorization and block-diagonal scaling takes fewer CG iterations to converge than previous similar algorithms on various kinds of problems. Furthermore, this algorithm is provably guaranteed to never break down and the matrix stays symmetric positive-definite throughout the process. We evaluate the algorithm on some large problems show it exhibits near-linear scaling. The factorization time is roughly $\mathcal{O}$(N), and the number of iterations grows slowly with N.

Implementation of the efficient communication layer for the highly parallel total FETI and hybrid total FETI solvers

This paper describes the implementation, performance, and scalability of our communication layer developed for Total FETI (TFETI) and Hybrid Total FETI (HTFETI) solvers. HTFETI is based on our variant of the Finite Element Tearing and Interconnecting (FETI) type domain decomposition method. In this approach a small number of neighboring subdomains is aggregated into clusters, which results in a smaller coarse problem. To solve the original problem TFETI method is applied twice: to the clusters and then to the subdomains in each cluster. The current implementation of the solver is focused on the performance optimization of the main CG iteration loop, including: implementation of communication hiding and avoiding techniques for global communications; optimization of the nearest neighbor communication - multiplication with a global gluing matrix; and optimization of the parallel CG algorithm to iterate over local Lagrange multipliers only. The performance is demonstrated on a linear elasticity 3D cube and real world benchmarks.

Time-Extrapolation Algorithm (TEA) for Linear Parabolic Problem

The fast solutions of Crank-Nicolson scheme on quasi-uniform mesh for parabolic problems are discussed. First, to decrease regularity requirements of solutions, some new error estimates are proved. Second, we analyze the two characteristics of parabolic discrete scheme, and find that the efficiency of Multigrid Method (MG) isgreatly reduced. Numerical experiments compare the efficiency of Direct Conjugate Gradient Method (DCG) and Extrapolation Cascadic Multigrid Method (EXCMG). Last, we propose a TimeExtrapolation Algorithm (TEA), which takes a linear combination of previous several level solutions as good initial values to accelerate the rate of convergence. Some typical extrapolation formulas are compared numerically. And we find that under certain accuracy requirement, the CG iteration count for the 3-order and 7-level extrapolation formula is about 1/3 of that of DCG’s. Since the TEA algorithm is independent of the space dimension, it is still valid for quasi-uniform meshes. As only the finest grid is needed, the proposed method is regarded very effective for nonlinear parabolic problems.

三次元ENRICHED FREE MESH METHODの開発とき裂解析への適用

In this paper, we describe a method for three-dimensional high accurate analysis of a crack included in a large-scale structure. The Enriched Free Mesh Method (EFMM) is a method for improving the accuracy of the Free Mesh Method (FMM), which is a kind of meshless method. First, we developed an algorithm of the three-dimensional EFMM. The elastic problem was analyzed using the EFMM and we find that its accuracy compares advantageously with the FMM, and the number of CG iterations is smaller. Next, we developed a method for calculating the stress intensity factor by employing the EFMM. The structure with a crack was analyzed using the EFMM, and the stress intensity factor was calculated by the developed method. The analysis results were very well in agreement with reference solution. It was shown that the proposed method is very effective in the analysis of the crack included in a large-scale structure.

A CG-FFT approach to the solution of a stress-velocity formulation of three-dimensional elastic scattering problems

In this paper we introduce a Conjugate Gradient Fast Fourier Transform (CG-FFT) scheme for the numerical solution of the integral equation formulating three-dimensional elastic scattering problems. The formulation is in terms of the stress tensor and particle velocities as the unknown field variables. In contrast with the formulation based on particle displacements, this approach leads to integral representations that do not involve derivatives of the unknown fields, thus resulting in simplified and more stable numerics. The numerical procedure is based on suitable quadrature formulas that provide (second order) accurate approximations while retaining the convolution nature of the relevant integrals that make them amenable to efficient evaluation via FFTs. The scheme is further improved through the introduction of (approximation-based) pre-conditioners that are shown to accelerate the convergence of the CG iterations. Numerical results are presented that demonstrate the accuracy and efficiency of the proposed methodology.

IBLU decompositions based on Padé approximants

AbstractIn this paper, we introduce a new class of frequency‐filtering IBLU decompositions that use continued‐fraction approximation for the diagonal blocks. This technique allows us to construct efficient frequency‐filtering preconditioners for discretizations of elliptic partial differential equations on domains with non‐trivial geometries. We prove theoretically for a class of model problems that the application of the proposed preconditioners leads to a convergence rate of up to 1−O(h1/4) of the CG iteration. Copyright © 2008 John Wiley & Sons, Ltd.

An adaptive pruning algorithm for the discrete L-curve criterion

We describe a robust and adaptive implementation of the L-curve criterion. The algorithm locates the corner of a discrete L-curve which is a log–log plot of corresponding residual norms and solution norms of regularized solutions from a method with a discrete regularization parameter (such as truncated SVD or regularizing CG iterations). Our algorithm needs no predefined parameters, and in order to capture the global features of the curve in an adaptive fashion, we use a sequence of pruned L-curves that correspond to considering the curves at different scales. We compare our new algorithm to existing algorithms and demonstrate its robustness by numerical examples.

Möbius Fermions: Improved Domain Wall Chiral Fermions

A new class of domain wall fermions is defined that interpolates between Shamir's and Bori\c{c}i's form without increasing the number of Dirac applications per CG iteration. This class represents a full (real) M\"obius transformation of the Wilson kernel. Simulations on quenched Wilson lattices with $\beta = 6.0$ show that the number of lattice sites ($L_s$) in the fifth dimension can be reduced by a factor of 2 or more at fixed value of chiral symmetry violations measured by the residual mass ($m_{res}$).

An iterative GMRES-based boundary element solver for acoustic scattering

High-frequency scattering of complex structures is studied using a new variant of the generalized minimum residual method (GMRES) along with an appropriate preconditioning, which was originally developed for iterative solutions in electrodynamics [Comparison of GMRES and CG Iterations on the Normal Form of Magnetic Field Integral Equation, submitted]. The starting point is a self-adjoint formulation of the Helmholtz integral equation for scattering. Three iterative methods, the Jacobi iteration with overrelaxation and two variants of the GMRES are presented, and the advantages of the improved GMRES are discussed. The GMRES is applied to the scattering of a plane wave from a cylinder-like, rigid structure comprising several ten thousands of boundary elements. Fast convergence of the iteration process is observed for all investigated cases. The relative error decreases rapidly and becomes smaller than the discretization error after a few iteration steps. The GMRES solver is only weakly affected by internal resonances, but could be combined with a dual-layer CHIEF approach. All computations are carried out on regular personal computers, even for the boundary element model with about 48,000 elements.

Algebraic multicolor ordering for parallelized ICCG solver in finite-element analyses

Proposes a new black-box-type parallel processing method for the incomplete Cholesky conjugate gradient (ICCG) solver. The new method is based on a multicolor ordering concept and an automatic reordering process in the solver. Parallel performance is evaluated in the context of three-dimensional finite edge-element eddy-current analysis. The proposed method attains high parallelism with a small increase in CG iterations and achieves high parallel performance.

Automatic Preconditioning by Limited Memory Quasi-Newton Updating

This paper proposes a preconditioner for the conjugate gradient method (CG) that is designed for solving systems of equations Ax=bi with different right-hand-side vectors or for solving a sequence of slowly varying systems Ak x = bk. The preconditioner has the form of a limited memory quasi-Newton matrix and is generated using information from the CG iteration. The automatic preconditioner does not require explicit knowledge of the coefficient matrix A and is therefore suitable for problems where only products of A times a vector can be computed. Numerical experiments indicate that the preconditioner has most to offer when these matrix-vector products are expensive to compute and when low accuracy in the solution is required. The effectiveness of the preconditioner is tested within a Hessian-free Newton method for optimization and by solving certain linear systems arising in finite element models.

Indefinitely preconditioned inexact Newton method for large sparse equality constrained non-linear programming problems

An inexact Newton algorithm for large sparse equality constrained non-linear programming problems is proposed. This algorithm is based on an indefinitely preconditioned smoothed conjugate gradient method applied to the linear KKT system and uses a simple augmented Lagrangian merit function for Armijo type stepsize selection. Most attention is devoted to the termination of the CG method, guaranteeing sufficient descent in every iteration and decreasing the number of required CG iterations, and especially, to the choice of a suitable preconditioner. We investigate four preconditioners, which have 2 × 2 block structure, and prove theoretically their good properties. The efficiency of the inexact Newton algorithm, together with a comparison of various preconditioners and strategies, is demonstrated by using a large collection of test problems. © 1998 John Wiley & Sons, Ltd.

Adaptive Multilevel Techniques for Mixed Finite Element Discretizations of Elliptic Boundary Value Problems

We consider mixed finite element discretizations of linear second-order elliptic boundary value problems with respect to an adaptively generated hierarchy of possibly highly nonuniform simplicial triangulations. By a well-known postprocessing technique the discrete problem is equivalent to a modified nonconforming discretizationwhich is solved by preconditioned CG iterations using a multilevel preconditioner in the spirit of Bramble, Pasciak, and Xu designed for standard nonconforming approximations. Local refinement of the triangulations is based on an a posteriori error estimator which can be easily derived from superconvergence results. The performance of the preconditioner and the error estimator is illustrated by several numerical examples.

FACTORIZED SPARSE APPROXIMATE INVERSE PRECONDITIONING II: SOLUTION OF 3D FE SYSTEMS ON MASSIVELY PARALLEL COMPUTERS

An iterative method for solving large linear systems with sparse symmetric positive definite matrices on massively parallel computers is suggested. The method is based on the Factorized Sparse Approximate Inverse (FSAI) preconditioning of ‘parallel’ CG iterations. Efficiency of a concurrent implementation of the FSAI-CG iterations is analyzed for a model hypercube, and an estimate of the optimal hypercube dimension is derived. For finite element applications, two strategies for selecting the preconditioner sparsity pattern are suggested. A high convergence rate of the resulting iterations is demonstrated numerically for the 3D equilibrium equations for linear elastic orthotropic materials approximated using both h- and p-versions of the FEM.