Variable Preconditioning Research Articles

The generalized residual cutting method and its convergence characteristics

AbstractIterative methods and especially Krylov subspace methods (KSM) are a very useful numerical tool in solving for large and sparse linear systems problems arising in science and engineering modeling. More recently, the nested loop KSM have been proposed that improve the convergence of the traditional KSM. In this article, we review the residual cutting (RC) and the generalized residual cutting (GRC) that are nested loop methods for large and sparse linear systems problems. We also show that GRC is a KSM that is equivalent to Orthomin with a variable preconditioning. We use the modified Gram–Schmidt method to derive a stable GRC algorithm. We show that GRC presents a general framework for constructing a class of “hybrid” (nested) KSM based on inner loop method selection. We conduct numerical experiments using nonsymmetric indefinite matrices from a widely used library of sparse matrices that validate the efficiency and the robustness of the proposed methods.

A New Type of Variable Preconditioning for a Generalized Minimum Residual Scheme

Krylov linear equation solvers have been widely used on distributed-memory-type supercomputers. Krylov-type linear equation solvers are frequently applied using preconditioning techniques to achieve fast and robust computations. Flexible Krylov solvers that accept variable preconditioning have recently been developed, whereas conventional ones are invariable throughout the Krylov solver iterations. The generalized minimal residual recursive method (GMRESR) is outstandingly unique because it is combined with a preconditioning update scheme based on Eirola and Nevanlinna’s rank-one update scheme. Moreover, the Broyden–Fletcher–Goldfarb–Shanno scheme (BFGS) provides another means to update preconditioning matrices. In this study, we applied BFGS to GMRESR as an alternative means of updating the preconditioning matrices and then measured the performance. As a result, the newly developed method exhibited an approximately 10-fold better convergence than Eirola and Nevanlinna’s approach.

Variable preconditioning for strongly nonlinear elliptic problems

Variable preconditioning has earlier been developed as a realization of quasi-Newton methods for elliptic problems with uniformly bounded nonlinearities. This paper presents a generalization of this approach to strongly nonlinear problems, first on an operator level, then for elliptic problems allowing power order growth of nonlinearities. Numerical tests reinforce the convergence results.

Flexible and deflated variants of the block shifted GMRES method

The solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications. In this paper we introduce new flexible and deflated variants of the shifted block GMRES method for this problem class. The proposed methods solve the whole sequence of linear systems simultaneously, detecting effectively the linear systems convergence and allowing the use of variable preconditioning which may be particularly useful in some applications. Numerical experiments are illustrated to show the overall significant robustness of the iterative method for solving general sparse multi-shifted and multiple right-hand-side systems, and in realistic PageRank calculations. To the best of our knowledge, this is the first Krylov subspace method that combines deflation techniques and variable preconditioning for solving sequences of multi-shifted linear systems with multiple right-hand sides simultaneously.

Variable Preconditioned Krylov Subspace Method With Communication Avoiding Technique for Electromagnetic Analysis

The variable preconditioned (VP) Krylov subspace method with communication avoiding (CA) technique is adopted for the solver of a linear system obtained from electromagnetic analysis, and the numerical features are investigated. A massive communication time between processing units (PUs) or graphics PU/many integrated core is a big issue for parallelization efficiency that needs to be solved. Although k-skip Krylov subspace method is one solution for the issue, convergence property of the method is becoming worse. For this reason, we adopt the k-skip Krylov subspace method as inner-loop solver of the VP Krylov subspace method. The result of computation shows that the VP Krylov subspace method with CA technique is very effective for the linear system obtained from electromagnetic analysis.

Variable Preconditioning of Krylov Subspace Methods for Hierarchical Matrices With Adaptive Cross Approximation

This paper discusses Krylov subspace methods to solve a linear system whose coefficient matrix is represented by a hierarchical matrix. We propose a preconditioning technique using a part of the original hierarchical matrix to accelerate the convergence of the Krylov subspace methods. The proposed preconditioning technique is based on the assumption that the submatrices on the original hierarchical matrix are approximated using the adaptive cross approximation or variants thereof. The performance of Krylov subspace methods with the proposed preconditioning technique is examined through numerical experiments on an electrostatic field analysis.

Analysis and Practical Use of Flexible BiCGStab

A flexible version of the BiCGStab algorithm for solving a linear system of equations is analyzed. We show that under variable preconditioning, the perturbation to the outer residual norm is of the same order as that to the application of the preconditioner. Hence, in order to maintain a similar convergence behavior to BiCGStab while reducing the preconditioning cost, the flexible version can be used with a moderate tolerance in the preconditioning Krylov solves. We explored the use of flexible BiCGStab in a large-scale reacting flow application, PFLOTRAN, and showed that the use of a variable multigrid preconditioner significantly accelerates the simulation time on extreme-scale computers using $$O(10^4)$$O(104)---$$O(10^5)$$O(105) processor cores.

Speedup of Iterative Solver for Electromagnetic Analysis Using Many Integrated Core Architecture

Speedup of Krylov subspace method for the linear system obtained from an electromagnetic analysis using Many Integrated Core (MIC) architecture is investigated. In recent years, MIC architecture appears on the scene of high-performance computing research fields. Since Xeon Phi coprocessor is x86 architecture, the ordinal numerical code can be implemented on the devices without transcribing. In this paper, the linear systems obtained by extended element-free Galerkin method and edge element are solved by mixed precision variable preconditioned (VP) Krylov subspace method, and the methods are implemented on MIC to get high performance. The results of computation show that speedup of VP generalized minimal residual on MIC is three times faster than that of sequential calculation. In addition, the speedup of VP conjugate gradient (CG) is 40.47 times faster than that of sequential CG method.

Block Krylov Methods to Solve Adjoint Problems in Aerodynamic Design Optimization

A new resolution method for solving linear systems resulting from the discrete adjoint approach in aerodynamic shape optimization with a single objective and multiple constraint functions is presented. The steady flow is governed by the three-dimensional compressible Favre-averaged Navier–Stokes equations combined with a two-equation turbulence model. Block flexible variants of the generalized minimal residual method are Krylov methods designed for the solution of linear systems with multiple right-hand sides allowing variable preconditioning. Then, they can be applied to the discrete adjoint equations of Favre-averaged Navier–Stokes equations for all objective and constraint functions as a right-hand side. The paper focuses on the development of a new algorithm combining the three following features: flexibility, multiple right-hand-side concurrent processing, and spectral deflation. This latter property enables to recycle approximate spectral information from one restart to the next one in order to avoid generalized minimal residual stagnation when short restart recurrences are performed. This novel block method is named block flexible generalized minimal residual with deflated restarting. It is evaluated on two aerodynamic configurations showing its capabilities to reduce computational times over standard Krylov methods.

Parallelization of Finite Element Analysis of Nonlinear Magnetic Fields Using GPU

The acceleration of a nonlinear magnetic field analysis by parallelizing the finite element method (FEM) is examined using the graphics processing unit (GPU). It is shown that the speedup of the magnetic field analysis is realized by parallelizing the variable preconditioned conjugate gradient (VPCG) method. The Jacobi over-relaxation (JOR) method, conjugate residual (CR) method and conjugate gradient (CG) method are also applied in the variable preconditioning. The results of computations demonstrate that VPCG using the GPU significantly improve the performance. Especially, CG applied by variable preconditioned on GPU is 39 times faster than ICCG on a CPU.

Variable preconditioning in complex Hilbert space and its application to the nonlinear Schrödinger equation

An iterative method is developed for nonlinear equations in complex Hilbert spaces, extending the method of variable preconditioning defined earlier in real spaces. We derive convergence of our method. The motivating example for this extension is the time-dependent nonlinear Schrödinger equation, where we use our iteration for the time discretization of the problem and test it numerically.

Iterative Solver for Linear System Obtained by Edge Element: Variable Preconditioned Method With Mixed Precision on GPU

The variable preconditioned (VP) Krylov subspace method with mixed precision is implemented on graphics processing unit (GPU) using compute unified device architecture (CUDA), and the linear system obtained from the edge element is solved by means of the method. The VPGCR method has the sufficient condition for the convergence. This sufficient condition leads us that the residual equation for the preconditioned procedure of VPGCR can be solved in the range of single precision. To stretch the sufficient condition, we propose the hybrid scheme of VP Krylov subspace method that uses single and double precision operations. The results of computations show that VPCG with mixed precision on GPU demonstrated significant achievement than that of CPU. Especially, VPCG-JOR on GPU with mixed precision is 41.853 times faster than that of VPCG-CG on CPU.

High Performance Iterative Solver for Linear System using Multi GPU

The variable preconditioned (VP) Krylov subspace method on multi Graphics Processing Unit (GPU) is numerically investigated. Besides, the linear system obtained by finite element method with an edge element is adopted for the problem. The results of computations show that VP conjugate gradient method on multi GPU demonstrated significant achievement than that of CPU. Especially, VP conjugate gradient method on multi GPU is 4.35 times faster than that of CPU. However, transmission rate between the PC using Gigabit Ethernet is the bottleneck of the performance.

Efficient Implementation of Inner-Outer Flexible GMRES for the Method of Moments Based on a Volume-Surface Integral Equation

This paper presents flexible inner-outer Krylov subspace methods, which are implemented using the fast multipole method (FMM) for solving scattering problems with mixed dielectric and conducting object. The flexible Krylov subspace methods refer to a class of methods that accept variable preconditioning. To obtain the maximum efficiency of the inner-outer methods, it is desirable to compute the inner iterations with the least possible effort. Hence, generally, inaccurate matrix-vector multiplication (MVM) is performed in the inner solver within a short computation time. This is realized by using a particular feature of the multipole techniques. The accuracy and computational cost of the FMM can be controlled by appropriately selecting the truncation number, which indicates the number of multipoles used to express far-field interactions. On the basis of the abovementioned fact, we construct a less-accurate but much cheaper version of the FMM by intentionally setting the truncation number to a sufficiently low value, and then use it for the computation of inaccurate MVM in the inner solver. However, there exists no definite rule for determining the suitable level of accuracy for the FMM within the inner solver. The main focus of this study is to clarify the relationship between the overall efficiency of the flexible inner-outer Krylov solver and the accuracy of the FMM within the inner solver. Numerical experiments reveal that there exits an optimal accuracy level for the FMM within the inner solver, and that a moderately accurate FMM operator serves as the optimal preconditioner.

A variable preconditioned GCR( [formula omitted]) method using the GSOR method for singular and rectangular linear systems

The Generalized Conjugate Residual (GCR) method with a variable preconditioning is an efficient method for solving a large sparse linear system A x = b . It has been clarified by some numerical experiments that the Successive Over Relaxation (SOR) method is more effective than Krylov subspace methods such as GCR and ILU(0) preconditioned GCR for performing the variable preconditioning. However, SOR cannot be applied for performing the variable preconditioning when solving such linear systems that the coefficient matrix has diagonal entries of zero or is not square. Therefore, we propose a type of the generalized SOR (GSOR) method. By numerical experiments on the singular linear systems, we demonstrate that the variable preconditioned GCR using GSOR is effective.

Steepest Descent and Conjugate Gradient Methods with Variable Preconditioning

We analyze the conjugate gradient (CG) method with variable preconditioning for solving a linear system with a real symmetric positive definite (SPD) matrix of coefficients $A$. We assume that the preconditioner is SPD on each step, and that the condition number of the preconditioned system matrix is bounded above by a constant independent of the step number. We show that the CG method with variable preconditioning under this assumption may not give improvement, compared to the steepest descent (SD) method. We describe the basic theory of CG methods with variable preconditioning with the emphasis on “worst case” scenarios, and provide complete proofs of all facts not available in the literature. We give a new elegant geometric proof of the SD convergence rate bound. Our numerical experiments, comparing the preconditioned SD and CG methods, not only support and illustrate our theoretical findings, but also reveal two surprising and potentially practically important effects. First, we analyze variable preconditioning in the form of inner-outer iterations. In previous such tests, the unpreconditioned CG inner iterations are applied to an artificial system with some fixed preconditioner as a matrix of coefficients. We test a different scenario, where the unpreconditioned CG inner iterations solve linear systems with the original system matrix $A$. We demonstrate that the CG-SD inner-outer iterations perform as well as the CG-CG inner-outer iterations in these tests. Second, we compare the CG methods using a two-grid preconditioning with fixed and randomly chosen coarse grids, and observe that the fixed preconditioner method is twice as slow as the method with random preconditioning.

Displacement decomposition and parallelisation of the PCG method for elasticity problems

This paper describes the displacement decomposition for elasticity problems solved by the finite element method. This decomposition can be used for parallel implementation of the conjugate gradient solvers and construction of parallelisable preconditioners. Both the fixed and variable preconditioning can be applied according to the solution of subproblems. Numerical efficiency of the parallel algorithms is demonstrated on an academic benchmark and real-life modelling problem. A comparison with a domain decomposition technique is also provided.

DQGMRES: a Direct Quasi-minimal Residual Algorithm Based on Incomplete Orthogonalization

We describe a Krylov subspace technique, based on incomplete orthogonalization of the Krylov vectors, which can be considered as a truncated version of GMRES. Unlike GMRES(m), the restarted version of GMRES, the new method does not require restarting. Like GMRES, it does not break down. Numerical experiments show that DQGMRES(k) often performs as well as the restarted GMRES using a subspace of dimension m=2k. In addition, the algorithm is flexible to variable preconditioning, i.e., it can accommodate variations in the preconditioner at every step. In particular, this feature allows the use of any iterative solver as a right-preconditioner for DQGMRES(k). This inner-outer iterative combination often results in a robust approach for solving indefinite non-Hermitian linear systems.