Block GMRES Method Research Articles

Numerical considerations of block GMRES methods when applied to linear discrete ill-posed problems

Linear systems of equations with a matrix whose singular values decay to zero with increasing index number without a significant gap are commonly referred to as linear discrete ill-posed problems. We are interested in solving large systems of this kind when the right-hand side has k>1 column vectors. The systems may be regarded as one system of equations with a block vector (with k columns) as the right-hand side and then solved by a block iterative method, or as k linear systems of equations (one for each right-hand side vector) that can be solved independently. Thus, the solution is a block vector with k columns. In many applications, including the restoration of color images, the right-hand side represents measurements that are contaminated by errors. Block iterative methods compute all columns of the solution block vector simultaneously. We will illustrate the performance of standard block GMRES methods and global GMRES methods, which also are block methods, and show that they may determine computed solutions of lower quality than when each column of the solution block vector is computed independently by a “standard” iterative method. We introduce a new local block GMRES method that can overcome the problems associated with block GMRES methods applied to linear discrete ill-posed problems.

Two New Variants of the Simpler Block GMRES Method with Vector Deflation and Eigenvalue Deflation for Multiple Linear Systems

In this paper, two effective methods based on the simpler block GMRES method are established in order to solve the linear systems of equations with multiple right-hand sides. The first method is derived from the simpler block GMRES method with vector deflation restarting (SBGMRES-DR). The second method is constructed from a combination of SBGMRES-DR with the eigenvalue deflation technique, which is called the deflated simpler block GMRES method with vector deflation restarting (D-SBGMRES-DR). To be more specific, SBGMRES-DR is capable of removing linearly or almost linearly dependent vectors created by the block Arnoldi process. On the other hand, D-SBGMRES-DR not only deletes linearly or almost linearly dependent vectors but also retains harmonic Ritz vectors associated with the smallest harmonic Ritz values in magnitude, and adds them to the new search subspace at the time of restart. Finally, a wide range of practical experiments are carried out to assess the efficiency of the proposed methods. The numerical results indicate that the D-SBGMRES-DR method outperforms the compared methods with respect to the number of matrix–vector products and the computational time.

On restarted and deflated block FOM and GMRES methods for sequences of shifted linear systems

The problem of shifted linear systems is an important and challenging issue in a number of research applications. Krylov subspace methods are effective techniques for different kinds of this problem due to their advantages in large and sparse matrix problems. In this paper, two new block projection methods based on respectively block FOM and block GMRES are introduced for solving sequences of shifted linear systems. We first express the original problem explicitly by a sequence of Sylvester matrix equations whose coefficient matrices are obtained from the shifted linear systems. Then, we show the restarted shifted block FOM (rsh-BFOM) method and derive some of its properties. We also present a framework for the restarted shifted block GMRES (rsh-BGMRES) method. In this regard, we describe two variants of rsh-BGMRES, including (1) rsh-BGMRES with an unshifted base system that applies a fixed unshifted base system and (2) rsh-BGMRES with a variable shifted base system in which the base block system can change after restart. Furthermore, we consider the use of deflation techniques for improving the performance of the rsh-BFOM and rsh-BGMRES methods. Finally, some numerical experiments are conducted to demonstrate the effectiveness of the proposed methods.

The simpler block CMRH method for linear systems

The block changing minimal residual method based on the Hessenberg reduction algorithm (in short BCMRH) is a recent block Krylov method that can solve large linear systems with multiple right-hand sides. This method uses the block Hessenberg process with pivoting strategy to construct a trapezoidal Krylov basis and minimizes a quasi-residual norm by solving a least squares problem. In this paper, we describe the simpler BCMRH method which is a new variant that avoids the QR factorization to solve the least-squares problem. Another major difference between the classical and simpler variants of BCMRH is that the simpler one allows to check the convergence within each cycle of the block Hessenberg process by using a recursive relation that updates the residual at each iteration. This is not possible with the classical BCMRH where we can only compute an estimate of the residual norm. Experiments are described to compare the behavior of the new proposed method with that of the classical and simpler versions of the block GMRES method. These numerical experiments show the good performances of the simpler BCMRH method.

Block Krylov Subspace Methods for Functions of Matrices II: Modified Block FOM

We analyze an expansion of the generalized block Krylov subspace framework of [Electron. Trans. Numer. Anal., 47 (2017), pp. 100--126]. This expansion allows the use of low-rank modifications of the matrix projected onto the block Krylov subspace and contains, as special cases, the block GMRES method and the new block Radau--Arnoldi method. Within this general setting, we present results that extend the interpolation property from the nonblock case to a matrix polynomial interpolation property for the block case, and we relate the eigenvalues of the projected matrix to the latent roots of these matrix polynomials. Some error bounds for these modified block FOM methods for solving linear systems are presented. We then show how cospatial residuals can be preserved in the case of families of shifted linear block systems. This result is used to derive computationally practical restarted algorithms for block Krylov approximations that compute the action of a matrix function on a set of several vectors simultaneously. We prove some error bounds and present numerical results showing that two modifications of FOM, the block harmonic and the block Radau--Arnoldi methods for matrix functions, can significantly improve the convergence behavior.

A distributed and parallel asynchronous unite and conquer method to solve large scale non-Hermitian linear systems with multiple right-hand sides

Many problems in the field of science and engineering often require to solve simultaneously large-scale non-Hermitian sparse linear systems with multiple right-hand sides (RHSs). Efficiently solving such problems on extreme-scale platforms requires the minimization of global communications, reduction of synchronization points and promotion of asynchronous communications. We develop an extension of Unite and Conquer GMRES/LS-ERAM (UCGLE) method [1] by replacing its GMRES Component with Block GMRES method to solve non-Hermitian linear systems with multiple RHSs. UCGLE is a hybrid method consisting of three computing algorithms with asynchronous communication that allows the use of approximated eigenvalues to accelerate the convergence of the solves of linear systems and to improve their fault tolerance. In this paper, the variant of UCGLE with new components and manager engine implementation are introduced. This engine is capable of allocating multiple Block GMRES at the same time, each Block GMRES solving the linear systems with a subset of RHSs and accelerating the convergence using the eigenvalues approximated by other eigensolvers. Dividing the entire linear system with multiple RHSs into subsets and solving them simultaneously with different allocated linear solvers allow localizing calculations, reducing global communication, and improving parallel performance. Meanwhile, the asynchronous preconditioning using eigenvalues is able to speed up the convergence and improve the fault tolerance. Numerical experiments using different test matrices on supercomputer ROMEO indicate that the proposed method achieves a substantial decrease in both computation time and iterative steps with good scaling performance.

A Note on the Adaptive Simpler Block GMRES Method

The adaptive simpler block GMRES method was investigated by Zhong et al. (J Comput Appl Math 282:139–156, 2015) where the condition number of the adaptively chosen basis for the Krylov subspace was evaluated. In this paper, the new upper bound for the condition number is investigated. Numerical tests show that the new upper bound is tighter.

Correction to: A New Shifted Block GMRES Method with Inexact Breakdowns for Solving Multi-Shifted and Multiple Right-Hand Sides Linear Systems

We consider the efficient solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously that arise frequently in large-scale scientific and engineering simulations. We introduce a new shifted block GMRES method that can solve the whole sequence of linear systems simultaneously, it handles effectively the situation of inexact breakdowns in the inner block Arnoldi procedure for improved robustness, and recycles spectral information at restart to achieve faster convergence. Numerical experiments are reported on a suite of sparse matrix problems and in realistic quantum chromodynamics application to show the potential of the new proposed method to solve general multi-shifted and multiple right-hand sides linear systems fast and efficiently.

A New Shifted Block GMRES Method with Inexact Breakdowns for Solving Multi-Shifted and Multiple Right-Hand Sides Linear Systems

We consider the efficient solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously that arise frequently in large-scale scientific and engineering simulations. We introduce a new shifted block GMRES method that can solve the whole sequence of linear systems simultaneously, it handles effectively the situation of inexact breakdowns in the inner block Arnoldi procedure for improved robustness, and recycles spectral information at restart to achieve faster convergence. Numerical experiments are reported on a suite of sparse matrix problems and in realistic quantum chromodynamics application to show the potential of the new proposed method to solve general multi-shifted and multiple right-hand sides linear systems fast and efficiently.

A spectrally preconditioned and initially deflated variant of the restarted block GMRES method for solving multiple right-hand sides linear systems

The solution of large linear systems with multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications modelled by either partial differential or boundary integral equations. Block Krylov subspace methods are attractive to use for this problem class as they can overcome the memory bottleneck of direct methods and they perform block matrix-vector products, achieving high computational efficiency on modern cache-based computer architectures. In this paper we introduce variants of the block GMRES method that combine initial deflation and eigenvalue recycling strategies to remedy some typical convergence problems of block Krylov solvers. The new class of block iterative solvers has the ability to handle the approximate linear dependence of the block of right-hand sides and exploits approximate invariant subspaces recycled over the iterations to mitigate the bad effects that small eigenvalues can have on the convergence, by adapting an existing preconditioner. We illustrate the numerical behavior of the spectrally updated and initially deflated block GMRES method on a set of linear systems arising from the discretization of the Dirac equation and of boundary integral equations in electromagnetics scattering.

Flexible and deflated variants of the block shifted GMRES method

The solution of linear systems with multiple shifts and multiple right-hand sides given simultaneously is required in many large-scale scientific and engineering applications. In this paper we introduce new flexible and deflated variants of the shifted block GMRES method for this problem class. The proposed methods solve the whole sequence of linear systems simultaneously, detecting effectively the linear systems convergence and allowing the use of variable preconditioning which may be particularly useful in some applications. Numerical experiments are illustrated to show the overall significant robustness of the iterative method for solving general sparse multi-shifted and multiple right-hand-side systems, and in realistic PageRank calculations. To the best of our knowledge, this is the first Krylov subspace method that combines deflation techniques and variable preconditioning for solving sequences of multi-shifted linear systems with multiple right-hand sides simultaneously.

The block CMRH method for solving nonsymmetric linear systems with multiple right-hand sides

CMRH method (Changing minimal residual with Hessenberg process) is an iterative method for solving nonsymmetric linear systems. This method is similar to QMR method but based on the Hessenberg process instead of the Lanczos process. On dense matrices, the CMRH method is less expensive and requires less storage than other Krylov methods. This paper presents a block version of the CMRH algorithm for solving linear systems with multiple right-hand sides. The new algorithm is based on the block Hessenberg process and the iterates are characterized by a block version of the quasi-minimization property. We analyze its main properties and show that under the condition of full rank of block residual the block CMRH method cannot break down. Finally, some numerical examples are presented to show the efficiency of the new method in comparison with the traditional CMRH method and a comparison with the block GMRES method is also provided.

A block GMRES method with deflated restarting for solving linear systems with multiple shifts and multiple right‐hand sides

SummaryThe restarted block generalized minimum residual method (BGMRES) with deflated restarting (BGMRES‐DR) was proposed by Morgan to dump the negative effect of small eigenvalues from the convergence of the BGMRES method. More recently, Wu et al. introduced the shifted BGMRES method (BGMRES‐Sh) for solving the sequence of linear systems with multiple shifts and multiple right‐hand sides. In this paper, a new shifted block Krylov subspace algorithm that combines the characteristics of both the BGMRES‐DR and the BGMRES‐Sh methods is proposed. Moreover, our method is enhanced with a seed selection strategy to handle the case of almost linear dependence of the right‐hand sides. Numerical experiments illustrate the potential of the proposed method to solve efficiently the sequence of linear systems with multiple shifts and multiple right‐hand sides, with and without preconditioner, also against other state‐of‐the‐art solvers.

Residual-Based Simpler Block GMRES for Nonsymmetric Linear Systems with Multiple Right-Hand Sides

We propose in this paper a residual-based simpler block GMRES method for solving a system of linear algebraic equations with multiple right-hand sides. We show that this method is mathematically equivalent to the block GMRES method and thus equivalent to the simpler block GMRES method. Moreover, it is shown that the residual-based method is numerically more stable than the simpler block GMRES method. Based on the deflation strategy proposed by Calandra et al. (2013), we derive a deflation strategy to detect the possible linear dependence of the residuals and a near rank deficiency occurring in the block Arnoldi procedure. Numerical experiments are conducted to illustrate the performance of the new method.

The Parallel Algorithm for Solving Toeplitz Systems

This paper studies a parallel algorithm for real Toeplitz systems, which is proposed based on the block Jacobi iteration and GMRES method. The algorithm has the advantage of less float operations, fast convergence speed and especially suitable for parallel computating. In this paper, we first use the block Jacobi iterative method to obtain the iterative process, and then the GMRES method is nested to obtain the iterative sequences {xk}. Therefore, the parallel algorithm for solving symmetric positive definite Toeplitz systems is constructed. The convergence of the algorithm is also discussed simply in the paper. At the end, we give some numerical examples to illustrate the effectiveness of the parallel algorithm.

Block GMRES Method with Inexact Breakdowns and Deflated Restarting

We consider the solution of large linear systems with multiple right-hand sides using a block GMRES approach. We introduce a new algorithm that effectively handles the situation of almost rank deficient block generated by the block Arnoldi procedure and that enables the recycling of spectral information at restart. The first feature is inherited from an algorithm introduced by Robbé and Sadkane [Linear Algebra Appl., 419 (2006), pp. 265--285], while the second one is obtained by extending the deflated restarting strategy proposed by Morgan [Appl. Numer. Math., 54 (2005), pp. 222--236]. Through numerical experiments, we show that the new algorithm combines efficiently the attractive numerical features of its two parents and outperforms them.

A Modified Block Flexible GMRES Method with Deflation at Each Iteration for the Solution of Non-Hermitian Linear Systems with Multiple Right-Hand Sides

We propose a variant of the block GMRES method for the solution of linear systems of equations with multiple right-hand sides. We investigate a deflation strategy to detect when a linear combination of approximate solutions is already known that avoids performing expensive computational operations with the system matrix. This is especially useful when the cost of the preconditioner is supposed to be larger than the cost of orthogonalization in the block Arnoldi procedure. We specifically focus on the block GMRES method incorporating deflation at the end of each iteration proposed by Robbe and Sadkane [M. Robbe and M. Sadkane, Linear Algebra Appl., 419 (2006), pp. 265--285]. We extend their contribution by proposing that deflation be performed also at the beginning of each cycle. This change leads to a modified least-squares problem to be solved at each iteration and gives rise to a different behavior especially when multiple restarts are required to reach convergence. Additionally we investigate truncatio...

A Weighted Block GMRES method for solving linear systems with multiple right-hand sides

A Weighted Block GMRES method for solving linear systems with multiple right-hand sides

On the block GMRES method with deflated restarting

In this paper, we consider the block-GMRES method with deflated restarting for solving nonsymmetric linear systems with multiple right-hand sides. We modify slightly the restarted block-GMRES method with deflation of eigenvalues proposed by Morgan to obtain a new one. It is shown that the modified method is mathematically equivalent to the Morgan’s original method. However, the error analysis shows the modified version minimizes the numerical errors during a restart and therefore is better suited if the linear systems have to be solved with high precision. Numerical experiments report the effectiveness of the modified method.

Exact and inexact breakdowns in the block GMRES method

This paper addresses the issue of breakdowns in the block GMRES method for solving linear systems with multiple right-hand sides of the form AX = B. An exact (inexact) breakdown occurs at iteration j of this method when the block Krylov matrix ( B, AB, … , A j−1 B) is singular (almost singular). Exact breakdowns are the sign that a part of the exact solution is in the range of the Krylov matrix. They are primarily of theoretical interest. From a computational point of view, inexact breakdowns are most likely to occur. In such cases, the underlying block Arnoldi process that is used to build the block Krylov space should not be continued as usual. A natural way to continue the process is the use of deflation. However, as shown by Langou [J. Langou, Iterative Methods for Solving Linear Systems with Multiple Right-Hand Sides, Ph.D. dissertation TH/PA/03/24, CERFACS, France, 2003], deflation in block GMRES may lead to a loss of information that slows down the convergence. In this paper, instead of deflating the directions associated with almost converged solutions, these are kept and reintroduced in next iterations if necessary. Two criteria to detect inexact breakdowns are presented. One is based on the numerical rank of the generated block Krylov basis, the second on the numerical rank of the residual associated to approximate solutions. These criteria are analyzed and compared. Implementation details are discussed. Numerical results are reported.