Arnoldi Decomposition Research Articles

Arnoldi decomposition, GMRES, and preconditioning for linear discrete ill-posed problems

GMRES is one of the most popular iterative methods for the solution of large linear systems of equations that arise from the discretization of linear well-posed problems, such as boundary value problems for elliptic partial differential equations. The method is also applied to the iterative solution of linear systems of equations that are obtained by discretizing linear ill-posed problems, such as many inverse problems. However, GMRES does not always perform well when applied to the latter kind of problems. This paper seeks to shed some light on reasons for the poor performance of GMRES in certain situations, and discusses some remedies based on specific kinds of preconditioning. The standard implementation of GMRES is based on the Arnoldi process, which also can be used to define a solution subspace for Tikhonov or TSVD regularization, giving rise to the Arnoldi–Tikhonov and Arnoldi-TSVD methods, respectively. The performance of the GMRES, the Arnoldi–Tikhonov, and the Arnoldi-TSVD methods is discussed. Numerical examples illustrate properties of these methods.

On the choice of subspace for large-scale Tikhonov regularization problems in general form

Many applications in science and engineering require the solution of large linear discrete ill-posed problems. The matrices that define these problems are very ill-conditioned and possibly numerically singular, and the right-hand sides, which represent the measured data, typically are contaminated by measurement error. Straightforward solution of these problems generally is not meaningful due to severe error propagation. Tikhonov regularization seeks to alleviate this difficulty by replacing the given linear discrete ill-posed problem by a penalized least-squares problem, whose solution is less sensitive to the error in the right-hand side. The penalty term is determined by a regularization matrix. A suitable choice of this matrix may result in a computed solution of higher quality than when the regularization matrix is the identity. Two iterative solution methods based on the global Arnoldi decomposition method have been proposed in the literature for the solution of large-scale penalized least-squares problems that stem from Tikhonov regularization. In one of these, the regularization matrix influences the choice of the solution subspace; in the other one, it does not. This paper compares these approaches both with respect to the quality of the computed solution and computing time.

On GMRES for Singular EP and GP Systems

In this contribution, we study the numerical behavior of the Generalized Minimal Residual (GMRES) method for solving singular linear systems. It is known that GMRES determines a least squares solution without breakdown if the coefficient matrix is range-symmetric (EP), or if its range and nullspace are disjoint (GP) and the system is consistent. We show that the accuracy of GMRES iterates may deteriorate in practice due to three distinct factors: (i) the inconsistency of the linear system; (ii) the distance of the initial residual to the nullspace of the coefficient matrix; (iii) the extremal principal angles between the ranges of the coefficient matrix and its transpose. These factors lead to poor conditioning of the extended Hessenberg matrix in the Arnoldi decomposition and affect the accuracy of the computed least squares solution. We also compare GMRES with the range restricted GMRES (RR-GMRES) method. Numerical experiments show typical behaviors of GMRES for small problems with EP and GP matrices.

Compact Rational Krylov Methods for Nonlinear Eigenvalue Problems

We propose a new uniform framework of compact rational Krylov (CORK) methods for solving large-scale nonlinear eigenvalue problems $A(\lambda) x = 0$. For many years, linearizations were used for solving polynomial and rational eigenvalue problems. On the other hand, for the general nonlinear case, $A(\lambda)$ can first be approximated by a (rational) matrix polynomial and then a convenient linearization is used. However, the major disadvantage of linearization-based methods is the growing memory and orthogonalization costs with the iteration count, i.e., in general they are proportional to the degree of the polynomial. Therefore, the CORK family of rational Krylov methods exploits the structure of the linearization pencils by using a generalization of the compact Arnoldi decomposition. In this way, the extra memory and orthogonalization costs due to the linearization of the original eigenvalue problem are negligible for large-scale problems. Furthermore, we prove that each CORK step breaks down into an orthogonalization step of the original problem dimension and a rational Krylov step on small matrices. We also briefly discuss implicit restarting of the CORK method and how to exploit low rank structure. The CORK method is illustrated with two large-scale examples.

Arnoldi–Tikhonov regularization methods

Tikhonov regularization for large-scale linear ill-posed problems is commonly implemented by determining a partial Lanczos bidiagonalization of the matrix of the given system of equations. This paper explores the possibility of instead computing a partial Arnoldi decomposition of the given matrix. Computed examples illustrate that this approach may require fewer matrix–vector product evaluations and, therefore, less arithmetic work. Moreover, the proposed range-restricted Arnoldi–Tikhonov regularization method does not require the adjoint matrix and, hence, is convenient to use for problems for which the adjoint is difficult to evaluate.

Electromagnetic inversion using a reduced-order three-dimensional homogeneous model

Nonlinearized electromagnetic inversion of a three-dimensional homogeneous model of arbitrary but known support is considered. We propose a reduced-order representation of the cost functional based on the shift invariance property of the Arnoldi decomposition. Numerical experiments demonstrate acceleration up to a factor of 100 with respect to the usual repetitive solution of the forward scattering problem. We also discuss the applicability of a homogeneous model for estimating the effective constitutive parameter of an inhomogeneous target.

A Krylov--Schur Algorithm for Large Eigenproblems

Sorensen's implicitly restarted Arnoldi algorithm is one of the most successful and flexible methods for finding a few eigenpairs of a large matrix. However, the need to preserve the structure of the Arnoldi decomposition on which the algorithm is based restricts the range of transformations that can be performed on the decomposition. In consequence, it is difficult to deflate converged Ritz vectors from the decomposition. Moreover, the potential forward instability of the implicit QR algorithm can cause unwanted Ritz vectors to persist in the computation. In this paper we introduce a general Krylov decomposition that solves both problems in a natural and efficient manner.