Large-scale Linear Discrete Ill-posed Problems Research Articles

Augmented flexible Krylov subspace methods with applications to Bayesian inverse problems

This paper presents two new augmented flexible (AF)-Krylov subspace methods, AF-GMRES and AF-LSQR, to compute solutions of large-scale linear discrete ill-posed problems that can be modeled as the sum of two independent random variables, exhibiting smooth and sparse stochastic characteristics respectively. Following a Bayesian modeling approach, this corresponds to adding a covariance-weighted quadratic term and a sparsity enforcing ℓ1 term in the original least-squares minimization scheme. To handle the ℓ1 regularization term, the proposed approach constructs a sequence approximating quadratic problems that are partially solved using augmented flexible Krylov–Tikhonov methods.Compared to other traditional methods used to solve this minimization problem, such as those based on iteratively reweighted norm schemes, the new algorithms build a single (augmented, flexible) approximation (Krylov) subspace that encodes information about the different regularization terms through adaptable “preconditioning”. The solution space is then expanded as soon as a new problem within the sequence is defined. This also allows for the regularization parameters to be chosen on-the-fly at each iteration. Compared to most recent work on generalized flexible Krylov methods [6], our methods offer theoretical assurance of convergence and a more stable numerical performance. The efficiency of the new methods is shown through a variety of experiments, including a synthetic image deblurring problem, a synthetic atmospheric transport problem, and fluorescence molecular tomography reconstructions using both synthetic and real-world experimental data.

The Regularized Global GMERR Method for Solving Large-Scale Linear Discrete Ill-Posed Problems

The Regularized Global GMERR Method for Solving Large-Scale Linear Discrete Ill-Posed Problems

Tensor Conjugate-Gradient methods for tensor linear discrete ill-posed problems

<abstract><p>This paper presents three types of tensor Conjugate-Gradient (tCG) methods for solving large-scale linear discrete ill-posed problems based on the t-product between third-order tensors. An automatic determination strategy of a suitable regularization parameter is proposed for the tCG method in the Fourier domain (A-tCG-FFT). An improved version and a preconditioned version of the tCG method are also presented. The discrepancy principle is employed to determine a suitable regularization parameter. Several numerical examples in image and video restoration are given to show the effectiveness of the proposed tCG methods.</p></abstract>

An Arnoldi-based preconditioner for iterated Tikhonov regularization

Many problems in science and engineering give rise to linear systems of equations that are commonly referred to as large-scale linear discrete ill-posed problems. These problems arise, for instance, from the discretization of Fredholm integral equations of the first kind. The matrices that define these problems are typically severely ill-conditioned and may be rank-deficient. Because of this, the solution of linear discrete ill-posed problems may not exist or be very sensitive to perturbations caused by errors in the available data. These difficulties can be reduced by applying Tikhonov regularization. We describe a novel “approximate Tikhonov regularization method” based on constructing a low-rank approximation of the matrix in the linear discrete ill-posed problem by carrying out a few steps of the Arnoldi process. The iterative method so defined is transpose-free. Our work is inspired by a scheme by Donatelli and Hanke, whose approximate Tikhonov regularization method seeks to approximate a severely ill-conditioned block-Toeplitz matrix with Toeplitz-blocks by a block-circulant matrix with circulant-blocks. Computed examples illustrate the performance of our proposed iterative regularization method.

Golub\u2013Kahan vs. Monte Carlo: a comparison of bidiagonlization and a randomized SVD method for the solution of linear discrete ill-posed problems

Randomized methods can be competitive for the solution of problems with a large matrix of low rank. They also have been applied successfully to the solution of large-scale linear discrete ill-posed problems by Tikhonov regularization (Xiang and Zou in Inverse Probl 29:085008, 2013). This entails the computation of an approximation of a partial singular value decomposition of a large matrix A that is of numerical low rank. The present paper compares a randomized method to a Krylov subspace method based on Golub–Kahan bidiagonalization with respect to accuracy and computing time and discusses characteristics of linear discrete ill-posed problems that make them well suited for solution by a randomized method.

Iteratively Reweighted FGMRES and FLSQR for Sparse Reconstruction

This paper presents two new algorithms to compute sparse solutions of large-scale linear discrete ill-posed problems. The proposed approach consists in constructing a sequence of quadratic problems...

A novel modified TRSVD method for large-scale linear discrete ill-posed problems

The truncated singular value decomposition (TSVD) is a popular method for solving linear discrete ill-posed problems with a small to moderately sized matrix A. This method replaces the matrix A by the closest matrix Ak of low rank k, and then computes the minimal norm solution of the linear system of equations with a rank-deficient matrix so obtained. The modified TSVD (MTSVD) method improves the TSVD method, by replacing A by a matrix that is closer to A than Ak in a unitarily invariant matrix norm and has the same spectral condition number as Ak. Approximations of the SVD of a large matrix A can be computed quite efficiently by using a randomized SVD (RSVD) method. This paper presents a novel modified truncated randomized singular value decomposition (MTRSVD) method for computing approximate solutions to large-scale linear discrete ill-posed problems. The rank, k, is determined with the aid of the discrepancy principle, but other techniques for selecting a suitable rank also can be used. Numerical examples illustrate the effectiveness of the proposed method and compare it to the truncated RSVD method.

The regularizing properties of global GMRES for solving large-scale linear discrete ill-posed problems with several right-hand sides

The global Krylov subspace iterative methods are an attractive class of iterative solvers for solving linear systems with several right-hand sides. In this paper, the global version of the GMRES method is applied to solve linear discrete ill-posed problems that arise from the discretization of linear ill-posed problems, pattern classification and dimensionality reduction. The regularizing properties of the global GMRES method are analyzed, and a regularized global GMRES method is developed for solving linear discrete ill-posed problems with multiple right-hand sides contaminated by errors. Some numerical experiments on test matrices are presented to show the efficiency of the proposed method.

The low rank approximations and Ritz values in LSQR for linear discrete ill-posed problem**The work was supported by the National Science Foundation of China (No. 11771249).

LSQR and its mathematically equivalent CGLS have been popularly used over the decades for large-scale linear discrete ill-posed problems, where the iteration number k plays the role of the regularization parameter. It has been long known that if the Ritz values in LSQR converge to the large singular values of A in natural order, that is, they interlace the first k + 1 large singular values of A, until the semi-convergence of LSQR occurs then LSQR must have the same the regularization ability as the truncated singular value decomposition (TSVD) method and can compute a 2-norm filtering best possible regularized solution. However, hitherto there has been no definitive rigorous result on the approximation behavior of the Ritz values in the context of ill-posed problems. In this paper, for severely, moderately and mildly ill-posed problems, we give accurate solutions of the two closely related fundamental and highly challenging problems on the regularization of LSQR: (i) how accurate are the low rank approximations generated by Golub–Kahan bidiagonalization? (ii) Whether or not the Ritz values involved in LSQR approximate the large singular values of A in natural order? We also show how to reliably judge the accuracy of low rank approximations cheaply. Numerical experiments confirm our results.

A special modified Tikhonov regularization matrix for discrete ill-posed problems

In this paper, we investigate the solution of large-scale linear discrete ill-posed problems with error-contaminated data. If the solution exists, it is very sensitive to perturbations in the data. Tikhonov regularization, i.e., replacement of the available ill-conditioned problem by a nearby better conditioned one with a fidelity term and a penalty term, can reduce this sensitivity and is a popular approach to determine meaningful approximate solutions of such problems. The penalty term is determined by a regularization matrix. The choice of this matrix may significantly affect the quality of the computed approximate solution. In order to get an appropriate solution with improved accuracy, the paper constructs a special modified Tikhonov regularization (SMTR) matrix so as to include more useful information. The parameters involved in the presented regularization matrix are discussed. Besides, the corresponding preconditioner PSMTR is designed to accelerate the convergence rate of the CGLS method for solving Tikhonov regularization least-squares system. Furthermore, numerical experiments illustrate that the SMTR method and PSMTR preconditioner significantly outperform the related methods in terms of solution accuracy.

Reordering for improving global Arnoldi–Tikhonov method in image restoration problems

This paper discusses the solution of large-scale linear discrete ill-posed problems arising from image restoration problems. Since the scale of the problem is usually very large, the computations with the blurring matrix can be very expensive. In this regard, we consider problems in which the coefficient matrix is the sum of Kronecker products of matrices to benefit the computation. Here, we present an alternative approach based on reordering of the image approximations obtained with the global Arnoldi–Tikhonov method. The ordering of the intensities is such that it makes the image approximation monotonic and thus minimizes the finite differences norm. We present theoretical properties of the method and numerical experiments on image restoration.

On the computation of a truncated SVD of a large linear discrete ill-posed problem

The singular value decomposition is commonly used to solve linear discrete ill-posed problems of small to moderate size. This decomposition not only can be applied to determine an approximate solution but also provides insight into properties of the problem. However, large-scale problems generally are not solved with the aid of the singular value decomposition, because its computation is considered too expensive. This paper shows that a truncated singular value decomposition, made up of a few of the largest singular values and associated right and left singular vectors, of the matrix of a large-scale linear discrete ill-posed problems can be computed quite inexpensively by an implicitly restarted Golub---Kahan bidiagonalization method. Similarly, for large symmetric discrete ill-posed problems a truncated eigendecomposition can be computed inexpensively by an implicitly restarted symmetric Lanczos method.

Regularization matrices determined by matrix nearness problems

This paper is concerned with the solution of large-scale linear discrete ill-posed problems with error-contaminated data. Tikhonov regularization is a popular approach to determine meaningful approximate solutions of such problems. The choice of regularization matrix in Tikhonov regularization may significantly affect the quality of the computed approximate solution. This matrix should be chosen to promote the recovery of known important features of the desired solution, such as smoothness and monotonicity. We describe a novel approach to determine regularization matrices with desired properties by solving a matrix nearness problem. The constructed regularization matrix is the closest matrix in the Frobenius norm with a prescribed null space to a given matrix. Numerical examples illustrate the performance of the regularization matrices so obtained.

Square smoothing regularization matrices with accurate boundary conditions

This paper is concerned with the solution of large-scale linear discrete ill-posed problems. The determination of a meaningful approximate solution of these problems requires regularization. We discuss regularization by the Tikhonov method and by truncated iteration. The choice of regularization matrix in Tikhonov regularization may significantly affect the quality of the computed approximate solution. The present paper describes the construction of square regularization matrices from finite difference equations with a focus on the boundary conditions. The regularization matrices considered have a structure that makes them easy to apply in iterative methods, including methods based on the Arnoldi process. Numerical examples illustrate the properties and effectiveness of the regularization matrices described.

Projected Tikhonov Regularization of Large-Scale Discrete Ill-Posed Problems

The solution of linear discrete ill-posed problems is very sensitive to perturbations in the data. Confidence intervals for solution coordinates provide insight into the sensitivity. This paper presents an efficient method for computing confidence intervals for large-scale linear discrete ill-posed problems. The method is based on approximating the matrix in these problems by a partial singular value decomposition of low rank. We investigate how to choose the rank. Our analysis also yields novel approaches to the solution of linear discrete ill-posed problems with solution norm or residual norm constraints.

A generalized global Arnoldi method for ill-posed matrix equations

This paper discusses the solution of large-scale linear discrete ill-posed problems with a noise-contaminated right-hand side. Tikhonov regularization is used to reduce the influence of the noise on the computed approximate solution. We consider problems in which the coefficient matrix is the sum of Kronecker products of matrices and present a generalized global Arnoldi method, that respects the structure of the equation, for the solution of the regularized problem. Theoretical properties of the method are shown and applications to image deblurring are described.

An interior-point method for large constrained discrete ill-posed problems

Ill-posed problems are numerically underdetermined. It is therefore often beneficial to impose known properties of the desired solution, such as nonnegativity, during the solution process. This paper proposes the use of an interior-point method in conjunction with truncated iteration for the solution of large-scale linear discrete ill-posed problems with box constraints. An estimate of the error in the data is assumed to be available. Numerical examples demonstrate the competitiveness of this approach.

Sylvester Tikhonov-regularization methods in image restoration

In this paper, we consider large-scale linear discrete ill-posed problems where the right-hand side contains noise. Regularization techniques such as Tikhonov regularization are needed to control the effect of the noise on the solution. In many applications such as in image restoration the coefficient matrix is given as a Kronecker product of two matrices and then Tikhonov regularization problem leads to the generalized Sylvester matrix equation. For large-scale problems, we use the global-GMRES method which is an orthogonal projection method onto a matrix Krylov subspace. We present some theoretical results and give numerical tests in image restoration.

Tikhonov Regularization with a Solution Constraint

Many numerical methods for the solution of linear ill-posed problems apply Tikhonov regularization. This paper presents a modification of a numerical method proposed by Golub and von Matt for quadratically constrained least-squares problems and applies it to Tikhonov regularization of large-scale linear discrete ill-posed problems. The method is based on partial Lanczos bidiagonalization and Gauss quadrature. Computed examples illustrating its performance are presented.