Large-scale Discrete Ill-posed Problems Research Articles

Tikhonov regularization with conjugate gradient least squares method for large-scale discrete ill-posed problem in image restoration

Image restoration is a large-scale discrete ill-posed problem, which can be transformed into a Tikhonov regularization problem that can approximate the original image. Kronecker product approximation is introduced into the Tikhonov regularization problem to produce an alternative problem of solving the generalized Sylvester matrix equation, reducing the scale of the image restoration problem. This paper considers solving this alternative problem by applying the conjugate gradient least squares (CGLS) method which has been demonstrated to be efficient and concise. The convergence of the CGLS method is analyzed, and it is demonstrated that the CGLS method converges to the least squares solution within the finite number of iteration steps. The effectiveness and superiority of the CGLS method are verified by numerical tests.

Tikhonov regularization with MTRSVD method for solving large-scale discrete ill-posed problems

Regularization is possibly the most popular method for solving discrete ill-posed problems, whose solution is less sensitive to the error in the observed vector in the right hand than the original solution. This paper presents a new modified truncated randomized singular value decomposition (TR-MTRSVD) method for large Tikhonov regularization in standard form. The proposed TR-MTRSVD algorithm introduces the idea of randomized algorithm into the improved truncated singular value decomposition (MTSVD) method to solve large Tikhonov regularization problems. The approximation matrix Ãℓ produced by randomized SVD is replaced by the closest matrix Ãk̃ in a unitarily invariant matrix norm with the same spectral condition number. The regularization parameters are determined by the discrepancy principle. Numerical examples show the effectiveness and efficiency of the proposed TR-MTRSVD algorithm for large Tikhonov regularization problems.

Randomized core reduction for discrete ill-posed problem

In this paper, we apply randomized algorithms to approximate the total least squares (TLS) solution of the problem Ax≈b in the large-scale discrete ill-posed problems. A regularization technique, based on the multiplicative randomization and the subspace iteration, is proposed to obtain the approximate core problem. In the error analysis, we provide upper bounds for the errors of the solution and the residual of the randomized core reduction. Illustrative numerical examples and comparisons are presented.

Matrix forms of iterative algorithms to solve large-scale discrete ill-posed problems with an application to image restoration

Iterative methods to solve linear large-scale discrete problems are well known in the literature. When the linear system is ill-posed and contaminated by noise, some kind of regularization must be applied in order to achieve a feasible solution. In the first part of this paper, we revisit briefly some known methods to solve large-scale ill-posed discrete linear problems which are easy to implement and have low computational cost, formulating them in a unified manner and also proposing simple modifications in order to improve their performances. Matrix forms of iterative algorithms can be formulated depending on certain conditions on the blurring process, and have the advantage of avoiding the formation and storage in memory of the matrix that represents the blurring process, which is generally of very large dimension. As an original contribution, in the final part of this paper we present the matrix forms of the iterative algorithms revisited and test them in the problem of restoration of an image degraded by blurring and noise.

Modified truncated randomized singular value decomposition (MTRSVD) algorithms for large scale discrete ill-posed problems with general-form regularization**This work was supported in part by the National Science Foundation of China (Nos. 11771249 and 11371219).

In this paper, we propose new randomization based algorithms for large scale linear discrete ill-posed problems with general-form regularization: subject to , where L is a regularization matrix. Our algorithms are inspired by the modified truncated singular value decomposition (MTSVD) method, which suits only for small to medium scale problems, and randomized SVD (RSVD) algorithms that generate good low rank approximations to A. We use rank-k truncated randomized SVD (TRSVD) approximations to A by truncating the rank- RSVD approximations to A, where q is an oversampling parameter. The resulting algorithms are called modified TRSVD (MTRSVD) methods. At every step, we use the LSQR algorithm to solve the resulting inner least squares problem, which is proved to become better conditioned as k increases so that LSQR converges faster. We present sharp bounds for the approximation accuracy of the RSVDs and TRSVDs for severely, moderately and mildly ill-posed problems, and substantially improve a known basic bound for TRSVD approximations. We prove how to choose the stopping tolerance for LSQR in order to guarantee that the computed and exact best regularized solutions have the same accuracy. Numerical experiments illustrate that the best regularized solutions by MTRSVD are as accurate as the ones by the truncated generalized singular value decomposition (TGSVD) algorithm, and at least as accurate as those by some existing truncated randomized generalized singular value decomposition (TRGSVD) algorithms.

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent CGLS applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. In this paper, we establish bounds for the distance between the $k$-dimensional Krylov subspace and the $k$-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank $k$ approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.

Projected Tikhonov Regularization of Large-Scale Discrete Ill-Posed Problems

The solution of linear discrete ill-posed problems is very sensitive to perturbations in the data. Confidence intervals for solution coordinates provide insight into the sensitivity. This paper presents an efficient method for computing confidence intervals for large-scale linear discrete ill-posed problems. The method is based on approximating the matrix in these problems by a partial singular value decomposition of low rank. We investigate how to choose the rank. Our analysis also yields novel approaches to the solution of linear discrete ill-posed problems with solution norm or residual norm constraints.

Minimization of functionals on the solution of a large-scale discrete ill-posed problem

In this work we study the minimization of a linear functional defined on a set of approximate solutions of a discrete ill-posed problem. The primary application of interest is the computation of confidence intervals for components of the solution of such a problem. We exploit the technique introduced by Elden in 1990, utilizing a parametric programming reformulation involving the solution of a sequence of quadratically constrained least squares problems. Our iterative method, which uses the connection between Lanczos bidiagonalization and Gauss-type quadrature rules to bound certain matrix functionals, is well-suited for large-scale problems, and offers a significant reduction in matrix-vector product evaluations relative to available methods.

GKB-FP: an algorithm for large-scale discrete ill-posed problems

We describe an algorithm for large-scale discrete ill-posed problems, called GKB-FP, which combines the Golub-Kahan bidiagonalization algorithm with Tikhonov regularization in the generated Krylov subspace, with the regularization parameter for the projected problem being chosen by the fixed-point method by Bazan (Inverse Probl. 24(3), 2008). The fixed-point method selects as regularization parameter a fixed-point of the function ‖rλ‖2/‖fλ‖2, where fλ is the regularized solution and rλ is the corresponding residual. GKB-FP determines the sought fixed-point by computing a finite sequence of fixed-points of functions \(\|r_{\lambda}^{(k)}\|_{2}/\|f_{\lambda}^{(k)}\|_{2}\), where \(f_{\lambda}^{(k)}\) approximates fλ in a k-dimensional Krylov subspace and \(r_{\lambda}^{(k)}\) is the corresponding residual. Based on this and provided the sought fixed-point is reached, we prove that the regularized solutions \(f_{\lambda}^{(k)}\) remain unchanged and therefore completely insensitive to the number of iterations. This and the performance of the method when applied to well-known test problems are illustrated numerically.

Minimization of Linear Functionals Defined on Solutions of Large-Scale Discrete Ill-Posed Problems

The minimization of linear functionals defined on the solutions of discrete ill-posed problems arises, e.g., in the computation of confidence intervals for these solutions. In 1990, Elden proposed an algorithm for this minimization problem based on a parametric programming reformulation involving the solution of a sequence of trust-region problems, and using matrix factorizations. In this paper, we describe MLFIP, a large-scale version of this algorithm where a limited-memory trust-region solver is used on the subproblems. We illustrate the use of our algorithm in connection with an inverse heat conduction problem.