Linear Discrete Ill-posed Problems Research Articles

The Regularized Block GMERR Method and Its Simpler Version for Solving Large-Scale Linear Discrete Ill-Posed Problems

The Regularized Block GMERR Method and Its Simpler Version for Solving Large-Scale Linear Discrete Ill-Posed Problems

The Regularized Global GMERR Method for Solving Large-Scale Linear Discrete Ill-Posed Problems

The Regularized Global GMERR Method for Solving Large-Scale Linear Discrete Ill-Posed Problems

Augmented flexible Krylov subspace methods with applications to Bayesian inverse problems

This paper presents two new augmented flexible (AF)-Krylov subspace methods, AF-GMRES and AF-LSQR, to compute solutions of large-scale linear discrete ill-posed problems that can be modeled as the sum of two independent random variables, exhibiting smooth and sparse stochastic characteristics respectively. Following a Bayesian modeling approach, this corresponds to adding a covariance-weighted quadratic term and a sparsity enforcing ℓ1 term in the original least-squares minimization scheme. To handle the ℓ1 regularization term, the proposed approach constructs a sequence approximating quadratic problems that are partially solved using augmented flexible Krylov–Tikhonov methods.Compared to other traditional methods used to solve this minimization problem, such as those based on iteratively reweighted norm schemes, the new algorithms build a single (augmented, flexible) approximation (Krylov) subspace that encodes information about the different regularization terms through adaptable “preconditioning”. The solution space is then expanded as soon as a new problem within the sequence is defined. This also allows for the regularization parameters to be chosen on-the-fly at each iteration. Compared to most recent work on generalized flexible Krylov methods [6], our methods offer theoretical assurance of convergence and a more stable numerical performance. The efficiency of the new methods is shown through a variety of experiments, including a synthetic image deblurring problem, a synthetic atmospheric transport problem, and fluorescence molecular tomography reconstructions using both synthetic and real-world experimental data.

Approximation of the Tikhonov regularization parameter through Aitken's extrapolation

In the present work, we study the determination of the regularization parameter and the computation of the regularized solution in Tikhonov regularization, by the Aitken's extrapolation method. In particular, this convergence acceleration method is adjusted for the approximation of quadratic forms that appear in regularization methods, such as the generalized cross-validation method, the quasi-optimality criterion, the Gfrerer/Raus method and the Morozov's discrepancy principle. We present several numerical examples to illustrate the effectiveness of the derived estimates for approximating the regularization parameter for several linear discrete ill-posed problems and we compare the described method with further existing methods, for the determination of the regularized solution.

Numerical considerations of block GMRES methods when applied to linear discrete ill-posed problems

Linear systems of equations with a matrix whose singular values decay to zero with increasing index number without a significant gap are commonly referred to as linear discrete ill-posed problems. We are interested in solving large systems of this kind when the right-hand side has k>1 column vectors. The systems may be regarded as one system of equations with a block vector (with k columns) as the right-hand side and then solved by a block iterative method, or as k linear systems of equations (one for each right-hand side vector) that can be solved independently. Thus, the solution is a block vector with k columns. In many applications, including the restoration of color images, the right-hand side represents measurements that are contaminated by errors. Block iterative methods compute all columns of the solution block vector simultaneously. We will illustrate the performance of standard block GMRES methods and global GMRES methods, which also are block methods, and show that they may determine computed solutions of lower quality than when each column of the solution block vector is computed independently by a “standard” iterative method. We introduce a new local block GMRES method that can overcome the problems associated with block GMRES methods applied to linear discrete ill-posed problems.

Limited memory restarted ℓp-ℓq minimization methods using generalized Krylov subspaces

Regularization of certain linear discrete ill-posed problems, as well as of certain regression problems, can be formulated as large-scale, possibly nonconvex, minimization problems, whose objective function is the sum of the p th power of the ℓp-norm of a fidelity term and the q th power of the ℓq-norm of a regularization term, with 0 < p,q ≤ 2. We describe new restarted iterative solution methods that require less computer storage and execution time than the methods described by Huang et al. (BIT Numer. Math. 57,351–378, 14). The reduction in computer storage and execution time is achieved by periodic restarts of the method. Computed examples illustrate that restarting does not reduce the quality of the computed solutions.

Tensor Conjugate-Gradient methods for tensor linear discrete ill-posed problems

&lt;abstract&gt;&lt;p&gt;This paper presents three types of tensor Conjugate-Gradient (tCG) methods for solving large-scale linear discrete ill-posed problems based on the t-product between third-order tensors. An automatic determination strategy of a suitable regularization parameter is proposed for the tCG method in the Fourier domain (A-tCG-FFT). An improved version and a preconditioned version of the tCG method are also presented. The discrepancy principle is employed to determine a suitable regularization parameter. Several numerical examples in image and video restoration are given to show the effectiveness of the proposed tCG methods.&lt;/p&gt;&lt;/abstract&gt;

Range restricted iterative methods for linear discrete ill-posed problems

Range restricted iterative methods for linear discrete ill-posed problems

Some numerical aspects of Arnoldi-Tikhonov regularization

Large linear discrete ill-posed problems are commonly solved by first reducing them to small size by application of a few steps of a Krylov subspace method, and then applying Tikhonov regularization to the reduced problem. A regularization parameter determines how much the given problem is regularized before solution. If the matrix of the linear discrete ill-posed problem is nonsymmetric, then the reduction often is carried out with the aid of the Arnoldi process, while when the matrix is symmetric, the Lanczos process is used. This paper discusses several numerical aspects of these solution methods. We illustrate that it may be beneficial to apply the Arnoldi process also when the matrix is symmetric and discuss how certain user-chosen basis vectors can be added to the solution subspace. Finally, we compare the application of the discrepancy principle for the determination of the regularization parameter to another approach that is based on the solution of a cubic equation and has been proposed in the literature.

Theoretical and numerical aspects of a non-stationary preconditioned iterative method for linear discrete ill-posed problems

This work considers some theoretical and computational aspects of the recent paper (Buccini et al., 2021), whose aim was to relax the convergence conditions in a previous work by Donatelli and Hanke, and thereby make the iterative method discussed in the latter work applicable to a larger class of problems. This aim was achieved in the sense that the iterative method presented convergences for a larger class of problems. However, while the analysis presented is correct, it does not establish the superior behavior of the iterative method described. The present note describes a slight modification of the analysis that establishes the superiority of the iterative method. The new analysis allows to discuss the behavior of the algorithm when varying the involved parameters, which is also useful for their empirical estimation.

An Arnoldi-based preconditioner for iterated Tikhonov regularization

Many problems in science and engineering give rise to linear systems of equations that are commonly referred to as large-scale linear discrete ill-posed problems. These problems arise, for instance, from the discretization of Fredholm integral equations of the first kind. The matrices that define these problems are typically severely ill-conditioned and may be rank-deficient. Because of this, the solution of linear discrete ill-posed problems may not exist or be very sensitive to perturbations caused by errors in the available data. These difficulties can be reduced by applying Tikhonov regularization. We describe a novel “approximate Tikhonov regularization method” based on constructing a low-rank approximation of the matrix in the linear discrete ill-posed problem by carrying out a few steps of the Arnoldi process. The iterative method so defined is transpose-free. Our work is inspired by a scheme by Donatelli and Hanke, whose approximate Tikhonov regularization method seeks to approximate a severely ill-conditioned block-Toeplitz matrix with Toeplitz-blocks by a block-circulant matrix with circulant-blocks. Computed examples illustrate the performance of our proposed iterative regularization method.

Updating the Landweber Iteration Method for Solving Inverse Problems

The Landweber iteration method is one of the most popular methods for the solution of linear discrete ill-posed problems. The diversity of physical problems and the diversity of operators that result from them leads us to think about updating the main methods and algorithms to achieve the best results. We considered in this work the linear operator equation and the use of a new version of the Landweber iterative method as an iterative solver. The main goal of updating the Landweber iteration method is to make the iteration process fast and more accurate. We used a polar decomposition to achieve a symmetric positive definite operator instead of an identity operator in the classical Landweber method. We carried out the convergence and other necessary analyses to prove the usability of the new iteration method. The residual method was used as an analysis method to rate the convergence of the iteration. The modified iterative method was compared to the classical Landweber method. A numerical experiment illustrates the effectiveness of this method by applying it to solve the inverse boundary value problem of the heat equation (IBVP).

Industry-oriented method for dynamic force identification in peripheral milling based on FSC-LSQR using acceleration signals

Online measurement of milling force is very important for machining process monitoring and control. In practice, it is difficult to measure the milling force directly during the milling process. This paper develops a method for milling force identification called least square QR-factorization with the fast stopping criterion (FSC-LSQR) method, and the queue buffer structure (QBS) is employed for the online identification of milling force using acceleration signals. The convolution integral of milling force and acceleration signals is discretized, which turns the problem of milling force identification into a linear discrete ill-posed problem. The FSC-LSQR algorithm is adopted for milling force identification because of its high efficiency and accuracy, which can effectively handle the linear discrete ill-posed problem. The online identification of milling force can be realized using the acceleration signal enqueue and the milling force dequeue operations of the QBS. Finally, the effectiveness of the method is verified by milling tests under different milling parameters.

Weighted tensor Golub–Kahan–Tikhonov-type methods applied to image processing using a t-product

This paper discusses weighted tensor Golub–Kahan-type bidiagonalization processes using the t-product. This product was introduced in M.E. Kilmer and C.D. Martin (2011). A few steps of a bidiagonalization process with a weighted least squares norm are carried out to reduce a large-scale linear discrete ill-posed problem to a problem of small size. The weights are determined by symmetric positive definite (SPD) tensors. Tikhonov regularization is applied to the reduced problem. An algorithm for tensor Cholesky factorization of SPD tensors is presented. The data is a laterally oriented matrix or a general third order tensor. The use of a weighted Frobenius norm in the fidelity term of Tikhonov minimization problems is appropriate when the noise in the data has a known covariance matrix that is not the identity. We use the discrepancy principle to determine both the regularization parameter in Tikhonov regularization and the number of bidiagonalization steps. Applications to image and video restoration are considered.

RBF approximation of three dimensional PDEs using tensor Krylov subspace methods

In this paper, we propose different algorithms for the solution of a tensor linear discrete ill-posed problem arising in the application of the meshless method for solving PDEs in three-dimensional space using multiquadric radial basis functions. It is well known that the truncated singular value decomposition (TSVD) is the most common effective solver for ill-conditioned systems, but unfortunately the operation count for solving a linear system with the TSVD is computationally expensive for large-scale matrices. In the present work, we propose algorithms based on the use of the well known Einstein product for two tensors to define the tensor global Arnoldi and the tensor Gloub Kahan bidiagonalization algorithms. Using the so-called Tikhonov regularization technique, we will be able to provide computable approximate regularized solutions in a few iterations. The tensor formulation will allow us to develop an RBF approximation for high dimensional PDEs based on Hierarchical tensors and Adaptive Cross Approximation, which in turn reduces significantly the storage and computational costs, while the accuracy of the method is preserved. The performance of the proposed methods is illustrated with a variety of benchmark examples and large-scale industrial applications with high degrees of freedom.

Tensor Arnoldi–Tikhonov and GMRES-Type Methods for Ill-Posed Problems with a t-Product Structure

This paper describes solution methods for linear discrete ill-posed problems defined by third order tensors and the t-product formalism introduced in (Linear Algebra Appl 435:641–658, 2011). A t-product Arnoldi (t-Arnoldi) process is defined and applied to reduce a large-scale Tikhonov regularization problem for third order tensors to a problem of small size. The data may be represented by a laterally oriented matrix or a third order tensor, and the regularization operator is a third order tensor. The discrepancy principle is used to determine the regularization parameter and the number of steps of the t-Arnoldi process. Numerical examples compare results for several solution methods, and illustrate the potential superiority of solution methods that tensorize over solution methods that matricize linear discrete ill-posed problems for third order tensors.

Tensor Krylov subspace methods with an invertible linear transform product applied to image processing

This paper discusses several transform-based methods for solving linear discrete ill-posed problems for third order tensor equations based on a tensor-tensor product defined by an invertible linear transform. Linear transform-based tensor-tensor products were first introduced in Kernfeld et al. (2015) [16]. These tensor-tensor products are applied to derive Tikhonov regularization methods based on Golub-Kahan-type bidiagonalization and Arnoldi-type processes. GMRES-type solution methods based on the latter process also are described. By applying only a fairly small number of steps of these processes, large-scale problems are reduced to problems of small size. The number of steps required by these processes and the regularization parameter are determined by the discrepancy principle. The data tensor is a general third order tensor or a tensor defined by a laterally oriented matrix. A quite general regularization tensor can be applied in Tikhonov regularization. Applications to color image and video restorations illustrate the effectiveness of the proposed methods.

Golub\u2013Kahan vs. Monte Carlo: a comparison of bidiagonlization and a randomized SVD method for the solution of linear discrete ill-posed problems

Randomized methods can be competitive for the solution of problems with a large matrix of low rank. They also have been applied successfully to the solution of large-scale linear discrete ill-posed problems by Tikhonov regularization (Xiang and Zou in Inverse Probl 29:085008, 2013). This entails the computation of an approximation of a partial singular value decomposition of a large matrix A that is of numerical low rank. The present paper compares a randomized method to a Krylov subspace method based on Golub–Kahan bidiagonalization with respect to accuracy and computing time and discusses characteristics of linear discrete ill-posed problems that make them well suited for solution by a randomized method.

Iteratively Reweighted FGMRES and FLSQR for Sparse Reconstruction

This paper presents two new algorithms to compute sparse solutions of large-scale linear discrete ill-posed problems. The proposed approach consists in constructing a sequence of quadratic problems...

A novel modified TRSVD method for large-scale linear discrete ill-posed problems

The truncated singular value decomposition (TSVD) is a popular method for solving linear discrete ill-posed problems with a small to moderately sized matrix A. This method replaces the matrix A by the closest matrix Ak of low rank k, and then computes the minimal norm solution of the linear system of equations with a rank-deficient matrix so obtained. The modified TSVD (MTSVD) method improves the TSVD method, by replacing A by a matrix that is closer to A than Ak in a unitarily invariant matrix norm and has the same spectral condition number as Ak. Approximations of the SVD of a large matrix A can be computed quite efficiently by using a randomized SVD (RSVD) method. This paper presents a novel modified truncated randomized singular value decomposition (MTRSVD) method for computing approximate solutions to large-scale linear discrete ill-posed problems. The rank, k, is determined with the aid of the discrepancy principle, but other techniques for selecting a suitable rank also can be used. Numerical examples illustrate the effectiveness of the proposed method and compare it to the truncated RSVD method.