Linear Ill-posed Problems Research Articles

Inverse learning in Hilbert scales

We study linear ill-posed inverse problems with noisy data in the framework of statistical learning. The corresponding linear operator equation is assumed to fit a given Hilbert scale, generated by some unbounded self-adjoint operator. Approximate reconstructions from random noisy data are obtained with general regularization schemes in such a way that these belong to the domain of the generator. The analysis has thus to distinguish two cases, the regular one, when the true solution also belongs to the domain of the generator, and the ‘oversmoothing’ one, when this is not the case. Rates of convergence for the regularized solutions will be expressed in terms of certain distance functions. For solutions with smoothness given in terms of source conditions with respect to the scale generating operator, then the error bounds can then be made explicit in terms of the sample size.

Tensor Conjugate-Gradient methods for tensor linear discrete ill-posed problems

<abstract><p>This paper presents three types of tensor Conjugate-Gradient (tCG) methods for solving large-scale linear discrete ill-posed problems based on the t-product between third-order tensors. An automatic determination strategy of a suitable regularization parameter is proposed for the tCG method in the Fourier domain (A-tCG-FFT). An improved version and a preconditioned version of the tCG method are also presented. The discrepancy principle is employed to determine a suitable regularization parameter. Several numerical examples in image and video restoration are given to show the effectiveness of the proposed tCG methods.</p></abstract>

Range restricted iterative methods for linear discrete ill-posed problems

Range restricted iterative methods for linear discrete ill-posed problems

On the acceleration of optimal regularization algorithms for linear ill-posed inverse problems

On the acceleration of optimal regularization algorithms for linear ill-posed inverse problems

Some numerical aspects of Arnoldi-Tikhonov regularization

Large linear discrete ill-posed problems are commonly solved by first reducing them to small size by application of a few steps of a Krylov subspace method, and then applying Tikhonov regularization to the reduced problem. A regularization parameter determines how much the given problem is regularized before solution. If the matrix of the linear discrete ill-posed problem is nonsymmetric, then the reduction often is carried out with the aid of the Arnoldi process, while when the matrix is symmetric, the Lanczos process is used. This paper discusses several numerical aspects of these solution methods. We illustrate that it may be beneficial to apply the Arnoldi process also when the matrix is symmetric and discuss how certain user-chosen basis vectors can be added to the solution subspace. Finally, we compare the application of the discrepancy principle for the determination of the regularization parameter to another approach that is based on the solution of a cubic equation and has been proposed in the literature.

Stochastic asymptotical regularization for linear inverse problems

We introduce stochastic asymptotical regularization (SAR) methods for the uncertainty quantification of the stable approximate solution of ill-posed linear-operator equations, which are deterministic models for numerous inverse problems in science and engineering. We demonstrate that SAR can quantify the uncertainty in error estimates for inverse problems. We prove the regularizing properties of SAR with regard to mean-square convergence. We also show that SAR is an order-optimal regularization method for linear ill-posed problems provided that the terminating time of SAR is chosen according to the smoothness of the solution. This result is proven for both a priori and a posteriori stopping rules under general range-type source conditions. Furthermore, some converse results of SAR are verified. Two iterative schemes are developed for the numerical realization of SAR, and the convergence analyses of these two numerical schemes are also provided. A toy example and a real-world problem of biosensor tomography are studied to show the accuracy and the advantages of SAR: compared with the conventional deterministic regularization approaches for deterministic inverse problems, SAR can provide the uncertainty quantification of the quantity of interest, which can in turn be used to reveal and explicate the hidden information about real-world problems, usually obscured by the incomplete mathematical modeling and the ascendence of complex-structured noise.

Theoretical and numerical aspects of a non-stationary preconditioned iterative method for linear discrete ill-posed problems

This work considers some theoretical and computational aspects of the recent paper (Buccini et al., 2021), whose aim was to relax the convergence conditions in a previous work by Donatelli and Hanke, and thereby make the iterative method discussed in the latter work applicable to a larger class of problems. This aim was achieved in the sense that the iterative method presented convergences for a larger class of problems. However, while the analysis presented is correct, it does not establish the superior behavior of the iterative method described. The present note describes a slight modification of the analysis that establishes the superiority of the iterative method. The new analysis allows to discuss the behavior of the algorithm when varying the involved parameters, which is also useful for their empirical estimation.

Ill-posed problems and the conjugate gradient method: Optimal convergence rates in the presence of discretization and modelling errors

Abstract In this paper, we prove order-optimal convergence rates for the conjugate gradient method applied to linear ill-posed problems when not only the data are noisy but also when the operator is perturbed via discretization and modelling errors.

A Trained Iterative Shrinkage Approach Based on Born Iterative Method for Electromagnetic Imaging

A trained-based Born iterative method (TBIM) is developed for electromagnetic imaging (EMI) applications. The proposed TBIM consists of a nested loop; the outer loop executes TBIM iteration steps, while the inner loop executes a trained iterative shrinkage thresholding algorithm (TISTA). The applied TISTA runs linear Landweber iterations implemented with a trained regularization network designed based on U-net architecture. A normalization process was imposed in TISTA that made TISTA training applicable within the proposed TBIM. The iterative utilization of the regularization network in TISTA is a bottleneck that demands high memory allocation through the training process. Therefore TISTA within each TBIM step was trained separately. The TISTA regularization network in each TBIM step was initialized using the weights from the previous TBIM step. The above approach achieved high-quality image restoration after running few TBIM steps while maintained low memory allocation through the training process. The proposed framework can be extended to Newton or quasi-Newton schemes, where within each Newton iteration, a linear ill-posed problem is optimized that differs from one example to another. The numerical results illustrated in this work show the superiority of the proposed TBIM compared to the conventional sparse-based Born iterative method (SBIM).

An Arnoldi-based preconditioner for iterated Tikhonov regularization

Many problems in science and engineering give rise to linear systems of equations that are commonly referred to as large-scale linear discrete ill-posed problems. These problems arise, for instance, from the discretization of Fredholm integral equations of the first kind. The matrices that define these problems are typically severely ill-conditioned and may be rank-deficient. Because of this, the solution of linear discrete ill-posed problems may not exist or be very sensitive to perturbations caused by errors in the available data. These difficulties can be reduced by applying Tikhonov regularization. We describe a novel “approximate Tikhonov regularization method” based on constructing a low-rank approximation of the matrix in the linear discrete ill-posed problem by carrying out a few steps of the Arnoldi process. The iterative method so defined is transpose-free. Our work is inspired by a scheme by Donatelli and Hanke, whose approximate Tikhonov regularization method seeks to approximate a severely ill-conditioned block-Toeplitz matrix with Toeplitz-blocks by a block-circulant matrix with circulant-blocks. Computed examples illustrate the performance of our proposed iterative regularization method.

On convergence rates of adaptive ensemble Kalman inversion for linear ill-posed problems

In this paper we discuss a deterministic form of ensemble Kalman inversion as a regularization method for linear inverse problems. By interpreting ensemble Kalman inversion as a low-rank approximation of Tikhonov regularization, we are able to introduce a new sampling scheme based on the Nyström method that improves practical performance. Furthermore, we formulate an adaptive version of ensemble Kalman inversion where the sample size is coupled with the regularization parameter. We prove that the proposed scheme yields an order optimal regularization method under standard assumptions if the discrepancy principle is used as a stopping criterion. The paper concludes with a numerical comparison of the discussed methods for an inverse problem of the Radon transform.

Solution of ill-posed problems with Chebfun

The analysis of linear ill-posed problems often is carried out in function spaces using tools from functional analysis. However, the numerical solution of these problems typically is computed by first discretizing the problem and then applying tools from finite-dimensional linear algebra. The present paper explores the feasibility of applying the Chebfun package to solve ill-posed problems with a regularize-first approach numerically. This allows a user to work with functions instead of vectors and with integral operators instead of matrices. The solution process therefore is much closer to the analysis of ill-posed problems than standard linear algebra-based solution methods. Furthermore, the difficult process of explicitly choosing a suitable discretization is not required.

Automatic balancing parameter selection for Tikhonov-TV regularization

This paper considers large-scale linear ill-posed inverse problems whose solutions can be represented as sums of smooth and piecewise constant components. To solve such problems we consider regularizers consisting of two terms that must be balanced. Namely, a Tikhonov term guarantees the smoothness of the smooth solution component, while a total-variation (TV) regularizer promotes blockiness of the non-smooth solution component. A scalar parameter allows to balance between these two terms and, hence, to appropriately separate and regularize the smooth and non-smooth components of the solution. This paper proposes an efficient algorithm to solve this regularization problem by the alternating direction method of multipliers (ADMM). Furthermore, a novel algorithm for automatic choice of the balancing parameter is introduced, using robust statistics. The proposed approach is supported by some theoretical analysis, and numerical experiments concerned with different inverse problems are presented to validate the choice of the balancing parameter.

Updating the Landweber Iteration Method for Solving Inverse Problems

The Landweber iteration method is one of the most popular methods for the solution of linear discrete ill-posed problems. The diversity of physical problems and the diversity of operators that result from them leads us to think about updating the main methods and algorithms to achieve the best results. We considered in this work the linear operator equation and the use of a new version of the Landweber iterative method as an iterative solver. The main goal of updating the Landweber iteration method is to make the iteration process fast and more accurate. We used a polar decomposition to achieve a symmetric positive definite operator instead of an identity operator in the classical Landweber method. We carried out the convergence and other necessary analyses to prove the usability of the new iteration method. The residual method was used as an analysis method to rate the convergence of the iteration. The modified iterative method was compared to the classical Landweber method. A numerical experiment illustrates the effectiveness of this method by applying it to solve the inverse boundary value problem of the heat equation (IBVP).

Industry-oriented method for dynamic force identification in peripheral milling based on FSC-LSQR using acceleration signals

Online measurement of milling force is very important for machining process monitoring and control. In practice, it is difficult to measure the milling force directly during the milling process. This paper develops a method for milling force identification called least square QR-factorization with the fast stopping criterion (FSC-LSQR) method, and the queue buffer structure (QBS) is employed for the online identification of milling force using acceleration signals. The convolution integral of milling force and acceleration signals is discretized, which turns the problem of milling force identification into a linear discrete ill-posed problem. The FSC-LSQR algorithm is adopted for milling force identification because of its high efficiency and accuracy, which can effectively handle the linear discrete ill-posed problem. The online identification of milling force can be realized using the acceleration signal enqueue and the milling force dequeue operations of the QBS. Finally, the effectiveness of the method is verified by milling tests under different milling parameters.

Convergence rates of a dual gradient method for constrained linear ill-posed problems

In this paper we consider a dual gradient method for solving linear ill-posed problems Ax = y, where A : X rightarrow Y is a bounded linear operator from a Banach space X to a Hilbert space Y. A strongly convex penalty function is used in the method to select a solution with desired feature. Under variational source conditions on the sought solution, convergence rates are derived when the method is terminated by either an a priori stopping rule or the discrepancy principle. We also consider an acceleration of the method as well as its various applications.

Weighted tensor Golub–Kahan–Tikhonov-type methods applied to image processing using a t-product

This paper discusses weighted tensor Golub–Kahan-type bidiagonalization processes using the t-product. This product was introduced in M.E. Kilmer and C.D. Martin (2011). A few steps of a bidiagonalization process with a weighted least squares norm are carried out to reduce a large-scale linear discrete ill-posed problem to a problem of small size. The weights are determined by symmetric positive definite (SPD) tensors. Tikhonov regularization is applied to the reduced problem. An algorithm for tensor Cholesky factorization of SPD tensors is presented. The data is a laterally oriented matrix or a general third order tensor. The use of a weighted Frobenius norm in the fidelity term of Tikhonov minimization problems is appropriate when the noise in the data has a known covariance matrix that is not the identity. We use the discrepancy principle to determine both the regularization parameter in Tikhonov regularization and the number of bidiagonalization steps. Applications to image and video restoration are considered.

Maximum Entropy Technique and Regularization Functional for Determining the Pharmacokinetic Parameters in DCE-MRI

This paper aims to solve the arterial input function (AIF) determination in dynamic contrast-enhanced MRI (DCE-MRI), an important linear ill-posed inverse problem, using the maximum entropy technique (MET) and regularization functionals. In addition, estimating the pharmacokinetic parameters from a DCE-MR image investigations is an urgent need to obtain the precise information about the AIF–the concentration of the contrast agent on the left ventricular blood pool measured over time. For this reason, the main idea is to show how to find a unique solution of linear system of equations generally in the form of y=Ax+b, named an ill-conditioned linear system of equations after discretization of the integral equations, which appear in different tomographic image restoration and reconstruction issues. Here, a new algorithm is described to estimate an appropriate probability distribution function for AIF according to the MET and regularization functionals for the contrast agent concentration when applying Bayesian estimation approach to estimate two different pharmacokinetic parameters. Moreover, by using the proposed approach when analyzing simulated and real datasets of the breast tumors according to pharmacokinetic factors, it indicates that using Bayesian inference—that infer the uncertainties of the computed solutions, and specific knowledge of the noise and errors—combined with the regularization functional of the maximum entropy problem, improved the convergence behavior and led to more consistent morphological and functional statistics and results. Finally, in comparison to the proposed exponential distribution based on MET and Newton’s method, or Weibull distribution via the MET and teaching–learning-based optimization (MET/TLBO) in the previous studies, the family of Gamma and Erlang distributions estimated by the new algorithm are more appropriate and robust AIFs.

On regularization via frame decompositions with applications in tomography

In this paper, we consider linear ill-posed problems in Hilbert spaces and their regularization via frame decompositions, which are generalizations of the singular-value decomposition. In particular, we prove convergence for a general class of continuous regularization methods and derive convergence rates under both a priori and a posteriori parameter choice rules. Furthermore, we apply our derived results to a standard tomography problem based on the Radon transform.

RBF approximation of three dimensional PDEs using tensor Krylov subspace methods

In this paper, we propose different algorithms for the solution of a tensor linear discrete ill-posed problem arising in the application of the meshless method for solving PDEs in three-dimensional space using multiquadric radial basis functions. It is well known that the truncated singular value decomposition (TSVD) is the most common effective solver for ill-conditioned systems, but unfortunately the operation count for solving a linear system with the TSVD is computationally expensive for large-scale matrices. In the present work, we propose algorithms based on the use of the well known Einstein product for two tensors to define the tensor global Arnoldi and the tensor Gloub Kahan bidiagonalization algorithms. Using the so-called Tikhonov regularization technique, we will be able to provide computable approximate regularized solutions in a few iterations. The tensor formulation will allow us to develop an RBF approximation for high dimensional PDEs based on Hierarchical tensors and Adaptive Cross Approximation, which in turn reduces significantly the storage and computational costs, while the accuracy of the method is preserved. The performance of the proposed methods is illustrated with a variety of benchmark examples and large-scale industrial applications with high degrees of freedom.