Linear Ill-posed Problems Research Articles

Low rank approximation in the computation of first kind integral equations with TauToolbox

Tau Toolbox is a mathematical library for the solution of integro-differential problems, based on the spectral Lanczos' Tau method. Over the past few years, a class within the library, called polynomial, has been developed for approximating functions by classical orthogonal polynomials and it is intended to be an easy-to-use yet efficient object-oriented framework.In this work we discuss how this class has been designed to solve linear ill-posed problems and we provide a description of the available methods, Tikhonov regularization and truncated singular value expansion. For the solution of the Fredholm integral equation of the first kind, which is built from a low-rank approximation of the kernel followed by a numerical truncated singular value expansion, an error estimate is given.Numerical experiments illustrate that this approach is capable of efficiently compute good approximations of linear discrete ill-posed problems, even facing perturbed available data function, with no programming effort. Several test problems are used to evaluate the performance and reliability of the solvers.The final product of this paper is the numerical solution of a first-kind integral equation, which is constructed using only two inputs from the user: the kernel and the right-hand side.

Feasibility of acousto-electric tomography

In acousto-electric tomography (AET) the goal is to reconstruct the electric conductivity in a domain from electrostatic boundary measurements of corresponding currents and voltages, while the domain is perturbed by a time-dependent acoustic wave, thus taking advantage of the acousto-electric effect. We approach the AET reconstruction in two steps: First, the interior power density is obtained from boundary measurements by solving a linear inverse and ill-posed problem; second, the interior conductivity is reconstructed from the power density by solving a non-linear and well-posed problem. Mathematically these inverse problems are fairly well understood, and reconstruction methods work well on synthetic data. This is in contrast to experimental findings. An effect can indeed be observed and data can be collected. However, the acousto-electric coupling is very weak, and consequently, the change in the measured voltage due to the acoustic perturbation might be too small compared to the background noise for viable reconstructions. In this paper, we take one step towards understanding the feasibility of AET. We provide an in-silico model of the coupled physics scenario based on standard models for the individual phenomena. Moreover, we formulate and implement numerically a full reconstruction method for the inverse problem via the two steps. We perform computational experiments with realistically chosen parameters from the context of medical imaging. The focus is on understanding the role of the acousto-electric coupling parameter and the signal-to-noise ratio (SNR). The critical signal strength is analyzed and the omnipresent Johnson–Nyquist noise is estimated. We obtain both positive and negative findings; we can reconstruct features even under severe noise conditions, but we also find that the SNR one is likely to face in practice is too low to obtain useful reconstructions.

The Regularized Block GMERR Method and Its Simpler Version for Solving Large-Scale Linear Discrete Ill-Posed Problems

The Regularized Block GMERR Method and Its Simpler Version for Solving Large-Scale Linear Discrete Ill-Posed Problems

Tractability of linear ill-posed problems in Hilbert space

We introduce a notion of tractability for ill-posed operator equations in Hilbert space. For such operator equations the asymptotics of the best possible rate of reconstruction in terms of the underlying noise level is known in many cases. However, the relevant question is, which level of discretization, again driven by the noise level, is required in order to achieve this best possible accuracy. The proposed concept adapts the one from Information-based Complexity. Several examples indicate the relevance of this concept in the light of the curse of dimensionality.

The Regularized Global GMERR Method for Solving Large-Scale Linear Discrete Ill-Posed Problems

The Regularized Global GMERR Method for Solving Large-Scale Linear Discrete Ill-Posed Problems

Augmented flexible Krylov subspace methods with applications to Bayesian inverse problems

This paper presents two new augmented flexible (AF)-Krylov subspace methods, AF-GMRES and AF-LSQR, to compute solutions of large-scale linear discrete ill-posed problems that can be modeled as the sum of two independent random variables, exhibiting smooth and sparse stochastic characteristics respectively. Following a Bayesian modeling approach, this corresponds to adding a covariance-weighted quadratic term and a sparsity enforcing ℓ1 term in the original least-squares minimization scheme. To handle the ℓ1 regularization term, the proposed approach constructs a sequence approximating quadratic problems that are partially solved using augmented flexible Krylov–Tikhonov methods.Compared to other traditional methods used to solve this minimization problem, such as those based on iteratively reweighted norm schemes, the new algorithms build a single (augmented, flexible) approximation (Krylov) subspace that encodes information about the different regularization terms through adaptable “preconditioning”. The solution space is then expanded as soon as a new problem within the sequence is defined. This also allows for the regularization parameters to be chosen on-the-fly at each iteration. Compared to most recent work on generalized flexible Krylov methods [6], our methods offer theoretical assurance of convergence and a more stable numerical performance. The efficiency of the new methods is shown through a variety of experiments, including a synthetic image deblurring problem, a synthetic atmospheric transport problem, and fluorescence molecular tomography reconstructions using both synthetic and real-world experimental data.

Nonstationary asymptotical regularization method with convex constraints for linear ill-posed problems

We investigate the method of nonstationary asymptotical regularization for solving linear ill-posed problems in Hilbert spaces. This method introduces the convex constraints that are proper lower semicontinuous and allowed to be non-smooth, therefore can be used for sparsity and discontinuity reconstruction. Under some suitable conditions , the convergence and regularity of the proposed method are established. Under the discretion of Runge–Kutta method, different iteration modes can be deduced for numerical implementation. The numerical results of iteration modes under one-stage explicit Euler, one-stage implicit Euler and two-stage explicit Runge–Kutta are presented to illustrate the efficiency of the proposed method.

Double Precision is not Necessary for LSQR for Solving Discrete Linear Ill-Posed Problems

Double Precision is not Necessary for LSQR for Solving Discrete Linear Ill-Posed Problems

Multiscale hierarchical decomposition methods for ill-posed problems

The multiscale hierarchical decomposition method (MHDM) was introduced in Tadmor et al (2004 Multiscale Model. Simul. 2 554–79; 2008 Commun. Math. Sci. 6 281–307) as an iterative method for total variation (TV) regularization, with the aim of recovering details at various scales from images corrupted by additive or multiplicative noise. Given its success beyond image restoration, we extend the MHDM iterates in order to solve larger classes of linear ill-posed problems in Banach spaces. Thus, we define the MHDM for more general convex or even nonconvex penalties, and provide convergence results for the data fidelity term. We also propose a flexible version of the method using adaptive convex functionals for regularization, and show an interesting multiscale decomposition of the data. This decomposition result is highlighted for the Bregman iteration method that can be expressed as an adaptive MHDM. Furthermore, we state necessary and sufficient conditions when the MHDM iteration agrees with the variational Tikhonov regularization, which is the case, for instance, for one-dimensional TV denoising. Finally, we investigate several particular instances and perform numerical experiments that point out the robust behavior of the MHDM.

On Convergence Rates of Proximal Alternating Direction Method of Multipliers

In this paper we consider from two different aspects the proximal alternating direction method of multipliers (ADMM) in Hilbert spaces. We first consider the application of the proximal ADMM to solve well-posed linearly constrained two-block separable convex minimization problems in Hilbert spaces and obtain new and improved non-ergodic convergence rate results, including linear and sublinear rates under certain regularity conditions. We next consider the proximal ADMM as a regularization method for solving linear ill-posed inverse problems in Hilbert spaces. When the data is corrupted by additive noise, we establish, under a benchmark source condition, a convergence rate result in terms of the noise level when the number of iterations is properly chosen.

Understanding the Feature Space and Decision Boundaries of Commercial WAFs Using Maximum Entropy in the Mean.

The security of a network requires the correct identification and characterization of the attacks through its ports. This involves the follow-up of all the requests for access to the networks by all kinds of users. We consider the frequency of connections and the type of connections to a network, and determine their joint probability. This leads to the problem of determining a joint probability distribution from the knowledge of its marginals in the presence of errors of measurement. Mathematically, this consists of an ill-posed linear problem with convex constraints, which we solved by the method of maximum entropy in the mean. This procedure is flexible enough to accommodate errors in the data in a natural way. Also, the procedure is model-free and, hence, it does not require fitting unknown parameters.

A relaxed iterated Tikhonov regularization for linear ill-posed inverse problems

Iterated Tikhonov regularization methods are prevalent regularization methods that can make the solution of ill-posed problems less sensitive to noise. As the iteration number grows, their solution gradually converges to the optimum approximation of the desired solution but then changes to the worse, causing a semiconvergence. This difficulty can be reduced by a reliable early termination rule or a strategy for choosing the appropriate regularization parameter at each iteration. Different from the existing methods, this paper describes a novel iterative Tikhonov regularization based on introducing a non-decreasing sequence of relaxation parameters into the stationary iterated Tikhonov regularization, providing a fast and roughly stable convergence. After investigating theoretically the convergence rate of the novel iterative Tikhonov regularization, we propose a series of numerical experiments for the evaluation of its accuracy and finally describe a real-world application in the field of 3-dimensional image restoration problems to verify its efficiency.

Homotopy analysis method for discrete quasi-reversibility mollification method of nonhomogeneous backward heat conduction problem

Abstract In this article, the inverse time problem is investigated. Regarding the ill-posed linear problem, utilize the quasi-reversibility method first. This problem has been regularized and after that provides an iterative regularizing strategy for noisy input data that are based on homotopy. For the regularizing solution, the error analysis is proved when we employ noisy measurement data as our initial guess. Finally, numerical implementations are presented.

Dual gradient method for ill-posed problems using multiple repeated measurement data

We consider determining -minimizing solutions of linear ill-posed problems , where is a bounded linear operator from a Banach space to a Hilbert space and is a proper strongly convex penalty function. Assuming that multiple repeated independent identically distributed unbiased data of y are available, we consider a dual gradient method to reconstruct the -minimizing solution using the average of these data. By terminating the method by either an a priori stopping rule or a statistical variant of the discrepancy principle, we provide the convergence analysis and derive convergence rates when the sought solution satisfies certain variational source conditions. Various numerical results are reported to test the performance of the method.

On the recovery of internal source for an elliptic system by neural network approximation

Abstract Consider a source detection problem for a diffusion system at its stationary status, which is stated as the inverse source problem for an elliptic equation from the measurement of the solution specified only in part of the domain. For this linear ill-posed problem, we propose to reconstruct the interior source applying neural network algorithm, which projects the problem into a finite-dimensional space by approximating both the unknown source and the corresponding solution in terms of two neural networks. By minimizing a novel loss function consisting of PDE-fit and data-fit terms but without the boundary condition fit, the modified deep Galerkin method (MDGM) is applied to solve this problem numerically. Based on the stability result for the analytic extension of the solution, we strictly estimate the generalization error caused by the MDGM algorithm employing the property of conditional stability and the regularity of the solution. Numerical experiments show that we can obtain satisfactory reconstructions even in higher-dimensional cases, and validate the effectiveness of the proposed algorithm for different model configurations. Moreover, our algorithm is stable with respect to noisy inversion input data for the noise in various structures.

Approximation of the Tikhonov regularization parameter through Aitken's extrapolation

In the present work, we study the determination of the regularization parameter and the computation of the regularized solution in Tikhonov regularization, by the Aitken's extrapolation method. In particular, this convergence acceleration method is adjusted for the approximation of quadratic forms that appear in regularization methods, such as the generalized cross-validation method, the quasi-optimality criterion, the Gfrerer/Raus method and the Morozov's discrepancy principle. We present several numerical examples to illustrate the effectiveness of the derived estimates for approximating the regularization parameter for several linear discrete ill-posed problems and we compare the described method with further existing methods, for the determination of the regularized solution.

Stochastic mirror descent method for linear ill-posed problems in Banach spaces

Consider linear ill-posed problems governed by the system for , where each A i is a bounded linear operator from a Banach space X to a Hilbert space Y i . In case p is huge, solving the problem by an iterative regularization method using the whole information at each iteration step can be very expensive, due to the huge amount of memory and excessive computational load per iteration. To solve such large-scale ill-posed systems efficiently, we develop in this paper a stochastic mirror descent method which uses only a small portion of randomly selected equations at each iteration step and incorporates convex regularization terms into the algorithm design. Therefore, our method scales very well with the problem size and has the capability of capturing features of sought solutions. The convergence property of the method depends crucially on the choice of step-sizes. We consider various rules for choosing step-sizes and obtain convergence results under a priori early stopping rules. In particular, by incorporating the spirit of the discrepancy principle we propose a choice rule of step-sizes which can efficiently suppress the oscillations in iterates and reduce the effect of semi-convergence. Furthermore, we establish an order optimal convergence rate result when the sought solution satisfies a benchmark source condition. Various numerical simulations are reported to test the performance of the method.

Numerical considerations of block GMRES methods when applied to linear discrete ill-posed problems

Linear systems of equations with a matrix whose singular values decay to zero with increasing index number without a significant gap are commonly referred to as linear discrete ill-posed problems. We are interested in solving large systems of this kind when the right-hand side has k>1 column vectors. The systems may be regarded as one system of equations with a block vector (with k columns) as the right-hand side and then solved by a block iterative method, or as k linear systems of equations (one for each right-hand side vector) that can be solved independently. Thus, the solution is a block vector with k columns. In many applications, including the restoration of color images, the right-hand side represents measurements that are contaminated by errors. Block iterative methods compute all columns of the solution block vector simultaneously. We will illustrate the performance of standard block GMRES methods and global GMRES methods, which also are block methods, and show that they may determine computed solutions of lower quality than when each column of the solution block vector is computed independently by a “standard” iterative method. We introduce a new local block GMRES method that can overcome the problems associated with block GMRES methods applied to linear discrete ill-posed problems.

Limited memory restarted ℓp-ℓq minimization methods using generalized Krylov subspaces

Regularization of certain linear discrete ill-posed problems, as well as of certain regression problems, can be formulated as large-scale, possibly nonconvex, minimization problems, whose objective function is the sum of the p th power of the ℓp-norm of a fidelity term and the q th power of the ℓq-norm of a regularization term, with 0 < p,q ≤ 2. We describe new restarted iterative solution methods that require less computer storage and execution time than the methods described by Huang et al. (BIT Numer. Math. 57,351–378, 14). The reduction in computer storage and execution time is achieved by periodic restarts of the method. Computed examples illustrate that restarting does not reduce the quality of the computed solutions.

A Study of an Iteratively-Regularized Simplified Landweber Iteration for Nonlinear Inverse Problems in Hilbert Spaces

Lanweber-type methods are a well-known regularization methods to solve linear and nonlinear ill-posed problems. In this article, we consider a simplified form of Landweber method, say, an iteratively-regularized simplified Landweber iteration for solving nonlinear ill-posed problems. We study a detailed convergence analysis of the method under standard conditions on the nonlinearity and the rate of convergence under a Hölder-type source condition. Finally, numerical simulations are performed to validate the performance of the method.