Methods For Linear Ill-posed Problems Research Articles

Stochastic mirror descent method for linear ill-posed problems in Banach spaces

Consider linear ill-posed problems governed by the system for , where each A i is a bounded linear operator from a Banach space X to a Hilbert space Y i . In case p is huge, solving the problem by an iterative regularization method using the whole information at each iteration step can be very expensive, due to the huge amount of memory and excessive computational load per iteration. To solve such large-scale ill-posed systems efficiently, we develop in this paper a stochastic mirror descent method which uses only a small portion of randomly selected equations at each iteration step and incorporates convex regularization terms into the algorithm design. Therefore, our method scales very well with the problem size and has the capability of capturing features of sought solutions. The convergence property of the method depends crucially on the choice of step-sizes. We consider various rules for choosing step-sizes and obtain convergence results under a priori early stopping rules. In particular, by incorporating the spirit of the discrepancy principle we propose a choice rule of step-sizes which can efficiently suppress the oscillations in iterates and reduce the effect of semi-convergence. Furthermore, we establish an order optimal convergence rate result when the sought solution satisfies a benchmark source condition. Various numerical simulations are reported to test the performance of the method.

Stochastic asymptotical regularization for linear inverse problems

We introduce stochastic asymptotical regularization (SAR) methods for the uncertainty quantification of the stable approximate solution of ill-posed linear-operator equations, which are deterministic models for numerous inverse problems in science and engineering. We demonstrate that SAR can quantify the uncertainty in error estimates for inverse problems. We prove the regularizing properties of SAR with regard to mean-square convergence. We also show that SAR is an order-optimal regularization method for linear ill-posed problems provided that the terminating time of SAR is chosen according to the smoothness of the solution. This result is proven for both a priori and a posteriori stopping rules under general range-type source conditions. Furthermore, some converse results of SAR are verified. Two iterative schemes are developed for the numerical realization of SAR, and the convergence analyses of these two numerical schemes are also provided. A toy example and a real-world problem of biosensor tomography are studied to show the accuracy and the advantages of SAR: compared with the conventional deterministic regularization approaches for deterministic inverse problems, SAR can provide the uncertainty quantification of the quantity of interest, which can in turn be used to reveal and explicate the hidden information about real-world problems, usually obscured by the incomplete mathematical modeling and the ascendence of complex-structured noise.

Optimal-order convergence of Nesterov acceleration for linear ill-posed problems* *Dedicated to A Neubauer on the occasion of his 60th birthday.

We show that Nesterov acceleration is an optimal-order iterative regularization method for linear ill-posed problems provided that a parameter is chosen accordingly to the smoothness of the solution. This result is proven both for an a priori stopping rule and for the discrepancy principle under Hölder source conditions. Furthermore, some converse results and logarithmic rates are verified. The essential tool to obtain these results is a representation of the residual polynomials via Gegenbauer polynomials.

An entropic Landweber method for linear ill-posed problems

The aim of this paper is to investigate the use of a Landweber-type method involving the Shannon entropy for the regularization of linear ill-posed problems. We derive a closed form solution for the iterates and analyze their convergence behaviour both in a case of reconstructing general nonnegative unknowns as well as for the sake of recovering probability distributions. Moreover, we discuss several variants of the algorithm and relations to other methods in the literature. The effectiveness of the approach is studied numerically in several examples.

Analysis of the Block Coordinate Descent Method for Linear Ill-Posed Problems

Block coordinate descent (BCD) methods approach optimization problems by performing gradient steps along alternating subgroups of coordinates. This is in contrast to full gradient descent, where a gradient step updates all coordinates simultaneously. BCD has been demonstrated to accelerate the gradient method in many practical large-scale applications. Despite its success no convergence analysis for inverse problems is known so far. In this paper, we investigate the BCD method for solving linear inverse problems. As the main theoretical result, we show that for operators having a particular tensor product form, the BCD method combined with an appropriate stopping criterion yields a convergent regularization method. To illustrate the theory, we perform numerical experiments comparing the BCD and the full gradient descent method for a system of integral equations. We also present numerical tests for a nonlinear inverse problem not covered by our theory, namely one-step inversion in multispectral X-ray tomography.

Some novel linear regularization methods for a deblurring problem

In this article, we consider a fractional backward heat conduction problem (BHCP) in the two-dimensional space which is associated with a deblurring problem. It is well-known that the classical Tikhonov method is the most important regularization method for linear ill-posed problems. However, the classical Tikhonov method over-smooths the solution. As a remedy, we propose two quasi-boundary regularization methods and their variants. We prove that these two methods are better than Tikhonov method in the absence of noise in the data. Deblurring experiment is conducted by comparing with some classical linear filtering methods for BHCP and the total variation method with the proposed methods.

Some Properties of the Arnoldi-Based Methods for Linear Ill-Posed Problems

In this paper we study some properties of the classical Arnoldi-based methods for solving infinite dimensional linear equations involving compact operators. These problems are intrinsically ill-posed since a compact operator does not admit a bounded inverse. We study the convergence properties and the ability of these algorithms to estimate the dominant singular values of the operator.

Accelerated Landweber methods based on co-dilated orthogonal polynomials

In this article, we introduce and study accelerated Landweber methods for linear ill-posed problems obtained by an alteration of the coefficients in the three-term recurrence relation of the ν-methods. The residual polynomials of the semi-iterative methods under consideration are linked to a family of co-dilated ultraspherical polynomials. This connection makes it possible to control the decay of the residual polynomials at the origin by means of a dilation parameter. Depending on the data, the approximation error of the ν-methods can be improved by altering this dilation parameter. The convergence order of the new semi-iterative methods turns out to be the same as the convergence order of the original ν-methods. The new algorithms are tested numerically and a simple adaptive scheme is developed in which an optimal dilation parameter is computed in every iteration step.

Wavelet-based multilevel methods for linear ill-posed problems

The representation of linear operator equations in terms of wavelet bases yields a multilevel framework, which can be exploited for iterative solution. This paper describes cascadic multilevel methods that employ conjugate gradient-type methods on each level. The iterations are on each level terminated by a stopping rule based on the discrepancy principle.

On convergence of local averaging regression function estimates for the regularization of inverse problems

In this paper, we combine the ideas of regression function estimation on the one hand and regularization by discretization on the other hand to a regularization method for linear ill-posed problems with additive stochastic noise. A general convergence result is provided and its assumptions are verified for the partitioning estimators and to some extent for kernel estimators. As a class of examples of inverse problems, we consider Volterra integral equations of the first kind.

Nonlinear iterative methods for linear ill-posed problems in Banach spaces

We introduce and discuss nonlinear iterative methods to recover the minimum-norm solution of the operator equation Ax = y in Banach spaces X, Y, where A is a continuous linear operator from X to Y. The methods are nonlinear due to the use of duality mappings which reflect the geometrical aspects of the underlying spaces. The space X is required to be smooth and uniformly convex, whereas Y can be an arbitrary Banach space. The case of exact as well as approximate and disturbed data and operator are taken into consideration and we prove the strong convergence of the sequence of the iterates.

Saturation of Regularization Methods for Linear Ill-Posed Problems in Hilbert Spaces

We prove the saturation of methods for solving linear ill-posed problems in Hilbert spaces for a wide class of regularization methods. It turns out that, under a certain convexity assumption, saturation must necessarily occur. We provide easy to verify assumptions, which allow us to calculate the rate at which saturation occurs.

A unified approach to regularization methods for linear ill-posed problems

The aim of this paper is to study regularization methods for linear ill-posed problems. Linear methods are Tikhonov-Phillips methods, iterative methods, truncated singular value decomposition, Backus-Gilbert-type methods and approximate inverse, for example. The first three are generally studied as filter methods where a special filter for the singular value decomposition can be computed. In the other methods mentioned the regularization is achieved by either smoothing the data or the solution. More general is the approximate inverse introduced by Louis (1996 Inverse Problems 12 175-90). Here we show that all these methods can be viewed either as smoothing the pseudo-inverse or equivalently as first smoothing the data and then applying the pseudo-inverse. The smoothing of the data or of the pseudo-inverse has to be at least of the order of the smoothing of the operator in the problem to be solved. Conditions for the order-optimality of the methods are given.

The conjugate gradient method for linear ill-posed problems with operator perturbations

We consider an ill-posed problem Ta = f* in Hilbert spaces and suppose that the linear bounded operator T is approximately available, with a known estimate for the operator perturbation at the solution. As a numerical scheme the CGNR-method is considered, that is, the classical method of conjugate gradients by Hestenes and Stiefel applied to the associated normal equations. Two a posteriori stopping rules are introduced, and convergence results are provided for the corresponding approximations, respectively. As a specific application, a parameter estimation problem is considered.

Maximum entropy method for solving operator equations of the first kind

The maximum entropy method for linear ill-posed problems with modeling error and noisy data is considered and the stability and convergence results are obtained. When the maximum entropy solution satisfies the "source condition", suitable rates of convergence can be derived. Considering the practical applications, an a posteriori choice for the regularization parameter is presented. As a byproduct, a characterization of the maximum entropy regularized solution is given.

On Converse and Saturation Results for Tikhonov Regularization of Linear Ill-Posed Problems

In this paper we prove some new converse and saturation results for Tikhonov regularization of linear ill-posed problems Tx=y, where T is a linear operator between two Hilbert spaces.

A convergence criterion for approximations in the residual method for linear ill-posed problems

A convergence criterion for approximations in the residual method for linear ill-posed problems

On the condition number of matrices arising in the tikhonov regularization method

In this work, we consider the regularization method for linear ill-posed problems. For operators and approximating subspaces satisfying certain conditions and for a specific form of the regularization parameter, upper and lower bounds are given for the condition number of the corresponding discrete problem.

Uniform convergence of regularization methods for linear ill-posed problems

After a general discussion about convergence and convergence rates for regularization methods in Banach spaces, we present a general method that can be used to modify regularization methods in L 2 in such a way that uniform convergence, which is often preferred in concrete applications to just L 2-convergence, is obtained. We prove results about convergence rates in the uniform norm and discuss questions of parameter choice. The theoretical results will be supported by numerical examples, which indicate that although the results are asymptotic in character, they are of some relevance also for actual computations.

On the Optimality of Methods for Ill-Posed Problems

The optimality of a class of regularization methods for linear ill-posed problems is investigated. The general results are applied to Lavrentjew’s and Tikhonov’s methods and to a class of iteration methods and their continuous versions.