Solution Of Linear Ill-posed Problems Research Articles

Dual gradient method for ill-posed problems using multiple repeated measurement data

We consider determining -minimizing solutions of linear ill-posed problems , where is a bounded linear operator from a Banach space to a Hilbert space and is a proper strongly convex penalty function. Assuming that multiple repeated independent identically distributed unbiased data of y are available, we consider a dual gradient method to reconstruct the -minimizing solution using the average of these data. By terminating the method by either an a priori stopping rule or a statistical variant of the discrepancy principle, we provide the convergence analysis and derive convergence rates when the sought solution satisfies certain variational source conditions. Various numerical results are reported to test the performance of the method.

Convergence Rates of First- and Higher-Order Dynamics for Solving Linear Ill-Posed Problems

Recently, there has been a great interest in analysing dynamical flows, where the stationary limit is the minimiser of a convex energy. Particular flows of great interest have been continuous limits of Nesterov’s algorithm and the fast iterative shrinkage-thresholding algorithm, respectively. In this paper, we approach the solutions of linear ill-posed problems by dynamical flows. Because the squared norm of the residual of a linear operator equation is a convex functional, the theoretical results from convex analysis for energy minimising flows are applicable. However, in the restricted situation of this paper they can often be significantly improved. Moreover, since we show that the proposed flows for minimising the norm of the residual of a linear operator equation are optimal regularisation methods and that they provide optimal convergence rates for the regularised solutions, the given rates can be considered the benchmarks for further studies in convex analysis.

A New Algorithm for a Posteriori Error Estimation for Approximate Solutions of Linear Ill-Posed Problems

A new algorithm for a posteriori estimation of the error in solutions to linear operator equations of the first kind in a Hilbert space is proposed and justified. The algorithm reduces the variational problem of a posteriori error estimation to two special problems of maximizing smooth functionals under smooth constraints. A finite-dimensional version of the algorithm is considered. The results of a numerical experiment concerning a posteriori error estimation for a typical inverse problem are presented. It is shown experimentally that the computation time required by the algorithm is less, on average, by a factor of 1.4 than in earlier proposed methods.

Augmented GMRES-type versus CGNE methods for the solution of linear ill-posed problems

In this paper we compare (augmented) GMRES-type methods and (augmented) CGNE methods. The numerical results show that the CGNE method is more robust and suitable for ill-posed problems with a much higher degree of ill-posedness. GMRES-type methods only yield useful results for very moderately ill-posed problems.

Solution of Linear Ill-Posed Problems Using Random Dictionaries

In the present paper, we consider an application of overcomplete dictionaries to the solution of general ill-posed linear inverse problems. In the context of regression problems, there has been an enormous amount of effort to recover an unknown function using such dictionaries. One of the most popular methods, lasso, and its versions, is based on minimizing the empirical likelihood and unfortunately, requires stringent assumptions on the dictionary, the so-called, compatibility conditions. Though compatibility conditions are hard to satisfy, it is well known that this can be accomplished by using random dictionaries. In the present paper, we show how one can apply random dictionaries to the solution of ill-posed linear inverse problems. We put a theoretical foundation under the suggested methodology and study its performance via simulations and real-data example.

Solution of linear ill-posed problems by model selection and aggregation

We consider a general statistical linear inverse problem, where the solution is represented via a known (possibly overcomplete) dictionary that allows its sparse representation. We propose two different approaches. A model selection estimator selects a single model by minimizing the penalized empirical risk over all possible models. By contrast with direct problems, the penalty depends on the model itself rather than on its size only as for complexity penalties. A Q-aggregate estimator averages over the entire collection of estimators with properly chosen weights. Under mild conditions on the dictionary, we establish oracle inequalities both with high probability and in expectation for the two estimators. Moreover, for the latter estimator these inequalities are sharp. The proposed procedures are implemented numerically and their performance is assessed by a simulation study.

Solution of linear ill-posed problems using overcomplete dictionaries

In the present paper, we consider the application of overcomplete dictionaries to the solution of general ill-posed linear inverse problems. In the context of regression problems, there has been an enormous amount of effort to recover an unknown function using an overcomplete dictionary. One of the most popular methods, Lasso and its variants, is based on maximizing the likelihood, and relies on stringent assumptions on the dictionary, the so-called compatibility conditions, for a proof of its convergence rates. While these conditions may be satisfied for the original dictionary functions, they usually do not hold for their images due to contraction properties imposed by the linear operator. In what follows, we bypass this difficulty by a novel approach, which is based on inverting each of the dictionary functions and matching the resulting expansion to the true function, thus, avoiding unrealistic assumptions on the dictionary and using Lasso in a predictive setting. We examine both the white noise and the observational model formulations, and also discuss how exact inverse images of the dictionary functions can be replaced by their approximate counterparts. Furthermore, we show how the suggested methodology can be extended to the problem of estimation of a mixing density in a continuous mixture. For all the situations listed above, we provide sharp oracle inequalities for the risk in a non-asymptotic setting.

On the regularizing behavior of the SDA and SDC gradient methods in the solution of linear ill-posed problems

We analyze the regularization properties of two recently proposed gradient methods, SDA and SDC, applied to discrete linear inverse problems. By studying their filter factors, we show that the tendency of these methods to eliminate first the eigencomponents of the gradient corresponding to large singular values allows to reconstruct the most significant part of the solution, thus yielding a useful filtering effect. This behavior is confirmed by numerical experiments performed on some image restoration problems. Furthermore, the experiments show that, for severely ill-conditioned problems and high noise levels, the SDA and SDC methods can be competitive with the Conjugate Gradient (CG) method, since they are slightly slower than CG, but exhibit a better semiconvergence behavior.

Near-Optimal Spectral Filtering and Error Estimation for Solving Ill-Posed Problems

We consider regularization methods for numerical solution of linear ill-posed problems, in particular image deblurring, when the singular value decomposition (SVD) of the operator is available. We assume that the noise-free problem satisfies the discrete Picard condition and define the Picard parameter, the index beyond which the data, expressed in the coordinate system of the SVD, are dominated by noise. We propose estimating the Picard parameter graphically or using standard statistical tests. Having this parameter available allows us to estimate the mean and standard deviation of the noise and drop noisy components, thus making filtered solutions much more reliable. We show how to compute a near-optimal choice of filter parameters for any filter. This includes the truncated SVD (TSVD) filter, the truncated singular component method (TSCM) filter, and several new filters which we define, including a truncated Tikhonov filter, a Tikhonov-TSVD filter, a Heaviside filter, and a spline filter. We show how to estimate the error in any spectral filter, regardless of how the filter parameters are chosen. We demonstrate the usefulness of our new filters, our near-optimal choice of parameters, and our error estimates for restoring blurred images.

Regularization of linear ill-posed problems by the augmented Lagrangian method and variational inequalities

We study the application of the augmented Lagrangian method to the solution of linear ill-posed problems. Previously, linear convergence rates with respect to the Bregman distance have been derived under the classical assumption of a standard source condition. Using the method of variational inequalities, we extend these results in this paper to convergence rates of lower order, both for the case of an a priori parameter choice and an a posteriori choice based on Morozov’s discrepancy principle. In addition, our approach allows the derivation of convergence rates with respect to distance measures different from the Bregman distance. As a particular application, we consider sparsity promoting regularization, where we derive a range of convergence rates with respect to the norm under the assumption of restricted injectivity in conjunction with generalized source conditions of Hölder type.

A New Conjugate Gradient Method for the Solution of Linear Ill-Posed Problem

A new conjugate gradient method is proposed for solving the linear ill-posed problem and the application to the identification of the multi-source dynamic loads on a surface of simply supported plate. The algorithm considered here is detailedly given and proved that the computational costs for the present method are nearly the same as the common conjugate gradient method, but the number of iteration steps is even less. Finally, the performances of numerical simulations are given, and verify the favorable theoretical properties of the present method.

Cascadic multilevel methods for ill-posed problems

Multilevel methods are popular for the solution of well-posed problems, such as certain boundary value problems for partial differential equations and Fredholm integral equations of the second kind. However, little is known about the behavior of multilevel methods when applied to the solution of linear ill-posed problems, such as Fredholm integral equations of the first kind, with a right-hand side that is contaminated by error. This paper shows that cascadic multilevel methods with a conjugate gradient-type method as basic iterative scheme are regularization methods. The iterations are terminated by a stopping rule based on the discrepancy principle.

Some approaches to reconstruction of nonsmooth solutions of linear ill-posed problems

The short review of methods of reconstruction non-smooth solutions to ill-posed problems is presented. In the Tikhonov method for some parametric classes of stabilizing functionals the convergence theorems for regularized solutions and their discrete approximations are stated. The questions concerning the numerical algorithms are considered.

A new L-curve for ill-posed problems

The truncated singular value decomposition is a popular method for the solution of linear ill-posed problems. The method requires the choice of a truncation index, which affects the quality of the computed approximate solution. This paper proposes that an L-curve, which is determined by how well the given data (right-hand side) can be approximated by a linear combination of the first (few) left singular vectors (or functions), be used as an aid for determining the truncation index.

Approximation of nonsmooth solutions of linear ill-posed problems

In the multidimensional case, for the Tikhonov regularization, two new families of stabilizers containing the norms of the Lipschitz spaces and the norms of the Sobolev spaces with fractional derivatives are suggested. Theorems of convergence of Tikhonov regularized approximate solutions and their discrete approximations are proved. Detailed step-by-step investigation of the solving algorithm is performed by the example of an integral Fredholm equation of the first kind.

A descent method for regularization of ill-posed problems

In this paper, we describe an iterative algorithm, called descent-TCG, based on truncated conjugate gradients iterations to compute Tikhonov regularized solutions of linear ill-posed problems. The sequence of approximate solutions and regularization parameters, computed by the algorithm, is shown to decrease the value of the Tikhonov functional. Suitable termination criteria are built-up to define an inner–outer iteration scheme that computes a regularized solution. Numerical experiments are performed to compare this algorithm with other well established regularization methods. We observe that the best descent-TCG results occur for highly noised data and we always get fairly reliable solutions, thus it prevents the dangerous error growth often appearing in other well established regularization methods. Finally, the descent-TCG method is computationally advantageous especially for large size problems.

Non-negativity and iterative methods for ill-posed problems

The Generalized Minimal RESidual (GMRES) method and conjugate gradient method applied to the normal equations (CGNR) are popular iterative schemes for the solution of large linear systems of equations. GMRES requires the matrix of the linear system to be square and nonsingular, while CGNR also can be applied to overdetermined or underdetermined linear systems of equations. When equipped with a suitable stopping rule, both GMRES and CGNR are regularization methods for the solution of linear ill-posed problems. Many linear ill-posed problems that arise in the sciences and engineering have non-negative solutions. This paper describes iterative schemes, based on the GMRES or CGNR methods, for the computation of non-negative solutions of linear ill-posed problems. The computations with these schemes are terminated as soon as a non-negative approximate solution which satisfies the discrepancy principle has been found. Several computed examples illustrate that the schemes of this paper are able to compute non-negative approximate solutions of higher quality with less computational effort than several available numerical methods.

Tikhonov Regularization with a Solution Constraint

Many numerical methods for the solution of linear ill-posed problems apply Tikhonov regularization. This paper presents a modification of a numerical method proposed by Golub and von Matt for quadratically constrained least-squares problems and applies it to Tikhonov regularization of large-scale linear discrete ill-posed problems. The method is based on partial Lanczos bidiagonalization and Gauss quadrature. Computed examples illustrating its performance are presented.

TIKHONOV REGULARIZATION OF LARGE LINEAR PROBLEMS

Many numerical methods for the solution of linear ill-posed problems apply Tikhonov regularization. This paper presents a new numerical method, based on Lanczos bidiagonalization and Gauss quadrature, for Tikhonov regularization of large-scale problems. An estimate of the norm of the error in the data is assumed to be available. This allows the value of the regularization parameter to be determined by the discrepancy principle.

New approaches to error estimation to ill-posed problems with applications to inverse problems of heat conductivity

Abstract - In this paper several new approaches for error estimation of an approximate solution of linear ill-posed problems are offered in the presence of the additional a priori information about the exact solution. This information is used when the exact solution belongs to a compact set or is sourcewise represented. All considered problems are reduced to convex programming problems in finite dimensional Euclidean spaces. The method to cut convex polyhedrons is applied to solve these problems and to construct areas, which the exact solutions belong to. Several model inverse problems for the heat conduction equation are considered.