Spectral Regularization Methods Research Articles

Convergence Rates of Spectral Regularization Methods: A Comparison between Ill-Posed Inverse Problems and Statistical Kernel Learning

In this paper we study the relation between convergence rates of spectral regularization methods under Hölder-type source conditions resulting from the theory of ill-posed inverse problems, when the noise level $\delta$ goes to 0, and convergence rates resulting from statistical kernel learning, when the number of samples n goes to infinity. Toward this aim, we introduce a family of hybrid estimators in the statistical learning context whose convergence rates have the following properties: first, they are equal to those of spectral methods, and second, they are connected to the rates of spectral regularization in ill-posed inverse problems, provided that a suitable inverse proportionality relation between n and $\delta$ holds true. This family of estimators allows us to convert upper rates depending on $n$ to upper rates depending on $\delta$ and to convert lower rates vice versa, quantifying their deviation. The analysis is carried out under general source conditions in the case the rank of the forward operator is both finite and infinite, and, in the latter case, both by not making any assumptions on the eigenvalues and by assuming a polynomial eigenvalue decay.

Concentration of weakly dependent Banach-valued sums and applications to statistical learning methods

We obtain a Bernstein-type inequality for sums of Banach-valued random variables satisfying a weak dependence assumption of general type and under certain smoothness assumptions of the underlying Banach norm. We use this inequality in order to investigate in the asymptotical regime the error upper bounds for the broad family of spectral regularization methods for reproducing kernel decision rules, when trained on a sample coming from a $\tau$-mixing process.

Optimal Adaptation for Early Stopping in Statistical Inverse Problems

For linear inverse problems Y = Aµ + ξ, it is classical to recover the unknown signal µ by iterative regularization methods (µ (m) , m = 0, 1,. . .) and halt at a data-dependent iteration τ using some stopping rule, typically based on a discrepancy principle, so that the weak (or prediction) squarederror A(µ (τ) − µ) 2 is controlled. In the context of statistical estimation with stochastic noise ξ, we study oracle adaptation (that is, compared to the best possible stopping iteration) in strong squarederror E µ (τ) − µ 2. For a residual-based stopping rule oracle adaptation bounds are established for general spectral regularization methods. The proofs use bias and variance transfer techniques from weak prediction error to strong L 2-error, as well as convexity arguments and concentration bounds for the stochastic part. Adaptive early stopping for the Landweber method is studied in further detail and illustrated numerically.

Optimal Rates for Regularization of Statistical Inverse Learning Problems

We consider a statistical inverse learning (also called inverse regression) problem, where we observe the image of a function f through a linear operator A at i.i.d. random design points $$X_i$$ , superposed with an additive noise. The distribution of the design points is unknown and can be very general. We analyze simultaneously the direct (estimation of Af) and the inverse (estimation of f) learning problems. In this general framework, we obtain strong and weak minimax optimal rates of convergence (as the number of observations n grows large) for a large class of spectral regularization methods over regularity classes defined through appropriate source conditions. This improves on or completes previous results obtained in related settings. The optimality of the obtained rates is shown not only in the exponent in n but also in the explicit dependency of the constant factor in the variance of the noise and the radius of the source condition set.

Characterizations of Variational Source Conditions, Converse Results, and Maxisets of Spectral Regularization Methods

We describe a general strategy for the verification of variational source condition by formulating two sufficient criteria describing the smoothness of the solution and the degree of ill-posedness of the forward operator in terms of a family of subspaces. For linear deterministic inverse problems we show that variational source conditions are necessary and sufficient for convergence rates of spectral regularization methods, which are slower than the square root of the noise level. A similar result is shown for linear inverse problems with white noise. In many cases variational source conditions can be characterized by Besov spaces. This is discussed for a number of prominent inverse problems.

Spectral Regularization Methods for an Abstract Ill-Posed Elliptic Problem

We study an abstract elliptic Cauchy problem associated with an unbounded self-adjoint positive operator which has a continuous spectrum. It is well-known that such a problem is severely ill-posed; that is, the solution does not depend continuously on the Cauchy data. We propose two spectral regularization methods to construct an approximate stable solution to our original problem. Finally, some other convergence results including some explicit convergence rates are also established under a priori bound assumptions on the exact solution.

On the existence of global saturation for spectral regularization methods with optimal qualification

Abstract. A family of real functions defining a spectral regularization method with optimal qualification is considered. Sufficient condition on the family and on the optimal qualification guaranteeing the existence of saturation are established. Appropriate characterizations of both the saturation function and the saturation set are found and some examples are provided.

OPTIMAL ERROR BOUND AND APPROXIMATION METHODS FOR A CAUCHY PROBLEM OF THE MODIFIED HELMHOLTZ EQUATION

We consider a Cauchy problem for a modified Helmholtz equation, especially when we give the optimal error bound for this problem. Some spectral regularization methods and a revised Tikhonov regularization method are used to stabilize the problem from the viewpoint of general regularization theory. Hölder-type stability error estimates are provided for these regularization methods. According to the optimality theory of regularization, the error estimates are order optimal.

On universal oracle inequalities related to high-dimensional linear models

This paper deals with recovering an unknown vector θ from the noisy data Y = Aθ + σξ, where A is a known (m × n)-matrix and ξ is a white Gaussian noise. It is assumed that n is large and A may be severely ill-posed. Therefore, in order to estimate θ, a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data Y. For spectral regularization methods related to the so-called ordered smoothers [see Kneip Ann. Statist. 22 (1994) 835–866], we propose new penalties in the principle of empirical risk minimization. The heuristical idea behind these penalties is related to balancing excess risks. Based on this approach, we derive a sharp oracle inequality controlling the mean square risks of data-driven spectral regularization methods.

Global Saturation of Regularization Methods for Inverse Ill-Posed Problems

In this article the concept of saturation of an arbitrary regularization method is formalized based upon the original idea of saturation for spectral regularization methods introduced by A. Neubauer in 1994. Necessary and sufficient conditions for a regularization method to have global saturation are provided. It is shown that for a method to have global saturation the total error must be optimal in two senses, namely as optimal order of convergence over a certain set which at the same time, must be optimal (in a very precise sense) with respect to the error. Finally, two converse results are proved and the theory is applied to find sufficient conditions which ensure the existence of global saturation for spectral methods with classical qualification of finite positive order and for methods with maximal qualification. Finally, several examples of regularization methods possessing global saturation are shown.

Generalized Qualification and Qualification Levels for Spectral Regularization Methods

The concept of qualification for spectral regularization methods (SRM) for inverse ill-posed problems is strongly associated to the optimal order of convergence of the regularization error (Engl et al. in Regularization of inverse problems. Mathematics and its applications, vol. 375, Kluwer Academic, Dordrecht, 1996; Mathé in SIAM J. Numer. Anal. 42(3):968–973, 2004; Mathé and Pereverzev in Inverse Probl. 19(3):789–803, 2003; Vainikko in USSR Comput. Math. Math. Phys. 22(3): 1–19, 1982). In this article, the definition of qualification is extended and three different levels are introduced: weak, strong and optimal. It is shown that the weak qualification extends the definition introduced by Mathé and Pereverzev (Inverse Probl. 19(3):789–803, 2003), mainly in the sense that the functions associated with orders of convergence and source sets need not be the same. It is shown that certain methods possessing infinite classical qualification (e.g. truncated singular value decomposition (TSVD), Landweber’s method and Showalter’s method) also have generalized qualification leading to an optimal order of convergence of the regularization error. Sufficient conditions for a SRM to have weak qualification are provided and necessary and sufficient conditions for a given order of convergence to be strong or optimal qualification are found. Examples of all three qualification levels are provided and the relationships between them as well as with the classical concept of qualification and the qualification introduced in Mathé and Pereverzev (Inverse Probl. 19(3):789–803, 2003) are shown. In particular, SRMs having extended qualification in each one of the three levels and having zero or infinite classical qualification are presented. Finally, several implications of this theory in the context of orders of convergence, converse results and maximal source sets for inverse ill-posed problems, are shown.

Spectral regularization methods for solving a sideways parabolic equation within the framework of regularization theory

We introduce three spectral regularization methods for solving a general sideways parabolic equation. For these three spectral regularization methods, we give some stability error estimates with optimal order under a-priori and a-posteriori regularization parameter choice rules. Numerical results show that these spectral methods are effective.

ON THREE SPECTRAL REGULARIZATION METHODS FOR A BACKWARD HEAT CONDUCTION PROBLEM

We introduce three spectral regularization methods for solving a backward heat conduction problem (BHCP). For the three spectral regularization methods, we give the stability error estimates with optimal order under an a-priori and an a-posteriori regularization parameter choice rule. Numerical results show that our theoretical results are effective.

Two approximate methods of a Cauchy problem for the Helmholtz equation

In this paper, we consider a Cauchy problem for the Helmholtz equation at fixed frequency, especially we give the optimal error bound for the ill-posed problem. Within the framework of general regularization theory, we present some spectral regularization methods and a modified Tikhonov regularization method to stabilize the problem. Moreover, Holder-type stability error estimates are proved for these regularization methods. According to the regularization theory, the error estimates are order optimal. Some numerical results are reported.

Wavelet and spectral regularization methods for a sideways parabolic equation

We consider a special sideways parabolic equation which appears in some applied subjects. The problem is ill-posed in the sense that the solution (if it exists) does not depend continuously on the data. In this paper some wavelet and spectral regularization methods for this problem are given. The error estimates are also established respectively.