Tikhonov Regularization Schemes Research Articles

Performance comparison of commonly used photoacoustic tomography reconstruction algorithms under various blurring conditions

A reconstructed image in photoacoustic tomography (PAT) imaging modality can be blurred because of some system-dependent effects – broad laser pulse, the limited bandwidth of the sensors, acoustically attenuating medium and transducers with finite-size apertures having Gaussian sensitivity profile. A robust deblurring method involving a system matrix is presented in this work. Image reconstruction was accomplished by using the backprojection (BP), interpolated time reversal (ITR), Tikhonov regularization (TH) and total variation minimization (TV) schemes. A modified version of the BP algorithm was also explored for image reconstruction for finite transducers. Each factor of blurring was considered at a time. For the model-based methods, the system matrix was constructed in presence of such a factor. Subsequently, the model matrix inversion method was deployed for image formation. Quantitative values of some standard metrics were computed to assess the performance of the reconstruction protocols. The visual inspection of the reconstructed images as well as numerical values of the figures of merit revealed that the BP and ITR techniques generally fail to remove blurring; however, the model-based methods can provide distortion-free images under various conditions. For example, the numerical values of the Pearson correlation coefficient (PCC) for all the cases showed a strong correlation between the nominal and reconstructed images for the TH and TV techniques compared to the BP and ITR methods. The TV method can be utilized in practice to facilitate the PAT image reconstruction free from system-dependent blurring.

Material profile reconstruction using plane electromagnetic waves in PML-truncated heterogeneous domains

This paper discusses a full-waveform inversion method for reconstructing the spatial distribution of permittivity in heterogeneous infinite domains, using measured electric field intensities at sparse sensor locations. In solving the electromagnetic wave problems numerically, perfectly matched layer (PML) absorbing boundaries are used to truncate the originally infinite extent to a finite computational domain of interest without introducing significant reflections. The full-waveform inversion is based on an optimization scheme where Maxwell’s equations endowed with the PML for plane electromagnetic waves are imposed as constraints. The approach seeks the optimal solution of permittivity profile to minimize the objective functional comprising the L2-norm of a misfit between calculated and measured electric fields. For casting the problem to an unconstrained optimization problem, a Lagrangian is constructed augmenting the objective functional with the PML-endowed Maxwell’s equations via Lagrange multipliers. Enforcing the stationarity of the Lagrangian yields time-dependent state, adjoint, and time-invariant control problems, which constitute Karush-Kuhn-Tucker (KKT) conditions for optimal solutions. The permittivity profile of the PML-truncated domain is iteratively updated by solving the KKT conditions in the reduced space of the control variable. A conjugate gradient method with inexact line search is used to update the permittivity profile in each inversion iteration. Tikhonov and total variation regularization schemes are explored to relieve the ill-posedness of the inverse problem. Through a set of numerical results, it is shown that both smooth and sharply-varying permittivity profiles can be recovered successfully using the proposed inversion method.

Optimal Margin Distribution Additive Machine

In recent years, sparse additive machines have attracted increasing attention in high dimensional classification due to their flexibility and representation interpretability. However, most of the existing methods are formulated under Tikhonov regularization schemes associated with the hinge loss, where the distribution information of observations is neglected usually. To circumvent this problem, we propose an optimal margin distribution additive machine (called ODAM) by incorporating the optimal margin distribution strategy into sparse additive models. The proposed approach can be implemented by a dual coordinate descent algorithm and its empirical effectiveness is confirmed on simulated and benchmark datasets.

Optimal learning with Gaussians and correntropy loss

Correntropy-based learning has achieved great success in practice during the last decades. It is originated from information-theoretic learning and provides an alternative to classical least squares method in the presence of non-Gaussian noise. In this paper, we investigate the theoretical properties of learning algorithms generated by Tikhonov regularization schemes associated with Gaussian kernels and correntropy loss. By choosing an appropriate scale parameter of Gaussian kernel, we show the polynomial decay of approximation error under a Sobolev smoothness condition. In addition, we employ a tight upper bound for the uniform covering number of Gaussian RKHS in order to improve the estimate of sample error. Based on these two results, we show that the proposed algorithm using varying Gaussian kernel achieves the minimax rate of convergence (up to a logarithmic factor) without knowing the smoothness level of the regression function.

Tikhonov Regularization Based Modeling and Sidereal Filtering Mitigation of GNSS Multipath Errors

In Global Navigation Satellite System (GNSS) relative positioning applications, multipath errors are non-negligible. Mitigation of the multipath error is an important task for precise positioning and it is possible due to the repeatability, even without any rigorous mathematical model. Empirical modeling is required for this mitigation. In this work, the multipath error modeling using carrier phase measurement residuals is realized by solving a regularization problem. Two Tikhonov regularization schemes, namely with the first and the second order differences, are considered. For each scheme, efficient numerical algorithms are developed to find the solutions, namely the Thomas algorithm and Cholesky rank-one update algorithm for the first and the second differences, respectively. Regularization parameters or Lagrange multipliers are optimized using the bootstrap method. In experiment, data on the first day are processed to construct a multipath model for each satellite (except the reference one), and then the model is used to correct the measurement on the second day, namely following the sidereal filtering approach. The smoothness of the coordinates calculated using the corrected measurements is improved significantly compared to those using the raw measurement. The efficacy of the proposed method is illustrated by the actual calculation.

The extraction of depth profiles from simulated ARXPS data: From parametric models to regularization methods

The extraction of concentration depth profiles from ARXPS data has been carried out using noisy simulated data and two different approaches, using either simple parametric models or general algorithms with Tikhonov regularization schemes. Among the single parametric models the only one that is stable and robust against noise is that using only one parameter, with the uncertainty of the parameter displaying a linear dependence on the input noise level. For Tikhonov regularization schemes, a guide is given to choose the appropriate regularization parameter, which is based on the use of the S-curve in conjunction with the constraints introduced by χ2, and which provides user-independent results.

Evaluating the conductivity distribution in isotropic polycrystalline graphite using spectroscopic eddy current technique for monitoring weight loss in advanced gas cooled reactors

Recently studies have shown correlation between the conductivity of graphite and its weight loss (density). This paper presents an eddy current technique for evaluating the conductivity profiles within 100mm thick layered graphite sections. Flat graphite plates with three different grades of density for which the corresponding conductivities were measured, have been used to construct these sections. An initial test was performed to examine the capability of the eddy current technique to penetrate to a 100mm depth in graphite. Then spectroscopic inductance measurements were acquired from different test profiles and compared with finite element method (FEM) simulated data. Conductivity profiles were reconstructed from the experimental data via the solution of an inverse eddy current problem. For an inverse solution, direct Tikhonov regularization and iterative Gauss Newton reconstruction schemes have been employed. Results show that the inverse methods are able to produce reasonable estimates of the conductivity profiles.

Learning Rates for Regularized Classifiers Using Trigonometric Polynomial Kernels

Regularized classifiers are known to be a kind of kernel-based classification methods generated from Tikhonov regularization schemes, and the trigonometric polynomial kernels are ones of the most important kernels and play key roles in signal processing. The main target of this paper is to provide convergence rates of classification algorithms generated by regularization schemes with trigonometric polynomial kernels. As a special case, an error analysis for the support vector machines (SVMs) soft margin classifier is presented. The norms of Fejer operator in reproducing kernel Hilbert space and properties of approximation of the operator in L 1 space with periodic function play key roles in the analysis of regularization error. Some new bounds on the learning rate of regularization algorithms based on the measure of covering number for normalized loss functions are established. Together with the analysis of sample error, the explicit learning rates for SVM are also derived.

Conditional quantiles with varying Gaussians

In this paper we study conditional quantile regression by learning algorithms generated from Tikhonov regularization schemes associated with pinball loss and varying Gaussian kernels. Our main goal is to provide convergence rates for the algorithm and illustrate differences between the conditional quantile regression and the least square regression. Applying varying Gaussian kernels improves the approximation ability of the algorithm. Bounds for the sample error are achieved by using a projection operator, a variance-expectation bound derived from a condition on conditional distributions and a tight bound for the covering numbers involving the Gaussian kernels.

The inverse medium problem in heterogeneous PML-truncated domains using scalar probing waves

We discuss the inverse medium problem associated with semi-infinite domains. In particular, we attempt to image the spatial variability of shear moduli or shear wave velocities from scant surficial measurements of an arbitrarily heterogeneous semi-infinite domain’s response to prescribed dynamic excitations. We use a full waveform approach to drive the inversion process, within a PDE-constrained optimization framework. Due to the semi-infinite extent of the targeted domains, we introduce perfectly-matched-layers (PMLs) to arrive at finite computational domains. The numerical implementation is based on a mixed finite-element method that is used to resolve the ensuing state and adjoint boundary-value problems, both of which are PML-endowed. To alleviate the inherent solution multiplicity, we use Tikhonov and total variation (TV) regularization schemes, in conjunction with a regularization factor continuation scheme. To further improve the optimizer’s chances to converge, we also discuss a source-frequency continuation scheme. We report on two-dimensional numerical experiments using synthetic data. Included are layered profiles, and profiles involving inclined layers and inclusions. We also report on our methodology’s reconstruction of the highly-heterogeneous Marmousi benchmark velocity model.

A New Tensorial Framework for Single-Shell High Angular Resolution Diffusion Imaging

Single-shell high angular resolution diffusion imaging data (HARDI) may be decomposed into a sum of eigenpolynomials of the Laplace-Beltrami operator on the unit sphere. The resulting representation combines the strengths hitherto offered by higher order tensor decomposition in a tensorial framework and spherical harmonic expansion in an analytical framework, but removes some of the conceptual weaknesses of either. In particular it admits analytically closed form expressions for Tikhonov regularization schemes and estimation of an orientation distribution function via the Funk-Radon Transform in tensorial form, which previously required recourse to spherical harmonic decomposition. As such it provides a natural point of departure for a Riemann-Finsler extension of the geometric approach towards tractography and connectivity analysis as has been stipulated in the context of diffusion tensor imaging (DTI), while at the same time retaining the natural coarse-to-fine hierarchy intrinsic to spherical harmonic decomposition.

Classification with Gaussians and convex loss II: improving error bounds by noise conditions

We continue our study on classification learning algorithms generated by Tikhonov regularization schemes associated with Gaussian kernels and general convex loss functions. Our main purpose of this paper is to improve error bounds by presenting a new comparison theorem associated with general convex loss functions and Tsybakov noise conditions. Some concrete examples are provided to illustrate the improved learning rates which demonstrate the effect of various loss functions for learning algorithms. In our analysis, the convexity of the loss functions plays a central role.

Classification with Gaussians and Convex Loss

This paper considers binary classification algorithms generated from Tikhonov regularization schemes associated with general convex loss functions and varying Gaussian kernels. Our main goal is to ...

Learning from non-identical sampling for classification

We consider the classification problem by learning from samples drawn from a non-identical sequence of probability measures. The learning algorithm is from Tikhonov regularization schemes associated with convex loss functions and reproducing kernel Hilbert spaces. Our main goal is to provide satisfactory estimates for the excess misclassification error of the produced classifiers.

Learning rates for regularized classifiers using multivariate polynomial kernels

Regularized classifiers (a leading example is support vector machine) are known to be a kind of kernel-based classification methods generated from Tikhonov regularization schemes, and the polynomial kernels are the original and also probably the most important kernels used in them. In this paper, we provide an error analysis for the regularized classifiers using multivariate polynomial kernels. We introduce Bernstein–Durrmeyer polynomials, whose reproducing kernel Hilbert space norms and approximation properties in L 1 space play a key role in the analysis of regularization error. We also introduce the standard estimation of sample error, and derive explicit learning rates for these algorithms.

Learning Theory Estimates via Integral Operators and Their Approximations

The regression problem in learning theory is investigated with least square Tikhonov regularization schemes in reproducing kernel Hilbert spaces (RKHS). We follow our previous work and apply the sampling operator to the error analysis in both the RKHS norm and the L2 norm. The tool for estimating the sample error is a Bennet inequality for random variables with values in Hilbert spaces. By taking the Hilbert space to be the one consisting of Hilbert-Schmidt operators in the RKHS, we improve the error bounds in the L2 metric, motivated by an idea of Caponnetto and de Vito. The error bounds we derive in the RKHS norm, together with a Tsybakov function we discuss here, yield interesting applications to the error analysis of the (binary) classification problem, since the RKHS metric controls the one for the uniform convergence.

Fully online classification by regularization

In this paper we consider fully online learning algorithms for classification generated from Tikhonov regularization schemes associated with general convex loss functions and reproducing kernel Hilbert spaces. For such a fully online algorithm, the regularization parameter in each learning step changes. This is the essential difference from the partially online algorithm which uses a fixed regularization parameter. We first present a novel approach to the drift error incurred by the change of the regularization parameter. Then we estimate the error of the learning process for the strong approximation in the reproducing kernel Hilbert space. Finally, learning rates are derived from decays of the regularization error. The convexity of the loss function plays an important role in our analysis. Concrete learning rates are given for the hinge loss and the support vector machine q-norm loss.

Multi-kernel regularized classifiers

A family of classification algorithms generated from Tikhonov regularization schemes are considered. They involve multi-kernel spaces and general convex loss functions. Our main purpose is to provide satisfactory estimates for the excess misclassification error of these multi-kernel regularized classifiers when the loss functions achieve the zero value. The error analysis consists of two parts: regularization error and sample error. Allowing multi-kernels in the algorithm improves the regularization error and approximation error, which is one advantage of the multi-kernel setting. For a general loss function, we show how to bound the regularization error by the approximation in some weighted L q spaces. For the sample error, we use a projection operator. The projection in connection with the decay of the regularization error enables us to improve convergence rates in the literature even for the one-kernel schemes and special loss functions: least-square loss and hinge loss for support vector machine soft margin classifiers. Existence of the optimization problem for the regularization scheme associated with multi-kernels is verified when the kernel functions are continuous with respect to the index set. Concrete examples, including Gaussian kernels with flexible variances and probability distributions with some noise conditions, are used to illustrate the general theory.

Maximum entropy regularization of Fredholm integral equations of the first kind

The problem of solving numerically a Fredholm integral equation of the first kind when it is known that f(x)>or=0, by a regularization method based on minimization is considered. The convergence of solutions of this minimization problem, given conditions on the data and the regularization parameter lambda , is demonstrated to show that the procedure leads to a correct regularization method. A generalized cross validation strategy for the selection of the regularization parameter is introduced. A number of numerical experiments are made and comparisons are made with Tikhonov regularization schemes based on differential operators.