Tikhonov Regularization Framework Research Articles

Current density impedance imaging with PINNs

In this paper, we introduce CDII-PINNs, a computationally efficient method for solving CDII using PINNs in the framework of Tikhonov regularization. This method constructs a physics-informed loss function by merging the regularized least-squares output functional with an underlying differential equation, which describes the relationship between the conductivity and voltage. A pair of neural networks representing the conductivity and voltage, respectively, are coupled by this loss function. Then, minimizing the loss function provides a reconstruction. A rigorous theoretical guarantee is provided. We give an error analysis for CDII-PINNs and establish a convergence rate, based on prior selected neural network parameters in terms of the number of samples. The numerical simulations demonstrate that CDII-PINNs are efficient, accurate and robust to noise levels ranging from 1% to 20%.

Enhanced PMU-Based Line Parameters Estimation and Compensation of Systematic Measurement Errors in Power Grids Considering Multiple Operating Conditions

Several monitoring, protection, and control applications designed for modern power grids are based on detailed grid models and thus require accurate knowledge of line parameters. A synchronized monitoring infrastructure based on phasor measurement units (PMUs) may be of great support to the task of line parameters estimation, but the accuracy of the estimated parameters may be largely affected by the uncertainty of all the elements of the PMU-based measurement chain. Thus, an accurate parameters estimation must appropriately consider the metrological behavior of all these elements, in particular that of instrument transformers (ITs). To address this challenge, this article proposes an enhanced multibranch method for accurate estimation of the line parameters and the systematic measurement errors introduced by the ITs when measurements for multiple operating conditions are considered. Indeed, multiple operating conditions are dealt with properly due to an in-depth analysis of the problem modeling within the framework of Tikhonov regularization. The validity of the proposed approach is confirmed by the results obtained on the IEEE 14-bus test system.

Inverse Estimation of Spatiotemporal Flux Boundary Conditions in Unsaturated Water Flow Modeling

AbstractAccurate estimation of boundary conditions is essential to the success of numerical schemes for modeling soil water flow processes. The flux‐type boundary condition, which is integration of many boundary fluxes and subsurface conditions, is traditionally estimated from hydrologic and meteorological information. However, such information could be rapidly variable in space and time and difficult to measure and quantify accurately. This study inversely estimates upper flux boundary conditions by calibrating interior observations of soil moisture. Unknown flux boundaries with both spatiotemporal and temporal only variabilities are studied. In the case of spatiotemporal variability, we formulate the inverse model by extending a Tikhonov regularization framework proposed in our previous study, which calibrated a macroscopic root water uptake model in one‐dimensional soil water flow. The developed method is efficient because it takes advantage of the strategy in the previous study that the inversion is transformed to a variational problem, and also because it incorporates locally one‐dimensional operator splitting technique to overcome the computation complexity when dealing with multidimensional flow processes. When flux boundaries vary with time only (special case), we have found that the inversion degenerates into a well‐posed problem and formulated the inversion explicitly without use of regularization. Numerical tests are conducted through both one‐dimensional (well‐posed) and two‐dimensional (ill‐posed) heterogeneous soil profiles to illustrate the effectiveness of the proposed inverse formulation. Particularly, the applicability of the inverse formulation in the special case is further validated in field experiments using in situ measurements in arid and semiarid northern China.

Insight into Construction of Tikhonov-Type Regularization for Atmospheric Retrievals

In atmospheric science we are confronted with inverse problems arising in applications associated with retrievals of geophysical parameters. A nonlinear mapping from geophysical quantities (e.g., atmospheric properties) to spectral measurements can be represented by a forward model. An inversion often suffers from the lack of stability and its stabilization introduced by proper approaches, however, can be treated with sufficient generality. In principle, regularization can enforce uniqueness of the solution when additional information is incorporated into the inversion process. In this paper, we analyze different forms of the regularization matrix L in the framework of Tikhonov regularization: the identity matrix L0, discrete approximations of the first and second order derivative operators L1 and L2, respectively, and the Cholesky factor of the a priori profile covariance matrix LC. Each form of L has its intrinsic pro/cons and thus may lead to different performance of inverse algorithms. An extensive comparison of different matrices is conducted with two applications using synthetic data from airborne and satellite sensors: retrieving atmospheric temperature profiles from microwave spectral measurements, and deriving aerosol properties from near infrared spectral measurements. The regularized solution obtained with L0 possesses a reasonable magnitude, but its smoothness is not always assured. The retrieval using L1 and L2 produces a solution in favor of the smoothness, and the impact of the a priori knowledge is less critical on the retrieval using L1. The retrieval performance of LC is affected by the accuracy of the a priori knowledge.

Neural networks-based regularization for large-scale medical image reconstruction

In this paper we present a generalized Deep Learning-based approach for solving ill-posed large-scale inverse problems occuring in medical image reconstruction. Recently, Deep Learning methods using iterative neural networks (NNs) and cascaded NNs have been reported to achieve state-of-the-art results with respect to various quantitative quality measures as PSNR, NRMSE and SSIM across different imaging modalities. However, the fact that these approaches employ the application of the forward and adjoint operators repeatedly in the network architecture requires the network to process the whole images or volumes at once, which for some applications is computationally infeasible. In this work, we follow a different reconstruction strategy by strictly separating the application of the NN, the regularization of the solution and the consistency with the measured data. The regularization is given in the form of an image prior obtained by the output of a previously trained NN which is used in a Tikhonov regularization framework. By doing so, more complex and sophisticated network architectures can be used for the removal of the artefacts or noise than it is usually the case in iterative NNs. Due to the large scale of the considered problems and the resulting computational complexity of the employed networks, the priors are obtained by processing the images or volumes as patches or slices. We evaluated the method for the cases of 3D cone-beam low dose CT and undersampled 2D radial cine MRI and compared it to a total variation-minimization-based reconstruction algorithm as well as to a method with regularization based on learned overcomplete dictionaries. The proposed method outperformed all the reported methods with respect to all chosen quantitative measures and further accelerates the regularization step in the reconstruction by several orders of magnitude.

Debiased magnitude-preserving ranking: Learning rate and bias characterization

Magnitude-preserving ranking (MPRank) under Tikhonov regularization framework has shown competitive performance on information retrial besides theoretical advantages for computation feasibility and statistical guarantees. In this paper, we further characterize the learning rate and asymptotic bias of MPRank, and then propose a new debiased ranking algorithm. In terms of the operator representation and approximation techniques, we establish their convergence rates and bias characterizations. These theoretical results demonstrate that the new model has smaller asymptotic bias than MPRank, and can achieve the satisfactory convergence rate under appropriate conditions. In addition, some empirical examples are provided to verify the effectiveness of our debiased strategy.

On the edge detection of an image by numerical differentiations for gray function

The detection of image edges is of great importance in image processing. One of the efficient implementations for this image recovery problem is based on the identification of sharp jumps of the gray function of the image. Mathematically, this problem can be modeled by the numerical differentiation of the gray function with 2 variables. For this ill‐posed problem with nonsmooth solution, we investigate the regularization schemes with total variation and L1 penalty term, respectively. We prove that the regularizing parameter under the Tikhonov regularization framework can be uniquely chosen in terms of the Morozov's discrepancy principle and then establish the convergence rate of the regularizing solutions in terms of the Bregman distance. The discrete schemes are performed by the lagged diffusivity fixed point iteration, with numerical implementations showing the validity of the proposed scheme.

A Closed Equation in Time Domain for Band-Limited Extensions of One-Sided Sequences

This paper suggests a method of optimal extension of one-sided semi-infinite sequences of a general type by traces of band-limited sequences in deterministic setting, i.e., without probabilistic assumptions. The method requires to solve a closed linear equation in the time domain connecting the past observations of the underlying process with the future values of the band-limited process. Robustness of the solution with respect to the input errors and data truncation is established in the framework of Tikhonov regularization.

Brain connectivity-informed regularization methods for regression.

One of the challenging problems in brain imaging research is a principled incorporation of information from different imaging modalities. Frequently, each modality is analyzed separately using, for instance, dimensionality reduction techniques, which result in a loss of mutual information. We propose a novel regularization-method to estimate the association between the brain structure features and a scalar outcome within the linear regression framework. Our regularization technique provides a principled approach to use external information from the structural brain connectivity and inform the estimation of the regression coefficients. Our proposal extends the classical Tikhonov regularization framework by defining a penalty term based on the structural connectivity-derived Laplacian matrix. Here, we address both theoretical and computational issues. The approach is first illustrated using simulated data and compared with other penalized regression methods. We then apply our regularization method to study the associations between the alcoholism phenotypes and brain cortical thickness using a diffusion imaging derived measure of structural connectivity. Using the proposed methodology in 148 young male subjects with a risk for alcoholism, we found a negative associations between cortical thickness and drinks per drinking day in bilateral caudal anterior cingulate cortex, left lateral OFC and left precentral gyrus.

Baseline and interferent correction by the Tikhonov regularization framework for linear least squares modeling

AbstractSpectroscopic data are usually perturbed by noise from various sources that should be removed prior to model calibration. After conducting a preprocessing step to eliminate unwanted multiplicative effects (effects that scale the pure signal in a multiplicative manner), we discuss how to correct a model for unwanted additive effects in the spectra. Our approach is described within the Tikhonov regularization (TR) framework for linear regression model building, and our focus is on ignoring the influence of noninformative polynomial trends. This is obtained by including an additional criterion in the TR problem penalizing the resulting regression coefficients away from a selected set of possibly disturbing directions in the sample space. The presented method builds on the extended multiplicative signal correction, and we compare the two approaches on several real data sets showing that the suggested TR‐based method may improve the predictive power of the resulting model. We discuss the possibilities of imposing smoothness in the calculation of regression coefficients as well as imposing selection of wavelength regions within the TR framework. To implement TR efficiently in the model building, we use an algorithm that is heavily based on the singular value decomposition. Because of some favorable properties of the singular value decomposition, it is possible to explore the models (including their generalized cross‐validation error estimates) associated with a large number of regularization parameter values at low computational cost.

Deep neural network-based bandwidth enhancement of photoacoustic data.

Photoacoustic (PA) signals collected at the boundary of tissue are always band-limited. A deep neural network was proposed to enhance the bandwidth (BW) of the detected PA signal, thereby improving the quantitative accuracy of the reconstructed PA images. A least square-based deconvolution method that utilizes the Tikhonov regularization framework was used for comparison with the proposed network. The proposed method was evaluated using both numerical and experimental data. The results indicate that the proposed method was capable of enhancing the BW of the detected PA signal, which inturn improves the contrast recovery and quality of reconstructed PA images without adding any significant computational burden.

Calculation of the time resolution of the J-PET tomograph using kernel density estimation

In this paper we estimate the time resolution of the J-PET scanner built from plastic scintillators. We incorporate the method of signal processing using the Tikhonov regularization framework and the kernel density estimation method. We obtain simple, closed-form analytical formulae for time resolution. The proposed method is validated using signals registered by means of the single detection unit of the J-PET tomograph built from a 30 cm long plastic scintillator strip. It is shown that the experimental and theoretical results obtained for the J-PET scanner equipped with vacuum tube photomultipliers are consistent.

Assessment of Tikhonov-type regularization methods for solving atmospheric inverse problems

Inverse problems occurring in atmospheric science aim to estimate state parameters (e.g. temperature or constituent concentration) from observations. To cope with nonlinear ill-posed problems, both direct and iterative Tikhonov-type regularization methods can be used. The major challenge in the framework of direct Tikhonov regularization (TR) concerns the choice of the regularization parameter λ, while iterative regularization methods require an appropriate stopping rule and a flexible λ-sequence.In the framework of TR, a suitable value of the regularization parameter can be generally determined based on a priori, a posteriori, and error-free selection rules. In this study, five practical regularization parameter selection methods, i.e. the expected error estimation (EEE), the discrepancy principle (DP), the generalized cross-validation (GCV), the maximum likelihood estimation (MLE), and the L-curve (LC), have been assessed. As a representative of iterative methods, the iteratively regularized Gauss–Newton (IRGN) algorithm has been compared with TR. This algorithm uses a monotonically decreasing λ-sequence and DP as an a posteriori stopping criterion.Practical implementations pertaining to retrievals of vertically distributed temperature and trace gas profiles from synthetic microwave emission measurements and from real far infrared data, respectively, have been conducted. Our numerical analysis demonstrates that none of the parameter selection methods dedicated to TR appear to be perfect and each has its own advantages and disadvantages. Alternatively, IRGN is capable of producing plausible retrieval results, allowing a more efficient manner for estimating λ.

A stochastic cloud model for cloud and ozone retrievals from UV measurements

The new generation of satellite instruments provides measurements in and around the Oxygen A-band on a global basis and with a relatively high spatial resolution. These data are commonly used for the determination of cloud properties. A stochastic model and radiative transfer model, previously developed by the authors, is used as the forward model component in retrievals of cloud parameters and ozone total and partial columns. The cloud retrieval algorithm combines local and global optimization routines, and yields a retrieval accuracy of about 1% and a fast computational time. Retrieved parameters are the cloud optical thickness and the cloud-top height. It was found that the use of the independent pixel approximation instead of the stochastic cloud model leads to large errors in the retrieved cloud parameters, as well as, in the retrieved ozone height resolved partial columns. The latter can be reduced by using the stochastic cloud model to compute the optimal value of the regularization parameter in the framework of Tikhonov regularization.

A coupled complex boundary method for the Cauchy problem

Considered in this paper is a Cauchy problem governed by an elliptic partial differential equation. In the Cauchy problem, one wants to recover the unknown Neumann and Dirichlet data on a part of the boundary from the measured Neumann and Dirichlet data, usually contaminated with noise, on the remaining part of the boundary. The Cauchy problem is an inverse problem with severe ill-posedness. In this paper, a coupled complex boundary method (CCBM), originally proposed in [Cheng XL, Gong RF, Han W, et al. A novel coupled complex boundary method for solving inverse source problems. Inverse Prob. 2014;30:055002], is applied to solve the Cauchy problem stably. With the CCBM, all the data, including the known and unknown ones on the boundary are used in a complex Robin boundary on the whole boundary. As a result, the Cauchy problem is transferred into a complex Robin boundary problem of finding the unknown data such that the imaginary part of the solution equals zero in the domain. Then the Tikhonov regularization is applied to the resulting new formulation. Some theoretical analysis is performed on the CCBM-based Tikhonov regularization framework. Moreover, through the adjoint technique, a simple solver is proposed to compute the regularized solution. The finite-element method is used for the discretization. Numerical results are given to show the feasibility and effectiveness of the proposed method.

OCReP: An Optimally Conditioned Regularization for pseudoinversion based neural training

In this paper we consider the training of single hidden layer neural networks by pseudoinversion, which, in spite of its popularity, is sometimes affected by numerical instability issues. Regularization is known to be effective in such cases, so that we introduce, in the framework of Tikhonov regularization, a matricial reformulation of the problem which allows us to use the condition number as a diagnostic tool for identification of instability. By imposing well-conditioning requirements on the relevant matrices, our theoretical analysis allows the identification of an optimal value for the regularization parameter from the standpoint of stability. We compare with the value derived by cross-validation for overfitting control and optimization of the generalization performance. We test our method for both regression and classification tasks. The proposed method is quite effective in terms of predictivity, often with some improvement on performance with respect to the reference cases considered. This approach, due to analytical determination of the regularization parameter, dramatically reduces the computational load required by many other techniques.

Compressive sensing of signals generated in plastic scintillators in a novel J-PET instrument

The J-PET scanner, which allows for single bed imaging of the whole human body, is currently under development at the Jagiellonian University. The discussed detector offers improvement of the Time of Flight (TOF) resolution due to the use of fast plastic scintillators and dedicated electronics allowing for sampling in the voltage domain of signals with durations of few nanoseconds. In this paper we show that recovery of the whole signal, based on only a few samples, is possible. In order to do that, we incorporate the training signals into the Tikhonov regularization framework and we perform the Principal Component Analysis decomposition, which is well known for its compaction properties. The method yields a simple closed form analytical solution that does not require iterative processing. Moreover, from the Bayes theory the properties of regularized solution, especially its covariance matrix, may be easily derived. This is the key to introduce and prove the formula for calculations of the signal recovery error. In this paper we show that an average recovery error is approximately inversely proportional to the number of acquired samples.

A fast solver for an inverse problem arising in bioluminescence tomography

Bioluminescence tomography (BLT) is a new method in biomedical imaging, with a promising potential in monitoring non-invasively physiological and pathological processes in vivo at the cellular and molecular levels. The goal of BLT is to quantitatively reconstruct a three dimensional bioluminescent source distribution within a small animal from two dimensional optical signals on the surface of the animal body. Mathematically, BLT is an under-determined inverse source problem and is severely ill-posed, making its numerical treatments very challenging. In this paper, we provide a new Tikhonov regularization framework for the BLT problem. Compared with the existing reconstruction methods about BLT, our new method uses an energy functional defined over the whole problem domain for measuring the data fitting, associated with two related but different boundary value problems. Based on the new formulation, a fast solver is introduced by transforming the proposed optimization model into a system of partial differential equations. Moreover, a finite element method is used to obtain a regularized discrete solution. Finally, numerical results show that the fast solver for BLT is feasible and effective.

The regularized implied local volatility equations -A new model to recover the volatility of underlying asset from observed market option price

In this paper, we propose a new continuous time model to recover the volatility of underlying asset from observed market European option price. The model is a couple of fully nonlinear parabolic partial differential equations (see (34), (36)). As an inverse problem, the model is deduced from a Tikhonov regularization framework. Based on our method, the recovering procedure is stable and accurate. It is justified not only in theoretical proofs, but also in the numerical experiments.

Overview of two‐norm (L2) and one‐norm (L1) Tikhonov regularization variants for full wavelength or sparse spectral multivariate calibration models or maintenance

Building a multivariate calibration model is typically accomplished using partial least squares, principal component regression, or ridge regression, also derived as the standard form of Tikhonov regularization (TR). These approaches can be used in a full variable mode (full wavelengths for spectroscopic data) or with wavelength selection (bands and/or individual for sparse models). Calibration maintenance is an important aspect of multivariate calibration and describes the situation of maintaining acceptable predictions from a model over time. In terms of TR, this amounts to updating an existing model (determined under primary conditions) to handle new secondary conditions such as a new instrument, sample matrix, or environmental conditions. This paper overviews TR in its ability to form a primary calibration model or update a primary model to new secondary conditions while using full wavelengths or simultaneously selecting wavelengths to form respective sparse models. These objectives can be accomplished in two‐norm (L2) or one‐norm (L1) or combined formats. Also included is a TR design that minimizes the effect of the standardization set composition, i.e., reduces the effect of an outlier. Ongoing work that removes all reference samples from the TR framework is also described. Copyright © 2012 John Wiley & Sons, Ltd.