Picard Condition Research Articles

Regularization of least squares problems in CHARMM parameter optimization by truncated singular value decompositions.

We examine the use of the truncated singular value decomposition and Tikhonov regularization in standard form to address ill-posed least squares problems Ax = b that frequently arise in molecular mechanics force field parameter optimization. We illustrate these approaches by applying them to dihedral parameter optimization of genotoxic polycyclic aromatic hydrocarbon-DNA adducts that are of interest in the study of chemical carcinogenesis. Utilizing the discrete Picard condition and/or a well-defined gap in the singular value spectrum when A has a well-determined numerical rank, we are able to systematically determine truncation and in turn regularization parameters that are correspondingly used to produce truncated and regularized solutions to the ill-posed least squares problem at hand. These solutions in turn result in optimized force field dihedral terms that accurately parameterize the torsional energy landscape. As the solutions produced by this approach are unique, it has the advantage of avoiding the multiple iterations and guess and check work often required to optimize molecular mechanics force field parameters.

A New Hybrid Inversion Method for 2D Nuclear Magnetic Resonance Combining TSVD and Tikhonov Regularization.

This paper is concerned with the reconstruction of relaxation time distributions in Nuclear Magnetic Resonance (NMR) relaxometry. This is a large-scale and ill-posed inverse problem with many potential applications in biology, medicine, chemistry, and other disciplines. However, the large amount of data and the consequently long inversion times, together with the high sensitivity of the solution to the value of the regularization parameter, still represent a major issue in the applicability of the NMR relaxometry. We present a method for two-dimensional data inversion (2DNMR) which combines Truncated Singular Value Decomposition and Tikhonov regularization in order to accelerate the inversion time and to reduce the sensitivity to the value of the regularization parameter. The Discrete Picard condition is used to jointly select the SVD truncation and Tikhonov regularization parameters. We evaluate the performance of the proposed method on both simulated and real NMR measurements.

Unbiased predictive risk estimation of the Tikhonov regularization parameter: convergence with increasing rank approximations of the singular value decomposition

The truncated singular value decomposition may be used to find the solution of linear discrete ill-posed problems in conjunction with Tikhonov regularization and requires the estimation of a regularization parameter that balances between the sizes of the fit to data function and the regularization term. The unbiased predictive risk estimator is one suggested method for finding the regularization parameter when the noise in the measurements is normally distributed with known variance. In this paper we provide an algorithm using the unbiased predictive risk estimator that automatically finds both the regularization parameter and the number of terms to use from the singular value decomposition. Underlying the algorithm is a new result that proves that the regularization parameter converges with the number of terms from the singular value decomposition. For the analysis it is sufficient to assume that the discrete Picard condition is satisfied for exact data and that noise completely contaminates the measured data coefficients for a sufficiently large number of terms, dependent on both the noise level and the degree of ill-posedness of the system. A lower bound for the regularization parameter is provided leading to a computationally efficient algorithm. Supporting results are compared with those obtained using the method of generalized cross validation. Simulations for two-dimensional examples verify the theoretical analysis and the effectiveness of the algorithm for increasing noise levels, and demonstrate that the relative reconstruction errors obtained using the truncated singular value decomposition are less than those obtained using the singular value decomposition. This is a pre-print of an article published in BIT Numerical Mathematics. The final authenticated version is available online at: https://doi.org/10.1007%2Fs10543-019-00762-7.

Considering New Regularization Parameter-Choice Techniques for the Tikhonov Method to Improve the Accuracy of Electrocardiographic Imaging.

The electrocardiographic imaging (ECGI) inverse problem highly relies on adding constraints, a process called regularization, as the problem is ill-posed. When there are no prior information provided about the unknown epicardial potentials, the Tikhonov regularization method seems to be the most commonly used technique. In the Tikhonov approach the weight of the constraints is determined by the regularization parameter. However, the regularization parameter is problem and data dependent, meaning that different numerical models or different clinical data may require different regularization parameters. Then, we need to have as many regularization parameter-choice methods as techniques to validate them. In this work, we addressed this issue by showing that the Discrete Picard Condition (DPC) can guide a good regularization parameter choice for the two-norm Tikhonov method. We also studied the feasibility of two techniques: The U-curve method (not yet used in the cardiac field) and a novel automatic method, called ADPC due its basis on the DPC. Both techniques were tested with simulated and experimental data when using the method of fundamental solutions as a numerical model. Their efficacy was compared with the efficacy of two widely used techniques in the literature, the L-curve and the CRESO methods. These solutions showed the feasibility of the new techniques in the cardiac setting, an improvement of the morphology of the reconstructed epicardial potentials, and in most of the cases of their amplitude.

A stopping criterion for iterative regularization methods

We present a discrepancy-like stopping criterium for iterative regularization methods for the solution of linear discrete ill-posed problems. The presented criterium terminates the iterations of the iterative method when the residual norm of the computed solution becomes less or equal to the residual norm of a regularized Truncated Singular Value Decomposition (TSVD) solution. We present two algorithms for the automatic computation of the TSVD residual norm using the Discrete Picard Condition. The first algorithm uses the SVD coefficients while the second one uses the Fourier coefficients. In this work, we mainly focus on the Conjugate Gradient Least Squares method, but the proposed criterium can be used for terminating the iterations of any iterative regularization method. Many numerical tests on some selected one dimensional and image deblurring problems are presented and the results are compared with those obtained by state-of-the-art parameter selection rules. The numerical results show the efficiency and robustness of the proposed criterium.

Inheritance of the discrete Picard condition in Krylov subspace methods

When projection methods are employed to regularize linear discrete ill-posed problems, one implicitly assumes that the discrete Picard condition (DPC) is somehow inherited by the projected problems. In this paper we show that, under some assumptions, the DPC holds for the projected uncorrupted systems computed by various Krylov subspace methods. By exploiting the inheritance of the DPC, some estimates on the behavior of the projected problems are also derived. Numerical examples are provided in order to illustrate the accuracy of the derived estimates.

Near-Optimal Spectral Filtering and Error Estimation for Solving Ill-Posed Problems

We consider regularization methods for numerical solution of linear ill-posed problems, in particular image deblurring, when the singular value decomposition (SVD) of the operator is available. We assume that the noise-free problem satisfies the discrete Picard condition and define the Picard parameter, the index beyond which the data, expressed in the coordinate system of the SVD, are dominated by noise. We propose estimating the Picard parameter graphically or using standard statistical tests. Having this parameter available allows us to estimate the mean and standard deviation of the noise and drop noisy components, thus making filtered solutions much more reliable. We show how to compute a near-optimal choice of filter parameters for any filter. This includes the truncated SVD (TSVD) filter, the truncated singular component method (TSCM) filter, and several new filters which we define, including a truncated Tikhonov filter, a Tikhonov-TSVD filter, a Heaviside filter, and a spline filter. We show how to estimate the error in any spectral filter, regardless of how the filter parameters are chosen. We demonstrate the usefulness of our new filters, our near-optimal choice of parameters, and our error estimates for restoring blurred images.

A GCV based Arnoldi-Tikhonov regularization method

For the solution of linear discrete ill-posed problems, in this paper we consider the Arnoldi-Tikhonov method coupled with the Generalized Cross Validation for the computation of the regularization parameter at each iteration. We study the convergence behavior of the Arnoldi method and its properties for the approximation of the (generalized) singular values, under the hypothesis that Picard condition is satisfied. Numerical experiments on classical test problems and on image restoration are presented.

A two-step discrete method for reconstruction of temperature distribution in a three-dimensional participating medium

A new two-step discrete method is proposed in this paper for reconstruction of three-dimensional temperature distribution in an absorbing, emitting and isotropically scattering medium. With this new method, the temperature of the wall is also considered. The local radiative source term is reconstructed in the first step through the discrete transfer method from the directional, exit radiation intensities measured by CCD cameras. Then, the temperature of each discrete element is calculated in the second step by subtracting the scattering contribution from the retrieved radiative source term through the discrete ordinate method. The least squares minimum residual algorithm is employed to solve the ill-posed reconstruction equations and the calculation is improved to reduce the computational cost. The performance of the proposed method is examined by numerical test problems with unimodal and bimodal temperature distributions. The ill-conditioning of the reconstruction problem is checked by the Picard condition. The effects of the measurement noise and the radiative properties on the reconstruction accuracy are discussed. The results show that the method proposed in this paper is capable of reconstructing the temperature distribution accurately in large, confined, participating media, even with noisy input data. The computation time reduction of this new method is significant when compared with other methods.

L1 regularization method in electrical impedance tomography by using the L1-curve (Pareto frontier curve)

Electrical impedance tomography (EIT), as an inverse problem, aims to calculate the internal conductivity distribution at the interior of an object from current–voltage measurements on its boundary. Many inverse problems are ill-posed, since the measurement data are limited and imperfect. To overcome ill-posedness in EIT, two main types of regularization techniques are widely used. One is categorized as the projection methods, such as truncated singular value decomposition (SVD or TSVD). The other categorized as penalty methods, such as Tikhonov regularization, and total variation methods. For both of these methods, a good regularization parameter should yield a fair balance between the perturbation error and regularized solution. In this paper a new method combining the least absolute shrinkage and selection operator (LASSO) and the basis pursuit denoising (BPDN) is introduced for EIT. For choosing the optimum regularization we use the L1-curve (Pareto frontier curve) which is similar to the L-curve used in optimising L2-norm problems. In the L1-curve we use the L1-norm of the solution instead of the L2 norm. The results are compared with the TSVD regularization method where the best regularization parameters are selected by observing the Picard condition and minimizing generalized cross validation (GCV) function. We show that this method yields a good regularization parameter corresponding to a regularized solution. Also, in situations where little is known about the noise level σ, it is also useful to visualize the L1-curve in order to understand the trade-offs between the norms of the residual and the solution. This method gives us a means to control the sparsity and filtering of the ill-posed EIT problem. Tracing this curve for the optimum solution can decrease the number of iterations by three times in comparison with using LASSO or BPDN separately.

Feasibility of U-curve method to select the regularization parameter for fluorescence diffuse optical tomography in phantom and small animal studies

When dealing with ill-posed problems such as fluorescence diffuse optical tomography (fDOT) the choice of the regularization parameter is extremely important for computing a reliable reconstruction. Several automatic methods for the selection of the regularization parameter have been introduced over the years and their performance depends on the particular inverse problem. Herein a U-curve-based algorithm for the selection of regularization parameter has been applied for the first time to fDOT. To increase the computational efficiency for large systems an interval of the regularization parameter is desirable. The U-curve provided a suitable selection of the regularization parameter in terms of Picard's condition, image resolution and image noise. Results are shown both on phantom and mouse data.

The method of (not so) ordinary least squares: what can go wrong and how to fix them

Most of us came to know about the method of least squares while trying to fit a curve through a set of data points. The parameters of the curve are obtained by solving a set of equations (called the normal equations). Although widely used, this approach is not foolproof and, in some cases, it can even give results that are plain wrong! This happens due to some subtleties that are often overlooked by the user. In this paper, we demonstrate, by means of a simple numerical example, what can go wrong and how to fix them. Using only elementary matrix algebra, we introduce (and show the importance of) singular value decomposition, discrete Picard condition, Tikhonov regularisation, the L-curve and the L-curve criterion in addressing the subtle points of this method so that stable and reliable results are obtained in the end.

Time domain response and the data analysis of ferroelectric discharge current

The time domain method is more reliable for the study of nonlinear dielectric response compared with frequency domain analysis. A Tikhonov regularization method, which is widely adopted for ill-posed problem, is described for derivation of the relaxation time distribution function, g(τ), from the ferroelectric discharge current in time domain. The new method allows study of the structure variation and the relaxation behavior of ferroelectrics at different temperatures. For barium stannate titanate ceramics (BTS20), g(τ) has been successfully derived; the relaxation peaks move to shorter times with increasing temperature in the range 20–60°C, which may indicate a space charge thermal activation process. However, g(τ) could not be derived from the discharge current by the regularization method for BTS20 at temperatures above 60°C or for lanthanum-doped lead zirconium titanate transparent ceramic (PLZT), since the data do not satisfy the discrete Picard condition, which is a valid criterion for regularization method.

The sensitivity of reconstructed images and process engineering metrics to key choices in practical electrical impedance tomography

This paper explores, from an experimental perspective, the dramatic effect that measurement strategy, reconstruction algorithm and reconstruction parameters can have on electrical impedance tomography images. Measurement data, from a stirred tank and jet mixer, have been acquired using two tomographs from the University of Cape Town and Industrial Tomography Systems Ltd respectively. Simulations consider conductively contrasting objects that are placed strategically in the vessel. The adjacent and opposite measurement strategies are employed to interrogate an unbaffled mixing tank. The superior sensitivity of the opposite strategy in the centre of the vessel is verified. For a central inclusion, simulated results suggest 4% ‘image error’ compared to 6% for the adjacent strategy. Five reconstruction algorithms: Linear Back Projection, Landweber, Conjugate Gradients, Generalized Singular Value Decomposition (GSVD) and Nonlinear Gauss Newton, have been considered. A measure of ‘image error’ is typically below 10%, but values as high as 30% are not unusual with algorithms such as Linear Back Projection. For a homogeneous step change in conductivity ‘image error’ is seen to vary from 3% for Nonlinear Gauss Newton to 150% for Linear Back Projection. Corresponding measures of the coefficient of variation range from 25% to 44%. Overall it is suggested that the GSVD algorithm provides the best balance of attributes for identifying discrete objects and for homogeneous step changes. The dramatic effect of regularization parameters is illustrated by considering the GSVD. The use of the discrete Picard condition to determine optimum values is demonstrated. Mixing lengths have been calculated from the reconstructed image data and this is seen to vary dramatically with the regularization parameter.

Study on three-dimensional flame temperature distribution reconstruction based on truncated singular value decomposition

Three-dimensional flame temperature distribution reconstruction in furnace according to the flame radiative energy images captured by CCD cameras was studied. Temperature reconstruction matrix equation is an ill-posed linear system of equations and the reconstruction problem is an ill-posed problem. Truncated singular value decomposition method (TSVD) was introduced to deal with the ill-posed matrix equation and L-curve method was adopted to choose regularization parameter. Singular value decomposition (SVD) and discrete Picard condition were used to analyze the ill-posed characteristics of the reconstruction problem carefully. The results show that reasonable solutions can be obtained by TSVD and the reconstructed temperature distribution can reproduce the feature of the assumed temperature distribution.

Evaluation of Tonal Aeroacoustic Sources in Subsonic Fans Using Inverse Models

This paper aims at quantifying the most acoustically radiating modes of the blade unsteady lift and inflow velocity in the circumferential spectral domain and at localizing hot spot interaction areas over the fan. The proposed method is based on the inversion of the Blake model for tonal noise from subsonic fans. The unsteady lift formulation is first used to reconstruct the circumferential blade loading variations from the tonal noise radiation in free field. Then the unsteady lift is related to the inflow velocity distortions by a compressible blade response function. Discretizing the lift and velocity in the direct model leads to ill-conditioned aeroacoustic transfer matrices. The Tikhonov regularization technique is used to stabilize the inversion. The curvature of the L-curve is used to choose the regularization parameter such that the sources' strength vectors are optimally reconstructed. The singular value decomposition and the discrete Picard condition are also used to analyze the stability of the sources' reconstructions. One experimental case is considered to demonstrate the capability of the inverse model to qualitatively reconstruct the blade loading and inflow velocity variations from acoustic pressure measurements in the case of an automotive engine cooling fan.

Pore Size Distribution Analysis of Selected Hexagonal Mesoporous Silicas by Grand Canonical Monte Carlo Simulations

We combine here a regularization procedure with individual adsorption isotherms obtained from grand canonical Monte Carlo simulations in order to obtain reliable pore size distributions. The methodology is applied to two hexagonal high-ordered silica materials: SBA-15 and PHTS, synthesized in our laboratory. Feasible pore size distributions are calculated through an adaptable procedure of deconvolution over the adsorption integral equation, with two necessary inputs: the experimental adsorption data and individual adsorption isotherms, assuming the validity of the independent pore model. The application of the deconvolution procedure implies an adequate grid size evaluation (i.e., numbers of pores and relative pressures to be considered for the inversion, or kernel size), the fulfillment of the discret Picard condition, and the appropriate choice of the regularization parameter (L-curve criteria). Assuming cylindrical geometry for both porous materials, the same set of individual adsorption isotherms generated from molecular simulations can be used to construct the kernel to obtain the PSD of SBA-15 and PHTS. The PSD robustness is measured imposing random errors over the experimental data. Excellent agreement is found between the calculated and the experimental global adsorption isotherms for both materials. Molecular simulations provide new insights into the studied systems, pointing out the need of high-resolution isotherms to describe the presence of complementary microporosity in these materials.

Search for a reliable methodology for PSD determination based on a combined molecular simulation–regularization–experimental approach: The case of PHTS materials

We present here a methodology for searching a robust pore size distribution (PSD) for adsorbent materials. The method is based on a combination of individual adsorption isotherms, obtained from Grand Canonical Monte Carlo simulations, a regularization procedure to invert the adsorption integral equation (Tikhonov regularization solved by singular value decomposition), and the needed experimental adsorption isotherm. The selection of several parameters from the available choices to start the procedure are discussed here: the size of the kernel (number of individual pores and number of experimental adsorption points to be included), the fulfillment of the Discrete Picard condition, and the L-curve criteria, all leading to find a reliable and robust PSD. The procedure is applied to plugged hexagonal templated silicas (PHTS), synthesized, and characterized in our laboratory.

Analysis of multiframe super-resolution reconstruction for image anti-aliasing and deblurring

Novel theoretical results for the super-resolution reconstruction (SRR) are presented under the case of arbitrary image warping. The SRR model is reasonably separated into two parts of anti-aliasing and deblurring. The anti-aliasing part is proved to be well-posed. The ill-posedness of the entire SRR process is shown to be mainly caused by the deblurring part. The motion estimation error results in a multiplicative perturbation to the warping matrix, and the perturbation bound is derived. The common regularization algorithms used in SRR are analyzed through the discrete Picard condition, which provides a theoretical measure for limits on SRR. Experiments and examples are supplied to validate the presented theories.

AN INVERSE AEROACOUSTIC PROBLEM ON ROTOR WAKE/STATOR INTERACTION

An inverse aeroacoustic model on rotor wake/stator interaction is proposed based on the linearized Euler equations. The sound field is related to the pressure distribution on the stator surface in the form of a Fredholm integral equation of the first kind which is a well-known ill-posed problem. Since the sound field is fully specified, numerical inversion allows the reconstruction of the pressure distribution on the stator surface. For solving the discrete ill-posed problem, the singular-value decomposition technique coupled with the Tikhonov regularization method is applied to stabilize the solution. The optimal regularization parameter is chosen by the generalized cross-validation criterion and the discrete Picard condition is employed to analyze the ill-posedness of the inverse problem. Numerical results show that the reconstruction is fairly good when the signal-to-noise ratio is not very low. The results become inaccurate when the noise dominates over the observer signal. In addition, numerical results also indicate the importance of the reduced frequency. The higher the reduced frequency, the better the reconstruction results.