Selection Of Regularization Parameter Research Articles

Microwave Imaging of 3-D Dielectric–Magnetic Penetrable Objects Based on Integral Equation Method

Some objects like mineral substances may be both dielectric and magnetic and their high-resolution inner imaging is very desirable when detecting and analyzing their ingredients. An integral equation method is developed for reconstructing or imaging such objects by using microwave illumination. Since the objects are both dielectric and magnetic, full volume integral equations (VIEs) are needed to describe the problem. The objects can be reconstructed by alternatively solving the forward scattering VIEs (FSVIE) and the inverse scattering VIEs (ISVIEs) under the Born iterative method (BIM) or distorted BIM (DBIM). The Nyström method is used to solve the FSVIEs while a Gauss-Newton minimization method (GNMM) with a multiplicative regularization scheme (MRS) is employed to solve the ISVIE. The Nyström method is more suitable for solving the FSVIEs of inverse problems because it is a point-matching method. To accelerate the solution, the multilevel fast multipole algorithm (MLFMA) is incorporated with the Nyström method which is new for solving inverse problems. Also, using the MRS-based GNMM to solve the ISVIEs can facilitate the selection of regularization parameter and reduce the requirements on measured data. Numerical examples are provided to demonstrate the approach and good results have been achieved.

Identify the Robin coefficient in an inhomogeneous time-fractional diffusion-wave equation

This paper is devoted to identifying the Robin coefficient in a one-dimensional inhomogeneous time-fractional diffusion-wave equation from the viewpoint of Bayesian inference approach. The priori modeling is implemented by a Markov Random Field (MRF). A hierarchical Bayesian model is adopted to select regularization parameters automatically. The Markov chain Monte Carlo (MCMC) algorithms are conducted to explore the posterior state space. Three numerical examples are carried out to demonstrate the performance of the proposed method.

A quasi-reversibility method for solving a two-dimensional time-fractional inverse heat conduction problem

In this paper, we consider a two-dimensional time-fractional inverse heat conduction problem, which is severely ill-posed. A quasi-reversibility method is proposed to solve this problem with disturbed boundary value. We give the selection of regularization parameters of quasi-reversibility method both under a priori and a posteriori rules, and give the proof of error estimates between the exact solution and its regularization approximation. Further more, numerical results are included to verify the effectiveness of the proposed method.

Prediction identification method of time-varying internal heat source based on inverse heat transfer problem

Purpose This paper aims to discuss the inverse problems that arise in various practical heat transfer processes. The purpose of this paper is to provide an identification method for predicting the internal boundary conditions for thermal analysis of mechanical structure. A few examples of heat transfer systems are given to illustrate the applicability of the method and the challenges that must be addressed in solving the inverse problem. Design/methodology/approach In this paper, the thermal network method and the finite difference method are used to model the two-dimensional heat conduction inverse problem of the tube structure, and the heat balance equation is arranged into an explicit form for heat load prediction. To solve the matrix ill-conditioned problem in the process of solving the inverse problem, a Tikhonov regularization parameter selection method based on the inverse computation-contrast-adjustment-approach was proposed. Findings The applicability of the proposed method is illustrated by numerical examples for different dynamically varying heat source functions. It is proved that the method can predict dynamic heat source with different complexity. Practical implications The modeling calculation method described in this paper can be used to predict the boundary conditions for the inner wall of the heat transfer tube, where the temperature sensor cannot be placed. Originality/value This paper presents a general method for the direct prediction of heat sources or boundary conditions in mechanical structure. It can directly obtain the time-varying heat flux load and thtemperature field of the machine structure.

Improving the Robustness of Variable Selection and Predictive Performance of Regularized Generalized Linear Models and Cox Proportional Hazard Models.

High-dimensional data applications often entail the use of various statistical and machine-learning algorithms to identify an optimal signature based on biomarkers and other patient characteristics that predicts the desired clinical outcome in biomedical research. Both the composition and predictive performance of such biomarker signatures are critical in various biomedical research applications. In the presence of a large number of features, however, a conventional regression analysis approach fails to yield a good prediction model. A widely used remedy is to introduce regularization in fitting the relevant regression model. In particular, a penalty on the regression coefficients is extremely useful, and very efficient numerical algorithms have been developed for fitting such models with different types of responses. This -based regularization tends to generate a parsimonious prediction model with promising prediction performance, i.e., feature selection is achieved along with construction of the prediction model. The variable selection, and hence the composition of the signature, as well as the prediction performance of the model depend on the choice of the penalty parameter used in the regularization. The penalty parameter is often chosen by K-fold cross-validation. However, such an algorithm tends to be unstable and may yield very different choices of the penalty parameter across multiple runs on the same dataset. In addition, the predictive performance estimates from the internal cross-validation procedure in this algorithm tend to be inflated. In this paper, we propose a Monte Carlo approach to improve the robustness of regularization parameter selection, along with an additional cross-validation wrapper for objectively evaluating the predictive performance of the final model. We demonstrate the improvements via simulations and illustrate the application via a real dataset.

Bayesian Trend Filtering via Proximal Markov Chain Monte Carlo

Proximal Markov chain Monte Carlo is a novel construct that lies at the intersection of Bayesian computation and convex optimization, which helped popularize the use of nondifferentiable priors in Bayesian statistics. Existing formulations of proximal MCMC, however, require hyperparameters and regularization parameters to be prespecified. In this article, we extend the paradigm of proximal MCMC through introducing a novel new class of nondifferentiable priors called epigraph priors. As a proof of concept, we place trend filtering, which was originally a nonparametric regression problem, in a parametric setting to provide a posterior median fit along with credible intervals as measures of uncertainty. The key idea is to replace the nonsmooth term in the posterior density with its Moreau-Yosida envelope, which enables the application of the gradient-based MCMC sampler Hamiltonian Monte Carlo. The proposed method identifies the appropriate amount of smoothing in a data-driven way, thereby automating regularization parameter selection. Compared with conventional proximal MCMC methods, our method is mostly tuning free, achieving simultaneous calibration of the mean, scale and regularization parameters in a fully Bayesian framework. Supplementary materials for this article are available online.

Multiparameter Elastic Full Waveform Inversion Based on Random Source-Encoding and Projection Regularization

Multi-parameter elastic full waveform inversion (FWI) makes full use of the dynamic and kinematic information of all seismic wavefield. Through the mutual constraint and verification of the three parameters of P-wave velocity, S-wave velocity, and density, the joint evaluation is carried out, which is helpful to understand the structural and lithologic information of underground media more comprehensively. The bottleneck restricting the multi-parameter FWI is the large amount of calculation and low efficiency. To improve this problem, multiple shots are directly superimposed to form super shots. While it usually results in an unstable inversion due to that a large amount of crosstalk noise will be easily generated between adjacent shots. In this paper, we introduce the random source-encoding strategy to improve the inversion efficiency and load the total-variation (TV) regularization term to suppress the crosstalk noise, but it also brings the problem of regularization parameters selection for multi-parameter FWI. Thus, the projection method is applied to directly load the regularization term into the model as a constraint, which avoids the unsatisfactory results caused by the improper selection of regularization parameters and effectively improves the ill-posedness of inversion. Finally, three examples of the graben, the 1994BP, and the overthrust model are used to prove that the proposed algorithm based on random source-encoding and projection regularization can effectively improve the inversion efficiency, suppress noise, and has good practicability and adaptability.

A gridless method for direction finding with sparse arrays in nonuniform noise

The performance of direction finding methods would deteriorate due to unknown nonuniform noise. To cope with this problem, we propose a novel gridless direction finding method based on atomic norm minimization exploiting sparse linear array in nonuniform noise. Specifically, after eliminating the concentrated nonuniform noise related term in coarray signal, the concept of array interpolation is used to recover both the noiseless counterpart of the removed term as well as the holes in coarray. Thus, the effect of nonuniform noise is removed. Besides, we impose a new constraint based on the estimation error distribution of the noise independent terms in the coarray signal. The regularization parameter can thus be selected directly from the Chi-square distribution probability table. In the proposed method, the tedious selection of regularization parameter and the effect of grid mismatch are avoided. Moreover, we derive the corresponding semidefinite programming (SDP) form. With its optimal solution, eigen-decomposition with high complexity is avoided for subsequent DOA estimation. Different from the traditional SDP form, it has an additional transformation matrix composed of the estimation error. Simulations show that the proposed method owns the highest estimation accuracy than previous algorithms in the nonuniform noise.

A Modified Regularization Method for a Spherically Symmetric Inverse Heat Conduction Problem

In this paper, we investigate a spherically symmetric inverse heat conduction problem, which determines the internal surface temperature distribution of the hollow sphere from measured data at the fixed location inside it. This problem is ill-posed, and a conditional stability result of its solution is given. A modified quasi-boundary value method is proposed to solve the ill-posed problem. Two Ho¨lder-type error estimates between the approximation solution and its exact solution are obtained under an a priori and an a posteriori regularization parameter selection rule, respectively.

Iterative hybrid regularization for extremely noisy full models in single particle analysis

Cryo-electron microscopy single particle analysis (SPA) represents a vital tool for structure determination of macromolecules. Discrete inverse problems arising in this field are extremely large and seriously contaminated by noise. The model matrix is highly structured and can not be stored explicitly. Iterative regularization methods are used here only rarely, since it is believed that they are computationally expensive and require complicated stopping criteria. In this paper, we overcome these difficulties and demonstrate that Hybrid Krylov subspace methods can be used to solve SPA inverse problems efficiently. We propose an application-driven regularization parameter selection approach and present a matrix-free implementation of the hybrid solver for GPU computations in single precision arithmetic. In comparison to Fourier-based techniques, the hybrid approach allows to compute reconstructions from full (nonreduced) SPA models even for highly noisy data sets.

Fast spectral solver for the inversion of boundary data problem of Poisson equation in a doubly connected domain

In this paper, we study the inversion of the inner boundary data for Poisson equation in a doubly connected domain by fast Fourier ultraspherical spectral solver. The solver depends on the truncated Fourier series expansion, where the differential equations of the Fourier coefficients are solved by an ultraspherical spectral method. Because this problem is seriously ill‐posed, the Cauchy data with noise will lead to ill‐conditioned linear system. Hence, we apply Tikhonov regularization to solve the obtained linear system and use generalized cross‐validation (GCV) criterion to select regularization parameters. The accuracy and efficiency of the proposed method are illustrated by several numerical results of different regions.

Selecting Regularization Parameters for Nuclear Norm--Type Minimization Problems

The reconstruction of a low-rank matrix from its noisy observation finds its usage in many applications. It can be reformulated into a constrained nuclear norm minimization problem, where the bound $\eta$ of the constraint is explicitly given or can be estimated by the probability distribution of the noise. When the Lagrangian method is applied to find the minimizer, the solution can be obtained by the singular value thresholding operator, where the thresholding parameter $\lambda$ is related to the Lagrangian multiplier. In this paper, we first show that the Frobenius norm of the discrepancy between the minimizer and the observed matrix is a strictly monotonically increasing function of $\lambda$. From that we derive a closed form solution for $\lambda$ in terms of $\eta$. Since $\lambda$ is the same as the regularization parameter for the unconstrained regularized problem, our results can be applied to automatically choosing a suitable regularization parameter for the nuclear norm--type regularized minimization problems using the discrepancy principle. The regularization parameters obtained are comparable to (and sometimes better than) those obtained by Stein's unbiased risk estimator approach, while the cost of solving the minimization problem can be reduced by 11--18 times. Numerical experiments with both synthetic data and real MRI data are performed to validate the proposed approach.

Compressed feature reconstruction for localized fault diagnosis with generalized minimax-concave penalty

Compressed sensing can effectively relieve the burden of data storage and transmission for mechanical condition monitoring. However, feature reconstruction from compressed observations has always been a challenge. A novel compressed feature reconstruction method for the impulse response signal caused by localized damage is proposed based on the generalized minimax-concave (GMC) penalty. To improve the efficiency of compression and reconstruction, the fault signal is divided into several segments before compression, and the segmented sparse coefficient is obtained by solving a GMC problem. The fault-caused impulse signal is obtained by combining all the reconstructed segments together. Furthermore, a strategy of regularization parameter selection based on the sparse coefficient proxy is presented. Compared with the commonly used l1 regularization-based reconstruction method, the proposed method can better preserve the amplitudes of impulse features. The effectiveness of the proposed method is further verified by simulation and experimental data.

The Landweber Iterative Regularization Method for Solving the Cauchy Problem of the Modified Helmholtz Equation

In this manuscript, the Cauchy problem of the modified Helmholtz equation is researched. This inverse problem is a serious ill-posed problem. The classical Landweber iterative regularization method is designed to find the regularized solution of this inverse problem. The error estimations between the exact solution and the regularization solution are all obtained under the a priori and the a posteriori regularization parameter selection rule. The Landweber iterative regularization method can also be applied to solve the Cauchy problem of the modified Helmholtz equation on the spherically symmetric and cylindrically symmetric regions.

Feature-guided regularization parameter selection in sparse de-noising for fault diagnosis

Using the sparse and redundant representation model to de-noise vibration signals is an effective method for mechanical fault diagnosis. The selection of the regularization parameter in the sparse de-noising algorithm plays a vital role in reconstructing fault features. The current choices, however, are primarily based on fixed-form parameters derived under the assumption of additive white noise or based on the “trial and error” method. As a result, the extracted de-noised signals often fail to show fault features. To solve this difficulty, a self-adaptive regularization parameter selection strategy is proposed, which directly measures the significance of fault features in sparse de-noised signals. Firstly, the entire solution path is obtained by the computationally efficient least angle regression (LARS) algorithm. Besides, a feature presence probability index (FP) is well designed to measure the significance of fault features in the reconstructed signal envelope demodulation spectrum. Accordingly, with the help of the maximal FP indicator, the optimal regularization parameter and LARS solution is determined. The proposed method is not limited by noise probability distributions; therefore, it shows the superiority of extracting fault features in both the simulation and experimental data analysis.

TSVD Regularization-Parameter Selection Method Based on Wilson-θ and Its Application to Vertical Wheel-Rail Force Identification of Rail Vehicles

A parameter-selection method is proposed to improve the accuracy of the truncated singular value decomposition (TSVD) method, which is based on the Wilson-θ method and the principle of minimum response error, for dynamic load identification. First, using the Green kernel-function matrix, the dynamic load-identification model of the multi-degree-of-freedom system is established. Second, the response corresponding to the dynamic load is identified using the Wilson-θ method, and the minimum error between the response and the input response is obtained. Then, the best regularization parameters are obtained, and the dynamic load is identified using the TSVD regularization method. Finally, the SIMPACK dynamic model of a rail vehicle is established. Taking the German high-interference spectrum as the input, the axle-box displacement and the vertical wheel-rail force of each wheelset at speeds of 100, 160, and 200 km/h are simulated. Taking the simulated axle-box displacement response with 0%, 5%, and 10% noise as the input, the proposed load-identification model and regularization-parameter selection method are used to identify the vertical wheel-rail force of a rail vehicle. The effects of different track spectra on the identification results are considered. The results indicate that this method has a high identification accuracy for the wheel-rail vertical dynamic load. With an increase in the vehicle speed, the correlation coefficient for identifying the dynamic load decreases, but the correlation remains strong. At the speed of 200 km/h, when the input response noise level is 0%, the dynamic load identification correlation coefficient is 0.9556, which corresponds to extremely strong correlation. When the input response contains 5% noise, this method has stronger robustness than L-curve method, and the dynamic load identification correlation coefficient is 0.6354, which corresponds to strong correlation. The proposed load-identification model and regularization-parameter selection method have important theoretical and engineering application value for wheel-rail force monitoring and safety assessment of running trains.

An improved sparsity-enhanced decomposition signal method based on GMC and TQWT for rolling bearing faults

Mechanical equipment is always exposed to poor working environments, such as high humidity, high temperature and heavy loads, which may lead to serious damage in key components. It is critical to identify the initial fault in time to avoid huge economic losses and casualties. In extracting the fault characteristics of a rolling bearing, its characteristic frequency is always disturbed by strong noise. In order to accurately separate the fault features from the strong noisy signal, an improved sparsity-enhanced decomposition signal method using the nonconvex penalty term of generalized minimax-concave and the dictionary term of tunable Q-factor wavelet transform is presented in this paper. An adaptive method for selecting regularization parameters is presented to subtly minimize the signal-to-noise ratio and root mean square error. Moreover, in order to reduce calculation cost, the forward–backward splitting algorithm is employed to maintain the convexity of the proposed sparsity. A simulation study and two practical fault experiments are used to validate the effectiveness of the proposed method in rolling bearing faults.

Research on Tikhonov regularization parameter selection in dynamic light scattering measurement of flowing particles

Research on Tikhonov regularization parameter selection in dynamic light scattering measurement of flowing particles

A parallel sampling algorithm for some nonlinear inverse problems

AbstractWe derive a parallel sampling algorithm for computational inverse problems that present an unknown linear forcing term and a vector of nonlinear parameters to be recovered. It is assumed that the data are noisy and that the linear part of the problem is ill-posed. The vector of nonlinear parameters ${m} $ is modeled as a random variable. A dilation parameter $\alpha $ is used to scale the regularity of the linear unknown and is also modeled as a random variable. A posterior probability distribution for $({m}, \alpha )$ is derived following an approach related to the maximum likelihood (ML) regularization parameter selection (Galatsanos, N. P. & Katsaggelos, A. K. (1992). Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation. IEEE Trans. Image Process., 1, 322–336). A major difference in our approach is that, unlike in Galatsanos, N. P. & Katsaggelos, A. K. (1992, Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation. IEEE Trans. Image Process., 1, 322–336), we do not limit ourselves to the maximum likelihood value of $\alpha $. We then derive a parallel sampling algorithm where we alternate computing proposals in parallel and combining proposals to accept or reject them as in Calderhead, B. (2014, A general construction for parallelizing metropolis- hastings algorithms. Proc. Natl Acad Sci, 111, 17408–17413). This algorithm is well suited to problems where proposals are expensive to compute. We then apply it to an inverse problem in seismology. We show how our results compare favorably to those obtained from the ML, the Generalized Cross Validation and the Constrained Least Squares algorithms.

Bootstrap confidence interval of ridge regression in linear regression model: A comparative study via a simulation study

It is well known that the variances of the least squares estimates are large and they can be far away from their true values in the case of multicollinearity. Therefore, the ridge regression method can be used as an alternative to the least squares method. However, the ridge estimator has a disadvantage that its distribution is unknown, so only asymptotic confidence intervals are obtained. The purpose of this paper is to study the impact of several ridge regularization parameters on the mean interval lengths of the confidence intervals and coverage probabilities constructed by the ridge estimator. A bootstrap method for the selection of ridge regularization parameter is used as well as the parametric methods. In order to compare the confidence intervals, standard normal approximation, student-t approximation and bootstrap methods are used and comparison is illustrated via real data and simulation study. The simulation study shows that the bootstrap choice of ridge regularization parameter yields narrower standard normal approximated confidence intervals than the PRESS choice of ridge regularization parameter but wider standard normal approximated, student-t approximated and bootstrap confidence intervals than the GCV choice of ridge regularization parameter.