Selection Of Regularization Parameter Research Articles

A Fractional Tikhonov Regularization Method for Identifying a Time-Independent Source in the Fractional Rayleigh–Stokes Equation

The aim of this paper is to identify a time-independent source term in the Rayleigh–Stokes equation with a fractional derivative where additional data are considered at a fixed time point. This inverse problem is proved to be ill-posed in the sense of Hadamard. By using a fractional Tikhonov regularization method, we construct a regularized solution. Then, according to a priori and a posteriori regularization parameter selection rules, we prove the convergence estimates of the regularization method. Finally, we provide some numerical examples to prove the effectiveness of the proposed method.

Improving electrocardiographic imaging solutions: A comprehensive study on regularization parameter selection in L-curve optimization in the Atria

BackgroundIn electrocardiographic imaging (ECGI), selecting an optimal regularization parameter (λ) is crucial for obtaining accurate inverse electrograms. The effects of signal and geometry uncertainties on the inverse problem regularization have not been thoroughly quantified, and there is no established methodology to identify when λ is sub-optimal due to these uncertainties. This study introduces a novel approach to λ selection using Tikhonov regularization and L-curve optimization, specifically addressing the impact of electrical noise in body surface potential map (BSPM) signals and geometrical inaccuracies in the cardiac mesh. MethodsNineteen atrial simulations (5 of regular rhythms and 14 of atrial fibrillation) ensuring variability in substrate complexity and activation patterns were used for computing the ECGI with added white Gaussian noise from 40 dB to -3dB. Cardiac mesh displacements (1–3 cm) were applied to simulate the uncertainty of atrial positioning and study its impact on the L-curve shape. The regularization parameter, the maximum curvature, and the most horizontal angle of the L-curve (β) were quantified. In addition, BSPM signals from real patients were used to validate our findings. ResultsThe maximum curvature of the L-curve was found to be inversely related to signal-to-noise ratio and atrial positioning errors. In contrast, the β angle is directly related to electrical noise and remains unaffected by geometrical errors. Our proposed adjustment of λ, based on the β angle, provides a more reliable ECGI solution than traditional corner-based methods. Our findings have been validated with simulations and real patient data, demonstrating practical applicability. ConclusionAdjusting λ based on the amount of noise in the data (or on the β angle) allows finding optimal ECGI solutions than a λ purely found at the corner of the L-curve. It was observed that the relevant information in ECGI activation maps is preserved even under the presence of uncertainties when the regularization parameter is correctly selected. The proposed criteria for regularization parameter selection have the potential to enhance the accuracy and reliability of ECGI solutions.

A kind regularization method for solving Cauchy problem of the Schrödinger equation

The potential-free field Schrödinger Cauchy problem is the major topic of this research. Because it is gravely ill-posed in the Hadamard sense. The Cauchy problem is modified through an improved boundary method. The regular approximate solution is created based on the priori and posteriori regularization parameter selection rules, and the convergence evidence is provided. The proposed method is practical under the priori and the posteriori regularization methods, according to numerical experiments. It works well and is resistant to data disruption noise.

Damage identification method based on interval regularization theory

In the field of damage identification, traditional regularization methods neglect the impact of uncertainty factors on the selection of regularization parameters, leading to a decrease in the accuracy of damage identification. Therefore, this study proposes a damage identification based on interval truncated singular value decomposition (DI-ITSVD) method that considers the uncertainty in the selection of regularization parameter. This method treats model errors and measurement noise as interval uncertainties, and introduces the quantified uncertainties into the damage identification solutions through uncertainty propagation methods to determine the interval boundary. Uncertainty regularization parameters are selected to balance residuals and solutions using interval and generalized cross-validation methods. The key aspect of the proposed method in this paper is the integration of interval uncertainty propagation with the truncated singular value decomposition method to ensure the accuracy and stability of the damage identification equation solution. A numerical example of a 29-bar planar truss has been performed to test the effectiveness of the proposed method. The superiority of this method is verified by comparing the identification results with other improved truncated singular value decomposition methods. Finally, the practical application effect of the proposed method was also verified through an experimental work.

Bayesian inversion of HFC-134a emissions in southern China from a new AGAGE site: Results from an observing system simulation experiment

The abundance of hydrofluorocarbons (HFCs) in the atmosphere is increasing and is of significant importance to the Earth's system. It is thus crucial to investigate the spatial distribution of HFC emissions. However, inversion modeling, a commonly used approach, is susceptible to errors from a-priori emissions, inversion grids, observations, and other parameters. In this study, we conduct Observing System Simulation Experiment (OSSE) to evaluate the impact of inversion parameters on HFC emission estimates, focusing on HFC-134a at the newly established Xichong (XCG) AGAGE site in Shenzhen, China. We regard the EDGAR (v7) dataset as a reference for “true” emissions and simulate the “true” atmosphere using the GEOS-Chem model. Our OSSE indicates that conducting inversions in the medium-sensitivity region with source-receptor sensitivity larger than 6.0 (nmol mol−1)/(mol m−2 s−1) minimizes the bias between a-posteriori and “true” total emissions to 0.58 %–1.88 %. In the high-sensitivity region, enhancing the spatial resolution of inversion grids lowers this error from −27.07 % to +0.36 %. The accuracy of a-priori spatial distribution crucially influences that of the a-posteriori: the higher correlation coefficient between a-priori and “true” emissions, the better a-posteriori agree with the “true” emissions, as evidenced by a reduced root mean square error (RMSE) of the a-posteriori vs. “true” emissions from 1.44 × 10−9 g m−2 s−1 (GDP-based) to 8.78 × 10−10 g m−2 s−1 (VIIRS-based). Furthermore, selecting regularization parameters and balancing instrumental and a-priori errors are also important. Our OSSE setups allow for parameter testing during the inversion, offering a framework to assess the regional representativeness of future HFC measurement sites.

Uncertainty-Aware Traction Force Microscopy.

Traction Force Microscopy (TFM) is a versatile tool to quantify cell-exerted forces by imaging and tracking fiduciary markers embedded in elastic substrates. The computations involved in TFM are ill-conditioned, and data smoothing or regularization is required to avoid overfitting the noise in the tracked substrate displacements. Most TFM calculations depend critically on the heuristic selection of regularization (hyper)parameters affecting the balance between overfitting and smoothing. However, TFM methods rarely estimate or account for measurement errors in substrate deformation to adjust the regularization level accordingly. Moreover, there is a lack of tools to quantify how these errors propagate to the recovered traction stresses. These limitations make it difficult to interpret TFM readouts and hinder comparing different experiments. This manuscript presents an uncertainty-aware TFM technique that estimates the variability in the magnitude and direction of the traction stress vector recovered at each point in space and time of each experiment. In this technique, substrate deformation and its uncertainty are quantified using a non-parametric bootstrap PIV method by resampling the microscopy image pixels (PIV-UQ). This information is passed to a hierarchical Bayesian framework that automatically selects its hyperparameters to perform spatially adaptive regularization conditioned on image quality and propagates the uncertainty to the traction stress readouts (TFM-UQ). We validate the performance of PIV-UQ and TFM-UQ using synthetic datasets with prescribed image quality variations and demonstrate the application of PIV-UQ and TFM-UQ to experimental datasets. These studies show that TFM-UQ locally adapts the level of smoothing, outperforming traditional regularization methods. They also illustrate how uncertainty-aware TFM tools can be used to objectively choose key image analysis parameters like PIV-UQ interrogation window size. We anticipate that these tools will allow for decoupling biological heterogeneity from measurement variability and facilitate automating the analysis of large datasets by parameter-free, input data-based regularization.

Regularization methods for the inverse initial value problem for the time-fractional diffusion equation with robin boundary condition ★ ★ The project is supported by Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX23_2549) and the Natural Science Foundation of Jiangsu Province, China (Grant No. BK20190578).

In the article, we focus on the inverse initial value problem for the time-fractional diffusion equation with robin boundary condition on a cylindrical symmetric field. The ill-posedness of this problem is proved. We introduce the modified Landweber iteration method(MLIM), the Truncated singular value decomposition(TSVD) method for solving it and propose a new regularization method, named as the TSVD-modified Landweber iteration method(TMLIM). The error estimates between the exact solution and the regularized approximate solution are presented by using two regularization parameter selection rules. Finally, numerical examples are provided to demonstrate the effectiveness and feasibility of the regularization methods.

Learning-Assisted Fast Determination of Regularization Parameter in Constrained Image Reconstruction.

To leverage machine learning (ML) for fast selection of optimal regularization parameter in constrained image reconstruction. Constrained image reconstruction is often formulated as a regularization problem and selecting a good regularization parameter value is an essential step. We solved this problem using an ML-based approach by leveraging the finding that for a specific constrained reconstruction problem defined for a fixed class of image functions, the optimal regularization parameter value is weakly subject-dependent and the dependence can be captured using few experimental data. The proposed method has four key steps: a) solution of a given constrained reconstruction problem for a few (say, 3) pre-selected regularization parameter values, b) extraction of multiple approximated quality metrics from the initial reconstructions, c) predicting the true quality metrics values from the approximated values using pre-trained neural networks, and d) determination of the optimal regularization parameter by fusing the predicted quality metrics. The effectiveness of the proposed method was demonstrated in two constrained reconstruction problems. Compared with L-curve-based method, the proposed method determined the regularization parameters much faster and produced substantially improved reconstructions. Our method also outperformed state-of-the-art learning-based methods when trained with limited experimental data. This paper demonstrates the feasibility and improved reconstruction quality by using machine learning to determine the regularization parameter in constrained reconstruction. The proposed method substantially reduces the computational burden of the traditional methods (e.g., L-curve) or relaxes the requirement of large training data by modern learning-based methods, thus enhancing the practical utility of constrained reconstruction.

Multi-Response Bridge Regularization Parameter Selection via Multivariate Generalized Information Criterion

This paper proposes a multivariate form of generalized information criterion (MGIC) for the multivariate response bridge regression (multi-bridge) model. Also, we prove the identifiability of the multi-bridge as a prerequisite for model selection. We introduce the general form of MGIC for regularization parameter selection in the multi-bridge model. We assess the performance of MGIC variants from three viewpoints: consistency of the obtained models, analysis of high-dimensional data, and comparison to other criteria. Based on the numerical study, we reach better performance for MGIC in comparison to other common criteria (cross-validation and GCV) using simulated and real datasets.

Identification of unknown sources in time-space fractional parabolic equation

This study considers the issue for recognizing unknown source within a time-space fractional parabolic equation. This particular issue is characterized by severe ill-posedness, where the solution does not depend continuously on the data. To tackle this issue, the wavelet dual least squares method is extended to handle ill-posedness of the issue under priori rule. Additionally, a posterior wavelet regularization parameter selection rule is introduced to address the Cauchy problem. Furthermore, error estimates between the exact solution and its approximation are established using both a priori and a posteriori methodologies. Numerical examples are given to validate the effectiveness and stability of the proposed method.

SAR target recognition through adaptive kernel sparse representation model based on local contrast perception

Sparse representation-based methods have shown promising prospects in synthetic aperture radar (SAR) target recognition due to their robustness under complex application conditions. However, these methods are still susceptible to the feature representation of SAR images. In this paper, we propose a new SAR target recognition method using an adaptive kernel sparse representation model based on local contrast perception (LCP). The LCP method combines a specially designed random projection matrix with two-dimensional filtering to capture extensive intensity contrasts between local regions in SAR images. A response map reconstruction technique shows that LCP can obtain discriminative properties between target and background regions and improve feature representation. The LCP features are then transformed into the kernel representation to improve linear separability by defining a reproducing kernel Hilbert space (RKHS) and using a nonlinear kernel function. The kernel representation is utilized to construct a new kernel sparse representation model for classification. Furthermore, neighborhood analysis (NA) is proposed to establish the relationship between the parameter and feature distribution, with an adaptive regularization parameter selection strategy to mitigate the parameter sensitivity. Experiments conducted on the moving and stationary target acquisition and recognition dataset demonstrate the superiority and robustness of the proposed method under different application conditions.

Solving two kinds of inverse source problems for the heat equations by a mollification regularization method with Dirichlet kernel

We study the inverse issues for the heat equations with spatial‐dependent source and time‐dependent source, respectively. In this work, the source identification issues are ill‐posed, and the numerical solutions (if they exist) are not continuously dependent on the data. A mollification regularization method with Dirichlet kernel is proposed to tackle the presented problems. Convergence analyses are carried out via two regularization parameter selection rules (a priori and a posteriori), respectively. Ultimately, a series of numerical experiments are presented to verify our theoretical results.

Study on applicable coverage extension of theory-based generalization errors bounds to the variants of RVFL network and ELM

This study investigates the coverage extendibility of generalized prediction error estimation for the random vector functional link (RVFL) network and generalized extreme learning machine (GELM). GELM is an ELM applying the limiting behavior of Moore–Penrose generalized inverse (M–P GI) to the output matrix of the last hidden layer of ELM. The RVFL network, the ELM, and their variants are currently employed in various areas due to their fast learning and are expected to be adopted in more applications. Additionally, obtaining prediction error bounds quickly for these networks could greatly increase their utilities. Therefore, exploring the error bounds of the variants is crucial. Recently, the upper and lower bounds of generalized prediction error and their associated tail probabilities for the RVFL network and GELM were analyzed. This study aims to extend the coverage of the existing error bounds to variants of RVFL and ELM. To this end, we investigate each solution of the objective functions for the existing RVFL and ELM variants, compare it with the solution obtained by using the limiting behavior of M–P GI, and then discuss the applicability of the probability inequalities for the prediction error to the variant. Specifically, we conceptualize a restricted solution vector and prove that the restricted solution of a variant of RVFL is identical to that of GELM, with an appropriately selected regularization parameter. The mechanism used in the proof can be applied to other variants, which have different objective functions but have similar forms of solution. Experiments are performed with the CAIDA (Center for Applied Internet Data Analysis) dataset. Numerical results verify the analysis, showing that the generalization prediction error bounds and their associated tail probabilities for ELM (RVFL) variants in applicability coverage are almost identical to those of the GELM (RVFL). In addition, performance-based network similarity is introduced and computed with the obtained results. Based on this analysis, we can conclude that the probability inequalities for prediction error derived can be directly applied to some variants of the RVFL network and ELM, such as robust regularized RVFL and ridge regression ELM. However, it cannot apply directly to the networks that use information obtained during the learning process in its learning, such as RVFL+ and its variants. The significance of this study is that it enables us to figure out the relationship between the size-restricted final weight vector of RVFL and the final weight vectors of ELM, the effect of the relationship on generalized prediction error estimation, and the performance-based similarity between networks. This study provides a theoretical basis for the generalized prediction error's upper and lower bounds and their associated tail probabilities for ELM (RVFL) variants.

Improved adaptive regularization for simulated annealing inversion of transient electromagnetic

Geophysical inversion usually involves ill-posed problem. Regularization is the most commonly used method to mitigate this problem. There are many regularization parameter selection methods, among which the adaptive regularization method can automatically update parameters during iteration, reducing the difficulty of parameter selection. Therefore, it is widely used in linear inversion. However, there are very few studies on the use of adaptive regularization methods in stochastic optimization algorithms. The biggest difficulty is that in stochastic optimization algorithms, the search direction of any iteration is completely random. Data fitting term and stabilizing term vary in a wide range, making it difficult for traditional methods to work. In this paper, we consider the contributions of the data fitting term and the stabilizing term in the objective function and give an improved adaptive regularization method for very fast simulated annealing (VFSA) inversion for transient electromagnetic (TEM) data. The optimized method adjusts the two terms dynamically to make them in balance. We have designed several numerical experiments, and the experimental results demonstrate that the method in this paper not only accelerates the convergence, but also the inversion results are very little affected by the initial regularization parameter. Finally, we apply this method to field data, and the inversion results show very good agreements with nearby borehole data.

Utilizing the L-curve criterion for the inverse magnetostatic problem of Hall drift current estimation

Aiming at achieving the in-orbit diagnostic of Hall drift current, this study focuses on estimation through the indirect measurement methodology using a magnetic sensor array. It elaborates on the application of a pseudo-seminorm defined for the Hall drift current solution to address the inverse magnetostatic problems, which are formulated with a two-dimensional Tikhonov regularization constraint, and thereby offering a systematic approach to select regularization parameters. Our investigation discusses factors influencing the formation of the L-curve and the accuracy of the resultant solution obtained via the L-curve criterion. The results reveal that the formation of the defined pseudo-seminorm of the Hall drift current solution in the semi-logarithmic coordinate system is independent of the number of calibrating current elements or the number of magnetic sensors. This effectively resolves the issue of failing to generate an L-curve during regularization parameter selection. Furthermore, the study indicates that expanding the number of calibrating current elements—essentially increasing the unknown variables in the inverse magnetostatic equations—contributes to a significant enhancement in the accuracy of Hall drift current solutions. It also has extensibility to be applied to other areas where the contactless current measuring is required.

On a Backward Problem for the Rayleigh–Stokes Equation with a Fractional Derivative

The Rayleigh–Stokes equation with a fractional derivative is widely used in many fields. In this paper, we consider the inverse initial value problem of the Rayleigh–Stokes equation. Since the problem is ill-posed, we adopt the Tikhonov regularization method to solve this problem. In addition, this paper not only analyzes the ill-posedness of the problem but also gives the conditional stability estimate. Finally, the convergence estimates are proved under two regularization parameter selection rules.

Whole-brain multivariate hemodynamic deconvolution for functional MRI with stability selection

Conventionally, analysis of functional MRI (fMRI) data relies on available information about the experimental paradigm to establish hypothesized models of brain activity. However, this information can be inaccurate, incomplete or unavailable in multiple scenarios such as resting-state, naturalistic paradigms or clinical conditions. In these cases, blind estimates of neuronal-related activity can be obtained with paradigm-free analysis methods such as hemodynamic deconvolution. Yet, current formulations of the hemodynamic deconvolution problem have three important limitations: (1) their efficacy strongly depends on the appropriate selection of regularization parameters, (2) being univariate, they do not take advantage of the information present across the brain, and (3) they do not provide any measure of statistical certainty associated with each detected event. Here we propose a novel approach that addresses all these limitations. Specifically, we introduce multivariate sparse paradigm free mapping (Mv-SPFM), a novel hemodynamic deconvolution algorithm that operates at the whole brain level and adds spatial information via a mixed-norm regularization term over all voxels. Additionally, Mv-SPFM employs a stability selection procedure that removes the need to select regularization parameters and also lets us obtain an estimate of the true probability of having a neuronal-related BOLD event at each voxel and time-point based on the area under the curve (AUC) of the stability paths. Besides, we present a formulation tailored for multi-echo fMRI acquisitions (MvME-SPFM), which allows us to better isolate fluctuations of BOLD origin on the basis of their linear dependence with the echo time (TE) and to assign physiologically interpretable units (i.e., changes in the apparent transverse relaxation ΔR2∗) to the resulting deconvolved events. Remarkably, we demonstrate that Mv-SPFM achieves comparable performance even when using a single-echo formulation. We demonstrate that this algorithm outperforms existing state-of-the-art deconvolution approaches, and shows higher spatial and temporal agreement with the activation maps and BOLD signals obtained with a standard model-based linear regression approach, even at the level of individual neuronal events. Furthermore, we show that by employing stability selection, the performance of the algorithm depends less on the selection of temporal and spatial regularization parameters λ and ρ. Consequently, the proposed algorithm provides more reliable estimates of neuronal-related activity, here in terms of ΔR2∗, for the study of the dynamics of brain activity when no information about the timings of the BOLD events is available. This algorithm will be made publicly available as part of the splora Python package.

Accelerated generalized minimax-concave sparse regularization for impact force reconstruction and localization

Impact force identification has always been of significance for structure health monitoring especially on the applications involving composite materials. As a typical inverse problem, impact force reconstruction and localization is undoubtedly a challenging task. The well-known ℓ1 sparse regularization has a tendency to underestimate the amplitude of impact forces. To alleviate this limitation, we propose an accelerated generalized minimax-concave (AGMC) for sparse regularization that employs a non-convex generalized minimax-concave (GMC) penalty as the regularizer and incorporates an acceleration technique to expedite the attainment of the global minimum. Compared with the classic ℓ1-norm penalty, the GMC penalty can not only induce sparsity in the estimation, but also maintain the convexity of the cost function, so that the global optimal solution can be obtained through convex optimization algorithms. This method is applied to solve the impact force identification problem with unknown force locations to simultaneously reconstruct and localize impact forces in the under-determined case utilizing a limited number of sensors. Meanwhile, K-sparsity criterion is used to adaptively select regularization parameters by taking advantage of the sparse prior knowledge on impact forces. Simulations and experiments are conducted on a composite plate to verify the computational efficiency and robustness of the AGMC method in terms of impact force reconstruction and localization, particularly in the presence of noise. Results demonstrate that the proposed AGMC method achieves faster convergence and provides more accurate and sparse reconstruction and localization of impact forces compared to other state-of-the-art sparse regularization methods.

Fast robust enhanced trend filter: A promising tool for automatically extracting high precision friction coefficient under unknown noise

Friction coefficient is the core parameter of material friction performance test. However, it often suffers from complex unknown noise, which frequently follows non-Gaussian distribution, such as outliers during the test phase. Moreover, the amount of test data is often very large that lead to the heavy computational burden. Thus, how to efficiently and accurately extract the friction coefficient is becoming a challenging task. To address this challenge, this paper proposes a fast trend filtering method based on Robust Enhanced Trend Filtering (RobustETF) which combines mix of Gaussian (MoG) and non-convex sparsity-inducing functions to extend l1 trend filtering, called Fast RobustETF. First, the sliding window strategy is used to segment the original data, and the wear stage is distinguished via judging the data margin factor inside the window. Then the window data with the largest and smallest margins is selected as the characteristic data segment of the stable and severe wear stage, and we consider the trend kurtosis and variance into the selection of optimal regularization parameters for different wear stages. Then, RobustETF is used in each window for extracting high precision friction coefficient under unknown noise. Finally, the trend signals in all windows are integrated and reconstructed to complete the precise extraction of the friction coefficient. Finally, simulation signals and real friction coefficient signals are used to verify the performance of Fast RobustETF by the comparison with other filtering methods.

A new type of generalized information criterion for regularization parameter selection in penalized regression with application to treatment process data

ABSTRACT We propose a new approach to select the regularization parameter using a new version of the generalized information criterion ( GIC ) in the subject of penalized regression. We prove the identifiability of bridge regression model as a prerequisite of statistical modeling. Then, we propose asymptotically efficient generalized information criterion ( AGIC ) and prove that it has asymptotic loss efficiency. Also, we verified the better performance of AGIC in comparison to the older versions of GIC . Furthermore, we propose MSE search paths to order the selected features by lasso regression based on numerical studies. The MSE search paths provide a way to cover the lack of feature ordering in lasso regression model. The performance of AGIC with other types of GIC is compared using MSE and model utility in simulation study. We exert AGIC and other criteria to analyze breast and prostate cancer and Parkinson disease datasets. The results confirm the superiority of AGIC in almost all situations.