Selection Of Regularization Parameter Research Articles

Data-Driven Regularization Parameter Selection in Dynamic MRI.

In dynamic MRI, sufficient temporal resolution can often only be obtained using imaging protocols which produce undersampled data for each image in the time series. This has led to the popularity of compressed sensing (CS) based reconstructions. One problem in CS approaches is determining the regularization parameters, which control the balance between data fidelity and regularization. We propose a data-driven approach for the total variation regularization parameter selection, where reconstructions yield expected sparsity levels in the regularization domains. The expected sparsity levels are obtained from the measurement data for temporal regularization and from a reference image for spatial regularization. Two formulations are proposed. Simultaneous search for a parameter pair yielding expected sparsity in both domains (S-surface), and a sequential parameter selection using the S-curve method (Sequential S-curve). The approaches are evaluated using simulated and experimental DCE-MRI. In the simulated test case, both methods produce a parameter pair and reconstruction that is close to the root mean square error (RMSE) optimal pair and reconstruction. In the experimental test case, the methods produce almost equal parameter selection, and the reconstructions are of high perceived quality. Both methods lead to a highly feasible selection of the regularization parameters in both test cases while the sequential method is computationally more efficient.

Hierarchical Bayesian 3D Traction Force Microscopy with Local Regularization Based on Image Quality

Traction force microscopy is extensively used to quantify the stresses exerted by adherent cells on elastic substrates, using as input the measured displacements of embedded marker beads and the cell contour. The analysis leads to a mathematical inverse problem, which is numerically ill-conditioned. It is widely recognized that the microbead image quality and cell segmentation uncertainties dramatically affect the inversion. However, the most current methods treat substrate displacements and segmented cell contours as ground truths and/or do not account for their regional variations. Such global regularization approaches cause systematic underestimation of true traction stresses and sub-optimal noise suppression. We present a new hierarchical Bayesian inference TFM (HiBi-TFM) method which incorporates local uncertainties in the images, automatized regularization parameter selection and physical constraints. HiBi-TFM uses the 3D elastostatic Green's function to deconvolve the measured displacements, and automatically assigns larger weights to regions with higher signal-to-noise ratio in the deconvolution residuals. The posterior space is further restricted by applying priors to enforce physical constraints (Balance of forces, moments, traction generation within cell area, etc.). The hierarchical nature of the inference allows for conditioning the priors on the segmentation quality of the cell contours. By choosing all the hyper-parameters from the image itself in this self-sufficient model, we infer the optimal, physically meaningful 3D traction field that faithfully reproduces the measured 3D displacements up to individual uncertainties. We tested HiBi-TFM on synthetic images for both microbeads (spatial heterogeneity, imaging artifacts, etc.) and cell contours (edge intensity gradients, etc.) and found that this hierarchical approach greatly improves previous methods. Of note, it increases spatial resolution and recovers biologically significant traction signatures in an unbiased manner, in the presence of noisy data.

Adaptive regularized parameter selection based on random projection

Arbitrary projection is a well-known AI calculation, which can be executed by neural organizations and prepared in an exceptionally productive way. Notwithstanding, the quantity of highlights should be huge enough when applied to a fairly enormous scope informational collection, which brings about moderate speed in testing technique and more extra room under certain conditions. Besides, a portion of the highlights are repetitive and even uproarious since they are arbitrarily created, so the presentation might be influenced by these highlights. In the proposed framework is to actualize Adaptive Regularization Parameter Selection. It will regularize the highlights which are significant and select a most appropriate component for an explicit issue. The primary objective is to diminish the computational expense of both characterization and relapse task by utilizing Randomization Algorithm. It gives the most ideal outcome to the high-dimensional improvement. It utilizes Multilayer neural organization for performing direct and non-straight capacity .

Determination of the reaction coefficient in a time dependent nonlocal diffusion process

This paper investigates the inverse reaction coefficient problem for a time dependent nonlocal diffusion equation by utilizing the nonlocal flux measurement from an accessible part of region, which is a continuation and an extension of our recent work (Zheng and Ding 2020 Inverse Problems 36 035006). The uniqueness of inverse reaction coefficient problem is proved. The variational regularization method is proposed to overcome the ill-posedness of inverse problem, and an alternating iteration method is adopted to automatically select regularization parameters. Furthermore, we use Laplace approximation method to capture the statistics information of the solution. In particular, we prove the lower bound of confidence width is rigorously estimated to characterize the reliability of the method. Finally, numerical results indicate that the proposed method could yield an accurate estimate and efficient uncertainty quantification of the solution.

Rotating acoustic source localization: A power propagation forward model and its high-resolution inverse methods

The rotating-acoustic source localization has played an important role in rotary machine fault diagnosis, engine noise reduction and propeller recognition etc. To achieve high-resolution localization of fast rotating sources, this paper proposes an improved power propagation forward model of rotating acoustic source based on the equivalent source hypothesis. Then the ill-posed inverse problem derived from the forward power model is solved robustly by proposed two sparsity-based methods: the Least Absolute Shrinkage and Select Operator for Rotating-acoustic Sources (LASSO-RS) and Least Angle Regression for Rotating-acoustic Sources (LAR-RS). The contributions of this paper are that, at one hand, the improved forward model is indeed time-invariant for the fast-rotating source owing to the time-domain de-Doppler technique; at the other hand, high-resolution acoustic maps can be obtained by proposed LASSO-RS and LAR-RS thanks to the sparsity-based regularization, even under very low signal-to-noise ratio (SNR). Moreover, the proposed LAR-RS is the data-driven method, rather than the parameter-dependent method such as the LASSO-RS with careful selection of regularization parameter. In simulations and experiments, the proposed methods can robustly localize rotating monopoles in the condition of 3000 RPM rotation speed (50 Hz), −5 dB SNR, and 2500 Hz working frequency.

Target Adaptive Differential Iterative Reconstruction (TADI): A Robust Algorithm for Real-Time Electrical Impedance Tomography

Electrical impedance tomography (EIT) is a noninvasive functional diagnostic technique that has been successfully applied to human lung and brain. EIT reconstruction is an ill-conditioned problem: the regularization parameters establish a trade-off between acceptable data fitting and acceptable solution stability. This trade-off is necessary to select the optimal regularization parameters for each data frame to continuously obtain high-quality images in real-time monitoring. However, using the objective regularization parameter selection method before each image reconstruction may reduce the reconstruction efficiency, and using heuristic selection for all images may cause significant quality degradation, thereby creating challenges in long-term clinical EIT monitoring. This study explores a robust EIT target adaptive differential iterative reconstruction algorithm that does not require the advance selection of the optimal regularization parameter to facilitate the clinical application of EIT long-term bedside monitoring. Specifically, second-order Taylor approximation expansion and Euler–Lagrange theorem were used to derive the conductivity differential iteration relationship. Furthermore, the reconstruction quality and generalization ability of the proposed algorithm were validated based on comparisons with the L-curve and the generalized cross-validation parameter selection methods through simulations, phantom experiments, and human lung recruitment data. Compared with the L-curve and generalized cross-validation methods, the TADI algorithm reduces the position error by 33.07% and 47.17%, the ringing by 26.49% and 32.69%, and the shape deformation by 18.34% and 19.40% on average, respectively. The results revealed that the TADI algorithm could achieve a significant improvement over the existing state-of-the-art methods in terms of stability, consistency, and efficiency.

TIKHONOV REGULARIZATION METHOD OF AN INVERSE SPACE-DEPENDENT SOURCE PROBLEM FOR A TIME-SPACE FRACTIONAL DIFFUSION EQUATION

The aim of this paper is to identify a space-dependent source term in the time-space fractional diffusion equation with an initial-boundary data and an additional measurement data at the final time point. A series expression for the solution of the direct problem is used to transfer the inverse problem into the first type of Fredholm integral equation. Before solving the inverse problem, the uniqueness of its solution is proved. We then use the Tikhonov regularization method to deal with the integral equation and obtain a series expression for the regularized solution of the inverse problem. Moreover, according to the prior and the posterior regularization parameter selection rules, we prove the convergence rates of the regularization solution. Finally, we provide some numerical experiments to show the effectiveness of our method.

Effect of APOE \u03b54 on multimodal brain connectomic traits: a persistent homology study

BackgroundAlthough genetic risk factors and network-level neuroimaging abnormalities have shown effects on cognitive performance and brain atrophy in Alzheimer’s disease (AD), little is understood about how apolipoprotein E (APOE) ε4 allele, the best-known genetic risk for AD, affect brain connectivity before the onset of symptomatic AD. This study aims to investigate APOE ε4 effects on brain connectivity from the perspective of multimodal connectome.ResultsHere, we propose a novel multimodal brain network modeling framework and a network quantification method based on persistent homology for identifying APOE ε4-related network differences. Specifically, we employ sparse representation to integrate multimodal brain network information derived from both the resting state functional magnetic resonance imaging (rs-fMRI) data and the diffusion-weighted magnetic resonance imaging (dw-MRI) data. Moreover, persistent homology is proposed to avoid the ad hoc selection of a specific regularization parameter and to capture valuable brain connectivity patterns from the topological perspective. The experimental results demonstrate that our method outperforms the competing methods, and reasonably yields connectomic patterns specific to APOE ε4 carriers and non-carriers.ConclusionsWe have proposed a multimodal framework that integrates structural and functional connectivity information for constructing a fused brain network with greater discriminative power. Using persistent homology to extract topological features from the fused brain network, our method can effectively identify APOE ε4-related brain connectomic biomarkers.

A Stable Approach for Numerical Differentiation by Local Regularization Method with its Regularization Parameter Selection Strategies

The local regularization method for solving the first-order numerical differentiation problem is considered in this paper. The a-priori and a-posteriori selection strategy of the regularization parameter is introduced, and the convergence rate of local regularization solution under some assumption of the exact derivative is also given. Numerical comparison experiments show that the local regularization method can reflect sharp variations and oscillations of the exact derivative while suppress the noise of the given data effectively.

A potential‐free field inverse time‐fractional Schrödinger problem: Optimal error bound analysis and regularization method

In this paper, an inverse time‐fractional Schrödinger problem of potential‐free field is studied. This problem is ill‐posed; that is, the solution (if it exists) does not depend continuously on the data. Based on an a priori bound condition, the optimal error bound analysis is given. Moreover, a modified kernel method is introduced. The convergence error estimate obtained by this method under the a priori regularization parameter selection rule is optimal, and the convergence error estimate obtained under the a posteriori regularization parameter selection rule is order‐optimal. Finally, some numerical examples are given to illustrate the effectiveness and stability of this method.

MCPeSe: Monte Carlo penalty selection for graphical lasso.

MotivationGraphical lasso (Glasso) is a widely used tool for identifying gene regulatory networks in systems biology. However, its computational efficiency depends on the choice of regularization parameter (tuning parameter), and selecting this parameter can be highly time consuming. Although fully Bayesian implementations of Glasso alleviate this problem somewhat by specifying a priori distribution for the parameter, these approaches lack the scalability of their frequentist counterparts.ResultsHere, we present a new Monte Carlo Penalty Selection method (MCPeSe), a computationally efficient approach to regularization parameter selection for Glasso. MCPeSe combines the scalability and low computational cost of the frequentist Glasso with the ability to automatically choose the regularization by Bayesian Glasso modeling. MCPeSe provides a state-of-the-art ‘tuning-free’ model selection criterion for Glasso and allows exploration of the posterior probability distribution of the tuning parameter.Availability and implementationR source code of MCPeSe, a step by step example showing how to apply MCPeSe and a collection of scripts used to prepare the material in this article are publicly available at GitHub under GPL (https://github.com/markkukuismin/MCPeSe/).Supplementary information Supplementary data are available at Bioinformatics online.

Momentum-Net: Fast and Convergent Iterative Neural Network for Inverse Problems.

Iterative neural networks (INN) are rapidly gaining attention for solving inverse problems in imaging, image processing, and computer vision. INNs combine regression NNs and an iterative model-based image reconstruction (MBIR) algorithm, often leading to both good generalization capability and outperforming reconstruction quality over existing MBIR optimization models. This paper proposes the first fast and convergent INN architecture, Momentum-Net, by generalizing a block-wise MBIR algorithm that uses momentum and majorizers with regression NNs. For fast MBIR, Momentum-Net uses momentum terms in extrapolation modules, and noniterative MBIR modules at each iteration by using majorizers, where each iteration of Momentum-Net consists of three core modules: image refining, extrapolation, and MBIR. Momentum-Net guarantees convergence to a fixed-point for general differentiable (non)convex MBIR functions (or data-fit terms) and convex feasible sets, under two asymptomatic conditions. To consider data-fit variations across training and testing samples, we also propose a regularization parameter selection scheme based on the "spectral spread" of majorization matrices. Numerical experiments for light-field photography using a focal stack and sparse-view computational tomography demonstrate that, given identical regression NN architectures, Momentum-Net significantly improves MBIR speed and accuracy over several existing INNs; it significantly improves reconstruction quality compared to a state-of-the-art MBIR method in each application.

Current distribution reconstruction in switching arcs by means of regularization based on GSVD

The knowledge of the variation of arc current is helpful in improving the breaking performance of a vacuum circuit breaker. As a novel non-intrusive method, arc magnetic performance testing technology has attracted attention. In view of the widely used regularization method of the standard form in arc current reconstructions cannot fully overcome the ill-posed nature of the Biot-Savart equation, the regularization approach based on generalized singular value decomposition (GSVD) is proposed to improve the situation. Moreover, a discrete 3-D arc model with the help of the finite element technique incorporating the current divergence-free and boundary conditions is introduced to improve the authenticity of the model. The performance of the truncated generalized singular value decomposition (TGSVD) and general-form Tikhonov regularization with derivative operators L 1 and L 2 are studied in numerical tests. In addition, the effectiveness of generalized cross validation criterion, quasi optimality criterion and normalized cumulative periodic graph method in selecting regularization parameters are compared. The methodology can help improve arc control and guide the design of vacuum circuit breakers with improved performance parameters.

Improved depiction of subthalamic nucleus and globus pallidus internus with optimized high-resolution quantitative susceptibility mapping at 7 T.

The subthalamic nucleus (STN) and globus pallidus internus (GPi) are commonly used targets in deep-brain stimulation (DBS) surgery for the treatment of movement disorders. The success of DBS critically depends on the spatial precision of stimulation. By taking advantage of good contrast between iron-rich deep-brain nuclei and surrounding tissues, quantitative susceptibility mapping (QSM) has shown promise in differentiating the STN and GPi from the adjacent substantia nigra and globus pallidus externus, respectively. Nonlinear morphology-enabled dipole inversion (NMEDI) is a widely used QSM algorithm, but the image quality of reconstructed susceptibility maps relies on the regularization parameter selection. To date, few studies have systematically optimized the regularization parameter at the ultra-high field of 7 T. In this study, we optimized the regularization parameter in NMEDI to improve the depiction of STN and GPi at different spatial resolutions at both 3 T and 7 T. The optimized QSM images were further compared with other susceptibility-based images, including T2*-weighted (T2*w), R2*, susceptibility-weighted, and phase images. QSM showed better depiction of deep-brain nuclei with clearer boundaries compared with the other methods, and 7 T QSM at 0.35 × 0.35 × 1.0 mm3 demonstrated superior performance to the others. Our findings suggest that optimized high-resolution QSM at 7 T allows for improved delineation of deep-brain nuclei with clear and sharp borders between nuclei, which may become a promising tool for DBS nucleus preoperative localization.

Comparative studies on the criteria for regularization parameter selection based on moving force identification

The studies on inverse problems exist extensively in aerospace, mechanical, identification, detection, scanning imaging and other fields. Its ill-posed characteristics often lead to large oscillations in the solution of the inverse problem. In this study, the truncated generalized singular value decomposition (TGSVD) method is introduced to identify two kinds of moving forces, single and multi-axial forces. The truncating point is the most influential regularization parameter of TGSVD, which is initially selected by two classic regularization parameter selection criteria, namely, the L-curve criterion and the generalized cross-validation (GCV) criterion. Due to numerical non-uniqueness and noise disturbance in moving force identification (MFI), numerical simulation results show that neither of the two criteria can effectively help select the optimal truncating point of TGSVD. Hence, a relative percentage error (RPE) criterion is proposed for selecting the truncating point of TGSVD. Comparative studies show that the RPE criterion can be used to select the optimal truncating point of TGSVD more accurately against the GCV criterion and L-curve criterion. Moreover, the RPE criterion can be used to reflect the connections between certain properties and the ill-posedness problem existing in MFI, which should be adopted priority for the optimal truncating point selection of TGSVD.

Some features of the analysis of broadband impedance data using distribution of relaxation times

The article discusses the preparation of the data of electrochemical impedance spectroscopy (EIS) for representation in the form of distribution of relaxation times (DRT). In the case of broadband EIS, the measurement errors are not normally distributed and, herewith, cannot be modeled by white Gaussian noise. This circumstance is essential because the DRT method is hugely susceptible to measurement errors. It is shown, that considering the variation of the measurement errors across the frequency range improves the quality of the DRT function. Thus due to a comprehensively optimized problem, the DRT may provide a better resolution of distinct charge relaxation processes than the conventional equivalent circuits. The Python application was written for the calculation of nonparametric DRT function, and the actually measured impedance data of solid electrolytes and RC circuits were used to illustrate the methodology of the calculation. The application uses least-square solver for non-negative solutions along with Tikhonov regularization. All the experience gained by using the prior version of the app, written in Matlab, is built into the new product. Some aspects of the selection of regularization parameter by using, so-called, L-curve is also discussed.

Adaptive total variation and second-order total variation-based model for low-rank tensor completion

Recently, low-rank regularization has achieved great success in tensor completion. However, only considering the global low-rankness is not sufficient, especially for a low sampling rate (SR). Total variation (TV) is introduced into low-rank tensor completion (LRTC) problem to promote the local smoothness by incorporating the first-order derivatives information. However, TV usually leads to undesirable staircase effects. To alleviate these staircase effects, we suggest a first- and second-order TV-based parallel matrix factorization model for LRTC problem, which integrates the local smoothness and global low-rankness by simultaneously exploiting the first- and second-order derivatives information. To solve the proposed model, an efficient proximal alternating optimization (PAO)-based algorithm is developed with theoretical guarantee. Moreover, we suggest a regularization parameter selection strategy to automatically update two regularization parameters, which is able to take advantage of the best properties of each of the two regularization terms. Extensive experiments on different tensor data show the superiority of the proposed method over other methods, particularly for extremely low SRs.

Nonparametric Identification of Nonlinear Added Mass Moment of Inertia and Damping Moment Characteristics of Large-Amplitude Ship Roll Motion

This study aims to investigate the nonlinear added mass moment of inertia and damping moment characteristics of large-amplitude ship roll motion based on transient motion data through the nonparametric system identification method. An inverse problem was formulated to solve the first-kind Volterra-type integral equation using sets of motion signal data. However, this numerical approach leads to solution instability due to noisy data. Regularization is a technique that can overcome the lack of stability; hence, Landweber’s regularization method was employed in this study. The L-curve criterion was used to select regularization parameters (number of iterations) that correspond to the accuracy of the inverse solution. The solution of this method is a discrete moment, which is the summation of nonlinear restoring, nonlinear damping, and nonlinear mass moment of inertia. A zero-crossing detection technique is used in the nonparametric system identification method on a pair of measured data of the angular velocity and angular acceleration of a ship, and the detections are matched with the inverse solution at the same discrete times. The procedure was demonstrated through a numerical model of a full nonlinear free-roll motion system in still water to examine and prove its accuracy. Results show that the method effectively and efficiently identified the functional form of the nonlinear added moment of inertia and damping moment.

Comparing RSVD and Krylov methods for linear inverse problems

In this work we address regularization parameter estimation for ill-posed linear inverse problems with an ℓ2 penalty. Regularization parameter selection is of utmost importance for all of inverse problems and estimating it generally relies on the experience of the practitioner. For regularization with an ℓ2 penalty there exist a lot of parameter selection methods that exploit the fact that the solution and the residual can be written in explicit form. Parameter selection methods are functionals that depend on the regularization parameter where the minimizer is the desired regularization parameter that should lead to a good solution. Evaluation of these parameter selection methods still requires solving the inverse problem multiple times. Efficient evaluation of the parameter selection methods can be done through model order reduction. Two popular model order reduction techniques are Lanczos based methods (a Krylov subspace method) and the Randomized Singular Value Decomposition (RSVD). In this work we compare the two approaches. We derive error bounds for the parameter selection methods using the RSVD. We compare the performance of the Lanczos process versus the performance of RSVD for efficient parameter selection. The RSVD algorithm we use is based on the Adaptive Randomized Range Finder algorithm which allows for easy determination of the dimension of the reduced order model. Some parameter selection also require the evaluation of the trace of a large matrix. We compare the use of a randomized trace estimator versus the use of the Ritz values from the Lanczos process. The examples we use for our experiments are two model problems from geosciences.

Dynamic time warping-based transfer learning for improving common spatial patterns in brain–computer interface

Objective. Common spatial patterns (CSP) is a prominent feature extraction algorithm in motor imagery (MI)-based brain–computer interfaces (BCIs). However, CSP is computed using sample-based covariance-matrix estimation. Hence, its performance deteriorates if the number of training trials is small. To address this problem, this paper proposes a novel regularized covariance matrix estimation framework for CSP (i.e. DTW-RCSP) based on dynamic time warping (DTW) and transfer learning. Approach. The proposed framework combines the subject-specific covariance matrix () estimated using the available trials from the new subject, with a novel DTW-based transferred covariance matrix () estimated using previous subjects’ trials. In the proposed , the available labelled trials from the previous subjects are temporally aligned to the average of the available trials of the new subject from the same class using DTW. This alignment aims to reduce temporal variations and non-stationarities between previous subjects’ trials and the available trials from the new subjects. Moreover, to tackle the problem of regularization parameter selection when only a few trials are available for training, an online method is proposed, where the best regularization parameter is selected based on the confidence scores of the trained classifier on the upcoming first few labelled testing trials. Main results. The proposed framework is evaluated on two datasets against two baseline algorithms. The obtained results reveal that DTW-RCSP significantly outperformed the baseline algorithms at various testing scenarios, particularly, when only a few trials are available for training. Significance. Impressively, our results show that successful BCI interactions could be achieved with a calibration session as small as only one trial per class.