Suitable Regularization Parameter Research Articles

Removing noise and overlapping spikes from extracellular recordings using a regularized denoising autoencoder

Abstract One of the difficulties in sorting extracellularly recorded neuronal action potentials, known as spikes, is noise from several sources superimposed on the target spike. For example, the activity of other neurons is often overlaid on the recorded spikes, denoted as neuronal noise interference (NNI), and interference from neighbouring neurons firing at the same time is known as overlapping spikes. In addition, the noise induced by the recording device is known as Gaussian white noise (GWN). Thus, the appropriate signal denoising process of extracellular recordings is an extremely challenging task, where the classical filtering approach may fail specially in removing the overlapping spikes due to the similarity between the spike waveforms of the neighbouring neurons and the wanted single-unit waveform. This issue is tackled in this work by training regularized denoising autoencoder (RDAE) model with the average spike-waveforms of previously identified single-units, denoted as ground truth. To capture the distinct features of the wanted single-unit waveforms, and weight regularization were employed at the encoder/decoder of the RDAE, respectively. With a suitable regularization parameter, the -RDAE model outperforms the -RDAE model. In general, RDAE shows extraordinary performance not only in removing the NNI and GWN, but also the overlapping spikes.

Automatic Data Curation and Analysis Pipeline for Electrochemical Impedance Spectroscopy Measurements Conducted on Solid Oxide Cell Stacks

In this work, we apply data from Electrochemical Impedance Spectroscopy (EIS) measurements, conducted on a Solid Oxide Cell (SOC) stack, to an automatic data curation and evaluation pipeline. Latter is developed to enable the use of historic data from EIS measurements conducted on SOC stack experiments for Machine Learning (ML) models. We show that the proposed procedure can curate parasitic, inductive impedances, obtained as a common effect of measurements on stack level. In addition, drifts induced by temperature and by steam supply gradients during the EIS measurement can be compensated. The results are experimentally validated on a two-layer SOC stack. For extraction of feature values for subsequent ML models distribution of relaxation times (DRT) deconvolution and equivalent circuit modeling (ECM) are used. To determine a suitable regularization parameter for DRT deconvolution, a variance test is implemented.

Gravity inversion method using L0-norm constraint with auto-adaptive regularization and combined stopping criteria

Abstract. We present a gravity inversion method that can produce compact and sharp images to assist the modeling of non-smooth geologic features. The proposed iterative inversion approach makes use of L0-norm-stabilizing functional, hard and physical parameter inequality constraints and a depth-weighting function. The method incorporates an auto-adaptive regularization technique, which automatically determines a suitable regularization parameter and error-weighting function that helps to improve both the stability and convergence of the method. The auto-adaptive regularization and error-weighting matrix are not dependent on the known noise level. Because of that, the method yields reasonable results even if the noise level of the data is not known properly. The utilization of an effectively combined stopping rule to terminate the inversion process is another improvement that is introduced in this work. The capacity and the efficiency of the new inversion method were tested by inverting randomly chosen synthetic and measured data. The synthetic test models consist of multiple causative blocky bodies, with different geometries and density distributions that are vertically and horizontally distributed adjacent to each other. Inversion results of the synthetic data show that the developed method can recover models that adequately match the real geometry, location and densities of the synthetic causative bodies. Furthermore, the testing of the improved approach using published real gravity data confirmed the potential and practicality of the method in producing compact and sharp inverse images of the subsurface.

Tensor Conjugate-Gradient methods for tensor linear discrete ill-posed problems

<abstract><p>This paper presents three types of tensor Conjugate-Gradient (tCG) methods for solving large-scale linear discrete ill-posed problems based on the t-product between third-order tensors. An automatic determination strategy of a suitable regularization parameter is proposed for the tCG method in the Fourier domain (A-tCG-FFT). An improved version and a preconditioned version of the tCG method are also presented. The discrepancy principle is employed to determine a suitable regularization parameter. Several numerical examples in image and video restoration are given to show the effectiveness of the proposed tCG methods.</p></abstract>

Selecting Regularization Parameters for Nuclear Norm--Type Minimization Problems

The reconstruction of a low-rank matrix from its noisy observation finds its usage in many applications. It can be reformulated into a constrained nuclear norm minimization problem, where the bound $\eta$ of the constraint is explicitly given or can be estimated by the probability distribution of the noise. When the Lagrangian method is applied to find the minimizer, the solution can be obtained by the singular value thresholding operator, where the thresholding parameter $\lambda$ is related to the Lagrangian multiplier. In this paper, we first show that the Frobenius norm of the discrepancy between the minimizer and the observed matrix is a strictly monotonically increasing function of $\lambda$. From that we derive a closed form solution for $\lambda$ in terms of $\eta$. Since $\lambda$ is the same as the regularization parameter for the unconstrained regularized problem, our results can be applied to automatically choosing a suitable regularization parameter for the nuclear norm--type regularized minimization problems using the discrepancy principle. The regularization parameters obtained are comparable to (and sometimes better than) those obtained by Stein's unbiased risk estimator approach, while the cost of solving the minimization problem can be reduced by 11--18 times. Numerical experiments with both synthetic data and real MRI data are performed to validate the proposed approach.

Legendre spectral projection methods for Fredholm integral equations of first kind

Abstract In this paper, we discuss the Legendre spectral projection method for solving Fredholm integral equations of the first kind using Tikhonov regularization. First, we discuss the convergence analysis under an a priori parameter strategy for the Tikhonov regularization using Legendre polynomial basis functions, and we obtain the optimal convergence rates in the uniform norm. Next, we discuss Arcangeli’s discrepancy principle to find a suitable regularization parameter and obtain the optimal order of convergence in uniform norm. We present numerical examples to illustrate the theoretical results.

A numerical comparison of some heuristic stopping rules for nonlinear Landweber iteration

The choice of a suitable regularization parameter is an important part of most regularization methods for inverse problems. In the absence of reliable estimates of the noise level, heuristic parameter choice rules can be used to accomplish this task. While they are already fairly well understood and tested in the case of linear problems, not much is known about their behaviour for nonlinear problems and even less in the respective case of iterative regularization. Hence, in this paper, we numerically study the performance of some of these rules when used to determine a stopping index for Landweber iteration for various nonlinear inverse problems. These are chosen from different practically relevant fields such as integral equations, parameter estimation, and tomography.

Bearing defect identification via evolutionary algorithm with adaptive wavelet mutation strategy

The vibration signals obtained from the bearings are associated with nonlinear and nonstationary characteristics, which hinder the fault identification of the bearing. A scheme of fault identification is proposed to identify the faults in bearing in this paper. The vibration signal first pre-processed by wavelet packet transform, followed by Hilbert-Huang transform, decomposes the reconstructed signal into the IMFs. The L-moment criterion is used to select a prominent IMF. The prominent IMF is further utilized to extract the features from the processed signal. The dataset prepared from features trains the SVM model. A novel optimization technique viz 'Diversity driven multi-parent evolutionary algorithm with adaptive wavelet mutation' is proposed to optimize the SVM parameters to increase its efficiency. The efficacy of the proposed algorithm is tested against benchmark functions; obtained results proved its effectiveness. The overall accuracy of built SVM is found to be 100% with suitable regularization and kernel parameter values.

Re-SSS: Rebalancing Imbalanced Data Using Safe Sample Screening

Different samples can have different effects on learning support vector machine (SVM) classifiers. To rebalance an imbalanced dataset, it is reasonable to reduce non-informative samples and add informative samples for learning classifiers. Safe sample screening can identify a part of non-informative samples and retain informative samples. This study developed a resampling algorithm for Rebalancing imbalanced data using Safe Sample Screening (Re-SSS), which is composed of selecting Informative Samples (Re-SSS-IS) and rebalancing via a Weighted SMOTE (Re-SSS-WSMOTE). The Re-SSS-IS selects informative samples from the majority class, and determines a suitable regularization parameter for SVM, while the Re-SSS-WSMOTE generates informative minority samples. Both Re-SSS-IS and Re-SSS-WSMOTE are based on safe sampling screening. The experimental results show that Re-SSS can effectively improve the classification performance of imbalanced classification problems.

A fast multilevel iteration method for solving linear ill-posed integral equations

Abstract We propose a new concept of noise level: R ⁢ ( K * ) \mathcal{R}(K^{*}) -noise level for ill-posed linear integral equations in Tikhonov regularization, which extends the range of regularization parameter. This noise level allows us to choose a more suitable regularization parameter. Moreover, we also analyze error estimates of the approximate solution with respect to this noise level. For ill-posed integral equations, finding fast and effective numerical methods is a challenging problem. For this, we formulate a matrix truncated strategy based on multiscale Galerkin method to generate the linear system of Tikhonov regularization for ill-posed linear integral equations, which greatly reduce the computational complexity. To further reduce the computational cost, a fast multilevel iteration method for solving the linear system is established. At the same time, we also prove convergence rates of the approximate solution obtained by this fast method with respect to the R ⁢ ( K * ) \mathcal{R}(K^{*}) -noise level under the balance principle. By numerical results, we show that R ⁢ ( K * ) \mathcal{R}(K^{*}) -noise level is very useful and the proposed method is a fast and effective method, respectively.

Failure prediction by regularized fuzzy learning with intelligent parameters selection

Software failure time prediction, which is critical in software dependable evaluation, is among one of the most critical issues in software quality assurance research. Conventional software reliability predictions often use single models to illustrate the overall characteristic of the process of software failure. However, those models often fail to produce good prediction performance owing to the constant changes in software failure patterns during different time periods. Fuzzy systems like Takagi–Sugeno–Kang are multimodel approaches that combine simple submodels to represent the overall characteristic of dynamic nonlinear complex systems. This paper presents a novel Fuzzy system based software reliability prediction model, called regularized extreme learning adaptive neuro-fuzzy inference system, and employs the quantum-inspired binary gravitational search algorithm to determine its suitable regularization parameters. The new model can effectively reduce randomness, computational complexity as well as avoid local optimization when searching for optimal parameters. An experimentation using four real-world software failure data sets was carried out to compare the performance of the proposed model with that of certain representative software reliability prediction models. The results showed that the proposed model with the optimized parameters exhibits superior performance.

Application of Evolution Strategies to the Design of SAR Efficient Parallel Transmit Multi-Spoke Pulses for Ultra-High Field MRI.

We present an evolution-strategy based approach to solve the magnitude least squares (MLS) design problem of low flip-angle slice-selective parallel transmit RF pulses for ultra-high field MRI using SAR and peak-RF-constraints. A combined transmit k-space trajectory and RF pulse weight optimization is proposed in two algorithmic steps. The first step is a coarse grid search to find an initial solution that fulfills all constraints for the subsequent multistage optimization. This avoids convergence to the next nearest local minimum. The second step attempts to refine the results using multiple evolution strategies. We compare the performance of our approach with the non-convex optimization methods described in the literature. The proposed algorithm converges for phantom and in vivo data and only requires an initial estimate of the range of suitable regularization parameters. It demonstrates improved excitation homogeneity compared to published spoke-design methods and allows optimization for homogeneity with a subsequent reduction in the SAR burden. Moreover, excitation homogeneity and the SAR burden can be balanced against each other, enabling a further reduction in SAR at the cost of minor relaxations in excitation homogeneity. This feature makes the algorithm a good candidate for SAR limited sequences in ultra-high field imaging. The algorithm is validated using phantom and in vivo measurements obtained with a 16-channel transmit array at 9.4T.

An [formula omitted]-[formula omitted] minimization method with cross-validation for the restoration of impulse noise contaminated images

Discrete ill-posed problems arise in many areas of science and engineering. Their solutions, if they exist, are very sensitive to perturbations in the data. Regularization aims to reduce this sensitivity. Many regularization methods replace the original problem by a minimization problem with a fidelity term and a regularization term. Recently, the use of a p-norm to measure the fidelity term and a q-norm to measure the regularization term has received considerable attention. The relative importance of these terms is determined by a regularization parameter. When the perturbation in the available data is made up of impulse noise and a sparse solution is desired, it often is beneficial to let 0<p,q<1. Then the p- and q-norms are not norms. The choice of a suitable regularization parameter is crucial for the quality of the computed solution. It therefore is important to develop methods for determining this parameter automatically, without user-interaction. However, the latter has so far not received much attention when the data is contaminated by impulse noise. This paper discusses two approaches based on cross validation for determining the regularization parameter in this situation. Computed examples that illustrate the performance of these approaches when applied to the restoration of impulse noise contaminated images are presented.

Uncertainty Analysis of Stray Field Measurements by Quantitative Magnetic Force Microscopy

Magnetic force microscopy (MFM) measurements generally provide phase images which represent the signature of domain structures on the surface of nanomaterials. To quantitatively determine magnetic stray fields based on an MFM image requires calibrated properties of the magnetic tip. In this work, an approach is presented for calibrating a magnetic tip using a Co/Pt multilayered film as a reference sample which shows stable well-known magnetic properties and well-defined perpendicular band domains. The approach is based on a regularized deconvolution process in Fourier domain with a Wiener filter and the L-curve method for determining a suitable regularization parameter to get a physically reasonable result. The calibrated tip is applied for a traceable quantitative determination of the stray fields of a test sample which has a patial frequency spectrum covered by that of the reference sample. According to the "Guide to the expression of uncertainty in measurement", uncertainties of the processing algorithm are estimated considering the fact that the regularization influences significantly the quantitative analysis. We discuss relevant uncertainty components and their propagations between real domain and Fourier domain for both, the tip calibration procedure and the stray field calculation, and propose an uncertainty evaluation procedure for quantitative magnetic force microscopy.

Finding the Optimal Regularization Parameter in Distribution of Relaxation Times Analysis

AbstractThe distribution of relaxation times (DRT) method allows the direct interpretation of electrochemical impedance data, yielding an increased resolution in the frequency domain. Calculating the DRT from experimental impedance spectra, however, is an intrinsically ill‐posed problem requiring special mathematical treatments such as, for example, the Tikhonov regularization. In this study, we propose a new approach for finding optimal regularization parameters in the Tikhonov regularization. The new test function is based on repetitive impedance measurements and a simple data resampling with a subsequent variance investigation. Furthermore, our new approach is combined with already published procedures considering the determination of best regularization parameters. Here, the combination of several tests enables a clear assignment of an upper and a lower boundary for suitable regularization parameters. Finally, both tests were applied to simulated as well as to experimental impedance data for validation. To analyze the sensitivity of the approach, varying error extent was generated to the synthetic and experimental measurements.

A modified quasi-boundary value method for a two-dimensional inverse heat conduction problem

In this study, we consider a two-dimensional inverse heat conduction problem, which determines the surface temperature and heat flux distribution from measured data at the fixed location. The problem is seriously ill posed in the Hadamard sense and a conditional stability is given for it. We propose a modified quasi-boundary value regularization method to deal with the ill-posed problem. By choosing suitable regularization parameters and introducing some technical inequalities, we obtain quite sharp error estimates between the approximate solution and its exact solution.

Parameter selection and solution algorithm for TGV-based image restoration model

In this paper, image restoration problem is formulated to solve a total generalized variation (TGV)-based minimization problem. The minimization problem includes an unknown regularization parameter. A Morozov’s discrepancy principle-based method is used to choose a suitable regularization parameter. Computationally, by introducing two dual variables, the TGV-based image restoration problem is reformulated as a convex-concave saddle-point problem. Meanwhile, the Chambolle–Pock’s first-order primal–dual algorithm is transformed into a different equivalent form which can be seen as a proximal-based primal–dual algorithm. Then, the different equivalent form is used to solve the saddle-point problem. At last, compared with several existing state-of-the-art methods, experimental results demonstrate the performance of our proposed method.

Isotropic sparse regularization for spherical harmonic representations of random fields on the sphere

This paper discusses isotropic sparse regularization for a random field on the unit sphere S 2 in R 3 , where the field is expanded in terms of a spherical harmonic basis. A key feature is that the norm used in the regularization term, a hybrid of the ℓ 1 and ℓ 2 -norms, is chosen so that the regularization preserves isotropy, in the sense that if the observed random field is strongly isotropic then so too is the regularized field. The Pareto efficient frontier is used to display the trade-off between the sparsity-inducing norm and the data discrepancy term, in order to help in the choice of a suitable regularization parameter. A numerical example using Cosmic Microwave Background (CMB) data is considered in detail. In particular, the numerical results explore the trade-off between regularization and discrepancy, and show that substantial sparsity can be achieved along with small L 2 error.

Reconstructing unknown nonlinear boundary conditions in a time‐fractional inverse reaction–diffusion–convection problem

This paper has focused on unknown functions identification in nonlinear boundary conditions of an inverse problem of a time‐fractional reaction–diffusion–convection equation. This inverse problem is generally ill‐posed in the sense of stability, that is, the solution of problem does not depend continuously on the input data. Thus, a combination of the mollification regularization method with Gauss kernel and a finite difference marching scheme will be introduced to solve this problem. The generalized cross‐validation choice rule is applied to find a suitable regularization parameter. The stability and convergence of the numerical method are investigated. Finally, two numerical examples are provided to test the effectiveness and validity of the proposed approach.

Reconstruction of a time-dependent source term in a time-fractional diffusion-wave equation

ABSTRACTThis paper is devoted to solve an inverse time-dependent source problem for a time-fractional diffusion-wave equation. The source function in the time-fractional diffusion-wave equation is assumed as a multiplication of a temporal function and a spatial function. We try to identify the time-dependent source term according to an additional solute concentration distribution measured at a point on the boundary or an inner point in the solution domain. Firstly, we discuss the continuity of the weak solution for the direct problem. Then, we transform the inverse problem into a first kind of Volterra integral equation and show its ill-posedness. We use a generalized Tikhonov regularization to solve the Volterra integral equation. The generalized cross-validation choice rule is applied to find a suitable regularization parameter. Lastly, we test three examples for the inverse time-dependent source problem and show the effectiveness of the proposed regularization method.