Optimal Regularization Research Articles

Numerical and experimental study on acoustic tomographic reconstruction of flow field

This paper reports the acoustic vector tomography (AVT) method to reconstruct the airflow as a vector field. In order to increase the efficiency of reconstruction process, proposed radial basis functions approximate the flow velocity components. Tikhonov-Gaussian regularization combined with Generalized Singular Value Decomposition method and modified L-curve are introduced to solve the ill-posed problem and select the optimal regularization parameters. The feasibility and validity of the method are verified by numerical simulation and experiment. The numerical results confirmed the effectiveness of the proposed method with higher accuracy, better anti-noise capability and shorter consuming time compared with other reconstruction methods. Besides, a physical experiment is performed for further testing. We measured the incident flow and vortex field using the proposed acoustic method. The results show that the tendency and data of the acoustical reconstructed flow field are consistent with the strip display and anemometer results. The relative errors of most measurement points are under 15% and our proposed acoustic method is effective in practical velocity field measurement.

Anisotropic Prodi–Serrin regularity criteria for the 3D Navier–Stokes equations involving the gradient of one velocity component

In this paper, we prove some optimal regularity criteria for the 3D incompressible Navier–Stokes equations involving the gradient of one velocity component in the framework of anisotropic Lebesgue spaces satisfying Prodi–Serrin condition. These results cover the previous ones established by Wolf (2015) and O (2023).

Robustness of electrocardiographic imaging in the presence of electrical noise

Abstract Funding Acknowledgements Type of funding sources: Other. Main funding source(s): Ministerio de Ciencia, Innovación y Universidades; Generalitat Valenciana Background Electrocardiographic Imaging (ECGI) allows evaluating the cardiac substrate non-invasively. However, the inverse solution is strongly dependent on the noise of the body surface signals. Zero-order Tikhonov regularization together with L-curve optimization, widely used in ECGI studies, allows minimizing the sources of noise to find an optimal solution. Objective Evaluate the effect of electrical and noise on the L-curve optimization in computer simulations of atrial activity and in the physiological interpretation of the inverse solution. Methods Five atrial simulations of regular rhythms (3 atrial flutter and 2 sinus rhythm) were used for computing the ECGI with five signal-to-noise ratios (SNR) (from 3dB to 40dB). ECGI was computed by zero-order Tikhonov regularization and L-curve optimization. The regularization parameter (λ) and the shape of the curve were analysed. Electrical noise was added to signals from an atrial flutter patient to study the effect on real ECGI signals. Local activation time (LAT) maps were obtained to evaluate the quality of the ECGI in front of noise. Correlation coefficient (CC) of LAT maps was computed between simulated electrograms and ECGIs. Results The level of noise was found to be correlated with the optimal regularization parameter (λ), (SNR 3dB, λ>10-6, SNR 40dB, λ<10-9). LAT maps for the optimal λ and higher were able to find relevant information about the simulation and patient independently on the SNR (CC3dB-EGM = 0.66±0.1, CC40dB-EGM = 0.76±0.1). Similar behaviour was observed in real atrial flutter signals (CC3dB-Ref = 0.83, CC10dB-Ref = 0.93). LAT maps from the atrial flutter were able to accurately locate the reentry circuit, as found in the electroanatomical map independently on the SNR. Conclusion LAT maps derived from ECGI are robust against the presence of noise. Although the reconstructed potentials are smoother with increasing levels of noise, LAT maps with and without noise show a good correlation thanks to an increased regularization. Figure 1. A. Local activation times (LAT) map of a reference electrogram (EGM) from a computer simulation of sinus rhythm in the atria. B. Electrocardiographic imaging maps (ECGI) obtained from the simulation A for different levels of electrical noise and regularization parameters. C. Example of LAT map obtained with ECGI and the endocardial map of a typical atrial flutter together with LAT maps with increased electrical noise but comparable results (CC > 0.83).

Optimal Poisson kernel regularity for elliptic operators with Hölder continuous coefficients in vanishing chord-arc domains

We show that if Ω is a vanishing chord-arc domain and L is a divergence-form elliptic operator with Hölder-continuous coefficient matrix, then log⁡kL∈VMO, where kL is the elliptic Poisson kernel for L in the domain Ω. This extends the previous work of Kenig and Toro in the case of the Laplacian.

Stability of strong exponential attractors for the Kirchhoff wave model with structural nonlinear damping

This paper investigates the stability of strong exponential attractors with respect to dissipative index θ∈[1/2,1) for the Kirchhoff wave model with structural nonlinear damping: utt−ϕ(‖∇u‖2)Δu+σ(‖∇u‖2)(−Δ)θut+f(u)=g(x). It proves that for each θ0∈[1/2,1), there exists a family of strong bi-space exponential attractors, which are also the standard exponential attractors of optimal regularity and are Hölder continuous at the point θ0 in the topology of the strong solution space provided that the nonlinearity f(u) is of the optimal subcritical growth. The method used here allows to overcome the difficulties in literatures before to get the Lipschitz continuity and quasi-stability of the evolution semigroup Sθ(t) to obtain the desired result.

Wheel-rail force identification for high-speed railway based on a modified weighted l1-norm regularization with optimal strain sensors

Wheel-rail force identification is an important inverse problem for structural health monitoring of high-speed railway train-track-bridge system. It is widely used for dynamic response reconstruction of bridges and early safety warning of dynamic reduction rate of wheel weight for the train. It can also serve as a reference for the calibration of wheel-rail force model. This paper proposes a wheel-rail force identification method for high-speed railway based on a modified weighted l1-norm regularization. Based on numerical simulations, it is found that variations of dynamic component of wheel-rail forces are very small. Therefore, the problem is transformed into the identification of three categories of static wheel weights and two categories of average dynamic components, which greatly simplifies the degree of freedom of the solution compared to traditional methods, thereby improving the solution accuracy and saving the number of sensors needed. The wheel-rail forces are decomposed into the product of a series of redundant basis functions and corresponding coefficients. The fast iterative shrinkage threshold algorithm (FISTA) and the Bayesian information criterion (BIC) are used to seek the solution of l1-norm regularization and to select the optimal regularization coefficients, respectively. Since the number of sensors and their locations have a significant impact on the accuracy of reconstructed wheel-rail forces, they are optimized using a genetic algorithm (GA), aiming to accurately reconstruct the wheel-rail forces with the minimum number of sensors at corresponding optimal positions. Numerical simulation of a 32 m box girder standard beam for high-speed railway shows that the reconstructed wheel-rail forces are in good agreement with the simulated results, which verifies the accuracy of the proposed method. In addition, the results of dynamic load test indirectly demonstrate the effectiveness of the proposed method.

An adaptive parameter selection strategy based on maximizing the probability of data for robust fluorescence molecular tomography reconstruction.

To alleviate the ill-posed of the inverse problem in fluorescent molecular tomography (FMT), many regularization methods based on L2 or L1 norm have been proposed. Whereas, the quality of regularization parameters affects the performance of the reconstruction algorithm. Some classical parameter selection strategies usually need initialization of parameter range and high computing costs, which is not universal in the practical application of FMT. In this paper, an universally applicable adaptive parameter selection method based on maximizing the probability of data (MPD) strategy was proposed. This strategy used maximum a posteriori (MAP) estimation and maximum likelihood (ML) estimation to establish a regularization parameters model. The stable optimal regularization parameters can be determined by multiple iterative estimates. Numerical simulations and in vivo experiments show that MPD strategy can obtain stable regularization parameters for both regularization algorithms based on L2 or L1 norm and achieve good reconstruction performance.

New regularization techniques for ill-conditioning problems and their applications: Choices of regularization parameters

This is a continued study of Li et al. (2011) for the regularization of ill-conditioning problems, where the minimal singular value σmin of discrete matrices is infinitesimal. To remove the effects of high frequency caused by the singular vector of σmin better, we combine the Tikhonov regularization (TR) with the truncated singular value decomposition (TSVD) (simply denoted by T-TR). New computational formulas of the traditional condition number (Cond) and the effective condition number (Cond_eff) are derived, and a brief error analysis is made. The regularization parameter λ is involved in the TR, and its better (or optimal) choices are essential in both theory and computation. For ill-conditioning problems, stability is more imperative than accuracy. The most important criterion is to reduce the Cond. Consider the linear algebraic equations Ax = b. The optimal regularization parameter for the TR is derived as λ=σmaxσmin, where σmax and σmin(>0) are the maximal and the minimal singular values of matrix A, respectively. The L-curve techniques emphasize the ill-conditioning of ‖xλ‖ (see Hansen (1998)). Note that the norm ‖xλ‖ is only part of the Cond in stability. In fact, by using regularization, the Cond may be greatly reduced, while the errors do not increase much. The second important criterion is for both stability and solution accuracy. The sensitivity index proposed in Zhang et al. (2021) indicates the severity of ill-conditioning via accuracy for comparing different numerical methods and techniques. The sensitivity index is effective not only to select the source nodes in the method of fundamental solutions (MFS), but also to select the regularization parameter in the TR. The regularization choices of the T-TR are similar to those of the TR because differences of the Cond and the errors between the TR and the T-TR are insignificant. Numerical experiments are reported by the MFS for Laplace’s equation, to support the new regularization techniques proposed. For data fitting, imaging processing, pattern recognition, and machine learning, the ‖xλ‖ is, however, more important than Cond for ill-conditioning. The L-curve techniques are used in wide applications, and discussed in Hansen (1998). The parameter choices are rather complicated if the crossover region is not small. In this paper, we apply the sensitivity index to L-curves, the optimal parameter λL−curve can be found by the minimal sensitivity index. The λL−curve may also be found by the lowest line that passes the origin. The new algorithms are simple and effective.

A data-driven model reduction method for parabolic inverse source problems and its convergence analysis

In this paper, we propose a data-driven model reduction method to solve parabolic inverse source problems with uncertain data efficiently. Our method consists of offline and online stages. In the offline stage, we explore the low-dimensional structures in the solution space of parabolic partial differential equations (PDEs) in the forward problems with a given class of source functions and construct a small number of proper orthogonal decomposition (POD) basis functions to achieve significant dimension reduction. Equipped with the POD basis functions, we can solve the forward problems extremely fast in the online stage. Thus, we develop a fast algorithm to solve the optimization problem in parabolic inverse source problems, which is referred to as the POD method. Moreover, we design an a posteriori algorithm to find the optimal regularization parameter in the optimization problem using the proposed POD method without knowing the noise level. Under a weak regularity assumption on the solution of the parabolic PDEs, we prove the convergence of the POD method in solving the forward parabolic PDEs. In addition, we obtain the error estimate of the POD method for parabolic inverse source problems. Finally, we present numerical examples to demonstrate the accuracy and efficiency of the proposed method. Numerical results show that the POD method provides considerable computational savings over the finite element method while maintaining the same accuracy.

Structural Changes in Nonlocal Denoising Models Arising Through Bi-Level Parameter Learning

We introduce a unified framework based on bi-level optimization schemes to deal with parameter learning in the context of image processing. The goal is to identify the optimal regularizer within a family depending on a parameter in a general topological space. Our focus lies on the situation with non-compact parameter domains, which is, for example, relevant when the commonly used box constraints are disposed of. To overcome this lack of compactness, we propose a natural extension of the upper-level functional to the closure of the parameter domain via Gamma-convergence, which captures possible structural changes in the reconstruction model at the edge of the domain. Under two main assumptions, namely, Mosco-convergence of the regularizers and uniqueness of minimizers of the lower-level problem, we prove that the extension coincides with the relaxation, thus admitting minimizers that relate to the parameter optimization problem of interest. We apply our abstract framework to investigate a quartet of practically relevant models in image denoising, all featuring nonlocality. The associated families of regularizers exhibit qualitatively different parameter dependence, describing a weight factor, an amount of nonlocality, an integrability exponent, and a fractional order, respectively. After the asymptotic analysis that determines the relaxation in each of the four settings, we finally establish theoretical conditions on the data that guarantee structural stability of the models and give examples of when stability is lost.

Derivative-free separable quadratic modeling and cubic regularization for unconstrained optimization

We present a derivative-free separable quadratic modeling and cubic regularization technique for solving smooth unconstrained minimization problems. The derivative-free approach is mainly concerned with building a quadratic model that could be generated by numerical interpolation or using a minimum Frobenius norm approach, when the number of points available does not allow to build a complete quadratic model. This model plays a key role to generate an approximated gradient vector and Hessian matrix of the objective function at every iteration. We add a specialized cubic regularization strategy to minimize the quadratic model at each iteration, that makes use of separability. We discuss convergence results, including worst case complexity, of the proposed schemes to first-order stationary points. Some preliminary numerical results are presented to illustrate the robustness of the specialized separable cubic algorithm.

WHU‐GRACE‐GPD01s: A Series of Constrained Monthly Gravity Field Solutions Derived From GRACE‐Based Geopotential Differences

AbstractTo suppress the correlated noise of Gravity Recovery and Climate Experiment (GRACE) spherical harmonic (SH) solutions, we developed a series of constrained monthly gravity field solutions named WHU‐GRACE‐GPD01s from August 2002 to July 2016 using GRACE‐based geopotential differences. The constrained solutions were estimated using Kaula regularization, and the optimal regularization parameters were adaptively determined from the GRACE data itself through variance component estimation. The performance of the constrained WHU‐GRACE‐GPD01s solutions was validated against the official SH solutions (GFZ, JPL, and CSR RL06) and mass concentration (mascon) solutions (CSR RL06M) at global and regional scales. The results demonstrate that mass changes derived from the constrained solutions and official SH solutions with DDK4 filtering are in good agreement, but the constrained solutions present weaker longitudinal stripes and have a lower noise level at regional scales (e.g., in the Middle Pacific Ocean and Sahara Desert). Furthermore, regional mass changes (e.g., major river basins, Greenland, and Antarctic ice sheet) inferred from the constrained solutions agree with the ones derived from the official SH solutions with DDK4 filtering. The constrained solutions also have a higher signal intensity and smaller spatial leakages as compared to the CSR RL06 SH solutions with spatial filtering (Gaussian filtering plus de‐striping). For the problematic months, the constrained solutions are more reliable than the official SH solutions using spatial filtering or DDK4 filtering, and are closer to CSR RL06M mascon solutions. These validations demonstrate that our constrained solutions are comparable to the official SH solutions and can be used without postprocessing.

Optimization of the regularization parameter in the Dual Annealing method used for the reconstruction of energy spectrum of electron beam generated by the AQURE mobile accelerator

IntroductionThe shape of the energy spectrum is an essential component of any electron beam Monte Carlo model. Due to specialized equipment and the long measurement time for the direct methods for determining the energy spectrum, attractive alternatives are backward spectrum reconstructions from the measured data. One such approach is solving the first-degree Fredholm integral equation with appropriate regularization. It makes it possible to calculate the depth distribution as the sum of the distributions from monoenergetic beams. This study aims to determine the optimal value of the regularization parameter for the problem of determining the spectrum of the electron beam produced by a mobile accelerator used during intraoperative radiotherapy. Material and methodsThe Geant4 package was used to generate the distributions of deep doses for monoenergetic beams for two models with different degrees of complexity, i.e. simple (theoretical) and full (for the mobile accelerator). The dose distributions for four different shapes of energy spectrum (for each model) were obtained similarly. They were established as the reference data for further calculations. The Dual Annealing optimization method was used to obtain the reconstructed spectrum. The multiple optimizations that differ by the regularization parameter (ranging from 0 to 1) were performed. For each reconstruction, similarity indicators of the energy spectrum and the dose distribution to the referenced data were calculated to determine the optimal regularization parameters. ResultsOptimal regularization parameters determined by similarity indicators for the spectrum and the dose distribution differ for geometry models considered in the study. The regularization parameter for the simple geometry ranged from 0.03 to 0.05, while for full geometry, they were from 0.05 to 0.06. The results for conventional linear accelerators found in the literature range from 0.5 to 1.1. ConclusionThe Dual Annealing optimization method can be effectively used to solve the Fredholm equation with Tikhonov regularization to reconstruct an electron beam's energy spectrum. The regularization parameter value depends on the beam-forming system. Its value for the mobile accelerator considered in the study ranges from 0.05 to 0.06, depending on the nominal beam energy value.

Long-time behavior of subcritical 2D vorticity Boussinesq equations

In this paper the subcritical Boussinesq system with fractional dissipation in T2 is considered. Our aim is to study the long-time behavior of solutions of Boussinesq system in its natural scale-invariant Sobolev space and prove the existence of a global attractor of optimal regularity. To this end we investigate the global well-posedness and global attractor for Boussinesq system in H2−α(T2)×H2−α(T2) via commutator estimates for nonlinear terms and a new energy estimate in Sobolev spaces to bootstrap the regularity, derived by means of nonlinear lower bounds on the fractional Laplacian. This estimate allows a sharp use of the subcritical nature of the L∞ bounds for this problem. Besides, we study the critical limit of the attractors and prove their stability and upper semicontinuity when α→1+.

A Fenchel dual gradient method enabling regularization for nonsmooth distributed optimization over time-varying networks

In this paper, we develop a regularized Fenchel dual gradient method (RFDGM), which allows nodes in a time-varying undirected network to find a common decision, in a fully distributed fashion, for minimizing the sum of their local objective functions subject to their local constraints. Different from most existing distributed optimization algorithms that also cope with time-varying networks, RFDGM is able to handle problems with general convex objective functions and distinct local constraints, and still has non-asymptotic convergence results. Specifically, under a standard network connectivity condition, we show that RFDGM is guaranteed to reach ϵ-accuracy in both optimality and feasibility within iterations. Such iteration complexity can be improved to if the local objective functions are strongly convex but not necessarily differentiable. Finally, simulation results demonstrate the competence of RFDGM in practice.

Loan classification using logistic regression

Objectives. The studied problem of loan classification is particularly important for financial institutions, which must efficiently allocate monetary assets between entities as part of the provision of financial services. Therefore, it is more important than ever for financial institutions to be able to identify reliable borrowers as accurately as possible. At the same time, machine learning is one of the tools for making such decisions. The aim of this work is to analyze the possibility of efficient use of logistic regression for solving the task of loan classification.Methods. Based on the logistic regression algorithm using historical data on loans issued, the following metrics are calculated: cost function, Accuracy, Precision, Recall и score. Polynomial regression and principal component analysis are used to determine the optimal set of input data for the being studied logistic regression algorithm.Results. The impact of data normalization on the final result is estimated, the optimal regularization parameter for solving this problem is determined, the impact of the balance of target values is assessed, the optimal boundary value for the logistic regression algorithm is calculated, the influence of increasing input indicators by means of filling in missing values and using polynomials of different degrees is considered and the existing set of input indicators is analyzed for redundancy.Conclusion. The research results confirm that the application of the logistic regression algorithm for solving loan classification problems is appropriate. The use of this algorithm allows to get quickly a working loan classification tool.

Uniformly convex neural networks and non-stationary iterated network Tikhonov (iNETT) method

We propose a non-stationary iterated network Tikhonov (iNETT) method for the solution of ill-posed inverse problems. The iNETT employs deep neural networks to build a data-driven regularizer, and it avoids the difficult task of estimating the optimal regularization parameter. To achieve the theoretical convergence of iNETT, we introduce uniformly convex neural networks to build the data-driven regularizer. Rigorous theories and detailed algorithms are proposed for the construction of convex and uniformly convex neural networks. In particular, given a general neural network architecture, we prescribe sufficient conditions to achieve a trained neural network which is component-wise convex or uniformly convex; moreover, we provide concrete examples of realizing convexity and uniform convexity in the modern U-net architecture. With the tools of convex and uniformly convex neural networks, the iNETT algorithm is developed and a rigorous convergence analysis is provided. Lastly, we show applications of the iNETT algorithm in 2D computerized tomography, where numerical examples illustrate the efficacy of the proposed algorithm.

Optimal Regularity for Elliptic Equations With Measurable Nonlinearities Under Nonstandard Growth

Abstract We are concerned with weak solutions of elliptic equations involving measurable nonlinearities with Orlicz growth to address what would be the weakest regularity condition on the associated nonlinearity for the Calderón–Zygmund theory. We prove that the gradient of weak solution is as integrable as the nonhomogeneous term under the assumption that the nonlinearity is only measurable in one of the variables while it has a small BMO assumption in the other variables. To this end, we develop a nonlinear Moser-type iteration argument for such a homogeneous reference problem with one variable–dependent nonlinearity under Orlicz growth to establish $W^{1,q}$–regularity for every $q>1$. Our results open a new path into the comprehensive understanding of the problem with nonstandard growth in the literature of optimal regularity theory in highly nonlinear elliptic and parabolic equations.

Traveling‐wave solutions for a nonlinear non‐Ohmic cardioelectrophysiological tissue conduction cable equation

In this article, families of solitary waves solutions of a general third‐order nonlinear non‐Ohmic cable equation in cardio‐electro‐physiology are obtained using the ‐expansion method. In this equation, the unknown function represents the trans‐membrane current, and the exact soliton‐like solutions are thoroughly derived using this analytic technique, and illustrated through surface and contour plots. To support the analytical findings presented in this work, the existence of solution and their optimal regularity is proved rigorously using Shauder's fixed‐point theorem.

IoT-Enabled WBAN and Machine Learning for Speech Emotion Recognition in Patients

Internet of things (IoT)-enabled wireless body area network (WBAN) is an emerging technology that combines medical devices, wireless devices, and non-medical devices for healthcare management applications. Speech emotion recognition (SER) is an active research field in the healthcare domain and machine learning. It is a technique that can be used to automatically identify speakers' emotions from their speech. However, the SER system, especially in the healthcare domain, is confronted with a few challenges. For example, low prediction accuracy, high computational complexity, delay in real-time prediction, and how to identify appropriate features from speech. Motivated by these research gaps, we proposed an emotion-aware IoT-enabled WBAN system within the healthcare framework where data processing and long-range data transmissions are performed by an edge AI system for real-time prediction of patients' speech emotions as well as to capture the changes in emotions before and after treatment. Additionally, we investigated the effectiveness of different machine learning and deep learning algorithms in terms of performance classification, feature extraction methods, and normalization methods. We developed a hybrid deep learning model, i.e., convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM), and a regularized CNN model. We combined the models with different optimization strategies and regularization techniques to improve the prediction accuracy, reduce generalization error, and reduce the computational complexity of the neural networks in terms of their computational time, power, and space. Different experiments were performed to check the efficiency and effectiveness of the proposed machine learning and deep learning algorithms. The proposed models are compared with a related existing model for evaluation and validation using standard performance metrics such as prediction accuracy, precision, recall, F1 score, confusion matrix, and the differences between the actual and predicted values. The experimental results proved that one of the proposed models outperformed the existing model with an accuracy of about 98%.