Regularization Parameter Research Articles

Sensitivity-Aware Density Estimation in Multiple Dimensions.

We formulate an optimization problem to estimate probability densities in the context of multidimensional problems that are sampled with uneven probability. It considers detector sensitivity as an heterogeneous density and takes advantage of the computational speed and flexible boundary conditions offered by splines on a grid. We choose to regularize the Hessian of the spline via the nuclear norm to promote sparsity. As a result, the method is spatially adaptive and stable against the choice of the regularization parameter, which plays the role of the bandwidth. We test our computational pipeline on standard densities and provide software. We also present a new approach to PET rebinning as an application of our framework.

On learning the optimal regularization parameter in inverse problems

Abstract Selecting the best regularization parameter in inverse problems is a classical and yet challenging problem. Recently, data-driven approaches have become popular to tackle this challenge. These approaches are appealing since they do require less a priori knowledge, but their theoretical analysis is limited. In this paper, we propose and study a statistical machine learning approach, based on empirical risk minimization. Our main contribution is a theoretical analysis, showing that, provided with enough data, this approach can reach sharp rates while being essentially adaptive to the noise and smoothness of the problem. Numerical simulations corroborate and illustrate the theoretical findings. Our results are a step towards grounding theoretically data-driven approaches to inverse problems.

A Particle Method for the Multispecies Landau Equation.

The multispecies Landau collision operator describes the two-particle, small scattering angle or grazing collisions in a plasma made up of different species of particles such as electrons and ions. Recently, a structure preserving deterministic particle method (Carrillo et al. in J. Comput. Phys. 7:100066, 2020) has been developed for the single species spatially homogeneous Landau equation. This method relies on a regularization of the Landau collision operator so that an approximate solution, which is a linear combination of Dirac delta distributions, is well-defined. Based on a weak form of the regularized Landau equation, the time dependent locations of the Dirac delta functions satisfy a system of ordinary differential equations. In this work, we extend this particle method to the multispecies case, and examine its conservation of mass, momentum, and energy, and decay of entropy properties. We show that the equilibrium distribution of the regularized multispecies Landau equation is a Maxwellian distribution, and state a critical condition on the regularization parameters that guarantees a species independent equilibrium temperature. A convergence study comparing an exact multispecies Bobylev-Krook-Wu (BKW) solution to the particle solution shows approximately 2nd order accuracy. Important physical properties such as conservation, decay of entropy, and equilibrium distribution of the particle method are demonstrated with several numerical examples.

Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization

AbstractThis paper deals with a second order dynamical system with a Tikhonov regularization term in connection to the minimization problem of a convex Fréchet differentiable function. The fact that beside the asymptotically vanishing damping we also consider an implicit Hessian driven damping in the dynamical system under study allows us, via straightforward explicit discretization, to obtain inertial algorithms of gradient type. We show that the value of the objective function in a generated trajectory converges rapidly to the global minimum of the objective function and depending the Tikhonov regularization parameter the generated trajectory converges weakly to a minimizer of the objective function or the generated trajectory converges strongly to the element of minimal norm from the $$\mathop {\text {argmin}}$$ argmin set of the objective function. We also obtain the fast convergence of the velocities towards zero and some integral estimates. Our analysis reveals that the Tikhonov regularization parameter and the damping parameters are strongly correlated, there is a setting of the parameters that separates the cases when weak convergence of the trajectories to a minimizer and strong convergence of the trajectories to the minimal norm minimizer can be obtained.

Flexible Krylov Methods for Group Sparsity Regularization

Abstract This paper introduces new solvers for efficiently computing solutions to large-scale inverse problems with group sparsity regularization, including both non- overlapping and overlapping groups. Group sparsity regularization refers to a type of structured sparsity regularization, where the goal is to impose additional structure in the regularization process by assigning variables to predefined groups that may represent graph or network structures. Special cases of group sparsity regularization include l1 and isotropic total variation regularization. In this work, we develop hybrid projection methods based on flexible Krylov subspaces, where we first recast the group sparsity regularization term as a sequence of 2-norm penalization terms using adaptive regularization matrices in an iterative reweighted norm fashion. Then, we exploit flexible preconditioning techniques to efficiently incorporate the weight updates. The main advantages of these methods are that they are computationally efficient (leveraging the advantages of flexible methods), they are general (and therefore very easily adaptable to new regularization term choices), and they are able to select the regularization parameters automatically and adaptively (exploiting the advantages of hybrid methods). Extensions to multiple regularization terms and solution decomposition frameworks (e.g., for anomaly detection) are described, and a variety of numerical examples demonstrate both the efficiency and accuracy of the proposed approaches compared to existing solvers.

Characterization of transport optimizers via graphs and applications to Stackelberg–Cournot–Nash equilibria

AbstractWe introduce graphs associated to transport problems between discrete marginals, that allow to characterize the set of all optimizers given one primal optimizer. In particular, we establish that connectivity of those graphs is a necessary and sufficient condition for uniqueness of the dual optimizers. Moreover, we provide an algorithm that can efficiently compute the dual optimizer that is the limit, as the regularization parameter goes to zero, of the dual entropic optimizers. Our results find an application in a Stackelberg–Cournot–Nash game, for which we obtain existence and characterization of the equilibria.

A Fractional Tikhonov Regularization Method for Identifying a Time-Independent Source in the Fractional Rayleigh–Stokes Equation

The aim of this paper is to identify a time-independent source term in the Rayleigh–Stokes equation with a fractional derivative where additional data are considered at a fixed time point. This inverse problem is proved to be ill-posed in the sense of Hadamard. By using a fractional Tikhonov regularization method, we construct a regularized solution. Then, according to a priori and a posteriori regularization parameter selection rules, we prove the convergence estimates of the regularization method. Finally, we provide some numerical examples to prove the effectiveness of the proposed method.

A revisit of the development of viscoplastic flow in pipes and channels

This study revisits the development of viscoplastic flow in pipes and channels, focusing on the flow of a Bingham plastic. Using finite element simulations and the Papanastasiou regularisation, results are obtained across a range of Reynolds and Bingham numbers. The novel contributions of this work include: (a) investigating a definition of the development length based on wall shear stress, a critical parameter in numerous applications; (b) considering alternative definitions of the Reynolds number in an effort to collapse the development length curves onto a single master curve, independent of the Bingham number; (c) examining the patterns of yielded and unyielded regions within the flow domain; and (d) assessing the impact of the regularisation parameter on the accuracy of the results. The findings enhance the existing literature, providing a more comprehensive understanding of this classic flow problem.

Fan noise radial mode detection based on [formula omitted] regularization method

Radial mode detection has been widely employed in fan noise research. Many radial mode detection algorithms by conducting the convex L1 norm optimization have been proposed by taking advantage of the fact that the fan noise sound field is usually dominated by a limited set of radial modes. Nevertheless, L1 norm optimization has been shown to be suboptimal and cannot enforce further sparsity. A fan noise radial mode detection method based on L1/2 regularization algorithm is proposed in this paper to improve its robustness and dynamic range. This method employs the L1/2 regularization to directly solve the in-duct sound propagation model. Synthesized simulations indicates that the L1/2 regularization method can detect the correct radial mode order and amplitude under low signal-to-noise ratio. Moreover, the new method is insensitive to regularization parameter. An experimental system of radial rake is implemented to validate the new method. It is shown that the L-curve method, which do not require any prior information of the acoustic mode distribution or contaminating measurement noise, is an appropriate choice of the regularization parameter determination method for the L1/2 regularizer. It is demonstrated that the new L1/2 regularization method can enhance the mode sparsity, especially under low signal-to-noise ratio. Moreover, the resolution and dynamic range can be improved by the L1/2 regularization method, which makes it an effective and robust fan noise radial mode detection technique in practical applications.

High on sparsity: Interbin compensation of cardiac motion for improved assessment of left-ventricular function using 5D whole-heart MRI.

Cardiac magnetic resonance is the gold standard for evaluating left-ventricular ejection fraction (LVEF). Standard protocols, however, can be inefficient, facing challenges due to significant operator and patient involvement. Although the free-running framework (FRF) addresses these challenges, the potential of the extensive data it collects remains underutilized. Therefore, we propose to leverage the large amount of data collected by incorporating interbin cardiac motion compensation into FRF (FRF-MC) to improve both image quality and LVEF measurement accuracy, while reducing the sensitivity to user-defined regularization parameters. FRF-MC consists of several steps: data acquisition, self-gating signal extraction, deformation field estimations, and motion-resolved reconstruction with interbin cardiac motion compensation. FRF-MC was compared with the original 5D-FRF method using LVEF and several image-quality metrics. The cardiac regularization weight ( ) was optimized for both methods by maximizing image quality without compromising LVEF measurement accuracy. Evaluations were performed in numerical simulations and in 9 healthy participants. In vivo images were assessed by blinded expert reviewers and compared with reference standard 2D-cine images. Both in silico and in vivo results revealed that FRF-MC outperformed FRF in terms of image quality and LVEF accuracy. FRF-MC reduced temporal blurring, preserving detailed anatomy even at higher cardiac regularization weights, and led to more accurate LVEF measurements. Optimized produced accurate LVEF for both methods compared with the 2D-cine reference (FRF-MC: 0.59% [-7.2%, 6.0%], p = 0.47; FRF: 0.86% [-8.5%, 6.7%], p = 0.36), but FRF-MC resulted in superior image quality (FRF-MC: 2.89 ± 0.58, FRF: 2.11 ± 0.47; p < 10-3). Incorporating interbin cardiac motion compensation significantly improved image quality, supported higher cardiac regularization weights without compromising LVEF measurement accuracy, and reduced sensitivity to user-defined regularization parameters.

Deep-Unfolded Tikhonov-Regularized Conjugate Gradient Algorithm for MIMO Detection

In addressing the multifaceted problem of multiple-input multiple-output (MIMO) detection in wireless communication systems, this work highlights the pressing need for enhanced detection reliability under variable channel conditions and MIMO antenna configurations. We propose a novel method that sets a new standard for deep unfolding in MIMO detection by integrating the iterative conjugate gradient method with Tikhonov regularization, combining the adaptability of modern deep learning techniques with the robustness of classical regularization. Unlike conventional techniques, our strategy treats the Tikhonov regularization parameter, as well as the step size and search direction coefficients of the conjugate gradient (CG) method, as trainable parameters within the deep learning framework. This enables dynamic adjustments based on varying channel conditions and MIMO antenna configurations. Detection performance is significantly improved by the proposed approach across a range of MIMO configurations and channel conditions, consistently achieving lower bit error rate (BER) and normalized minimum mean square error (NMSE) compared to well-known techniques like DetNet and CG. The proposed method has superior performance over CG and other model-based methods, especially with a small number of iterations. Consequently, the simulation results demonstrate the flexibility of the proposed approach, making it a viable choice for MIMO systems with a range of antenna configurations, modulation orders, and different channel conditions.

Sparse-group boosting: Unbiased group and variable selection

For grouped covariates, we propose a framework for boosting that allows for sparsity within and between groups. By using component-wise and group-wise gradient ridge boosting simultaneously with adjusted degrees of freedom or penalty parameters, a model with similar properties as the sparse-group lasso can be fitted through boosting. We show that within-group and between-group sparsity can be controlled by a mixing parameter, and discuss similarities and differences to the mixing parameter in the sparse-group lasso. Furthermore, we show under which conditions variable selection on a group or individual variable basis happens and provide selection bounds for the regularization parameters depending solely on the singular values of the design matrix in a boosting iteration of linear Ridge penalized boosting. In special cases, we characterize the selection chance of an individual variable vs. a group of variables through a generalized beta prime distribution. With simulations as well as two real datasets from ecological and organizational research data, we show the effectiveness and predictive competitiveness of this novel estimator. The results suggest that in the presence of grouped variables, sparse-group boosting is associated with less biased variable selection and higher predictability compared to component-wise or group-component-wise boosting.

Least squares regression under weak moment conditions

In this paper we consider the robust regression problem when the output variable may be heavy-tailed. In such scenarios, the traditional least squares regression paradigm is usually thought to be not a good choice as it lacks robustness to outliers. By projecting the outputs onto an adaptive interval, we show the regularized least squares regression can still work well when the conditional distribution satisfies a weak moment condition. Fast convergence rates in various norm are derived by tuning the projection scale parameter and regularization parameter in according with the sample size and the moment condition.

A fast impact force identification method via constructing a dynamic reduced dictionary

Impact force identification is an important aspect of structural health monitoring for online assessment of the integrity of engineering structures. Increasing the number of monitoring positions is undoubtedly beneficial to accurate force localization, however, it may increase sharply the dimension of the transfer matrix and thus lead to a large-scale inverse problem. In this case, traditional methods, such as the Iterative Shrinkage Threshold algorithm (IST) and the Two-Step Iterative Shrinkage Thresholding algorithm (TwIST), suffer from low solving efficiency. In this paper, the dynamic reduced dictionary is firstly introduced according to the sparsity of impact force in both time and spatial domains so as to lower significantly the dimension of the transfer matrix. Then, a new reduced unconstrained optimization problem is established via the dynamic reduced dictionary for impact force identification. By integrating the reduced unconstrained optimization problem into each iteration step of IST/TwIST, two novel algorithms, which are the reduced Newton iteration shrinkage threshold algorithm (rNIST) and the reduced Newton two-step iteration shrinkage threshold algorithm (rNTwIST), are proposed to solve quickly the large-scale impact force identification problem using ℓ1-norm sparse regularization. Meanwhile, the adaptive setting of regularization parameters strategy is used to construct force more fast and accurately. The proposed algorithms are numerically and experimentally demonstrated through identifying impact forces at 16-point and 81-point distributions of a plate using one sensor response. The results show that rNIST/rNTwIST can more quickly and accurately identify impact forces under numerous monitoring points compared with other ℓ1-norm regularization methods and nonconvex sparse regularization methods. Moreover, the more the number of monitoring points, the more obvious the efficiency improvement of the impact force identification by rNIST/rNTwIST method.

Regularized dynamic mode decomposition algorithm for time sequence predictions

Dynamic Mode Decomposition (DMD) aims at extracting intrinsic mechanisms in a time sequence via linear recurrence relation of its observables, thereby predicting later terms in the sequence. Stability is a major concern in DMD predictions. We adopt a regularized form and propose a Regularized DMD (ReDMD) algorithm to determine the regularization parameter. This leverages stability and accuracy. Numerical tests for Burgers’ equation demonstrate that ReDMD effectively stabilizes the DMD prediction while maintaining accuracy. Comparisons are made with the truncated DMD algorithm.

A theoretical framework for multi-physics modeling of poro-visco-hyperelasticity-induced time-dependent fracture of blood clots

Fracture resistance of blood clots plays a crucial role in physiological hemostasis and pathological thromboembolism. Although recent experimental and computational studies uncovered the poro-viscoelastic property of blood clots and its connection to the time-dependent deformation behavior, the effect of these physical processes on clot fracture and the underlying fracture mechanisms are not well understood. This work aims to formulate a thermodynamically consistent, multi-physics theoretical framework for describing the time-dependent deformation and fracture of blood clots. This theory concurrently couples fluid transport through porous fibrin networks, non-linear visco-hyperelastic deformation of the solid skeleton, solid-fluid interactions, mechanical degradation of tissues, gradient enhancement of energy, and protein unfolding of fibrin molecules. The constitutive relations of tissue constituents and the governing equation of fluid transport are derived within the framework of porous media theory by extending non-linear continuum thermodynamics at large strains. A physics-based, compressible network model is developed for the fibrin network of blood clots to describe its mechanical response. The kinetic equations of the internal variables, introduced for describing the non-linear viscoelastic deformation, non-local damage driving force and protein unfolding, are formulated according to the thermodynamics principles by incorporating a non-equilibrium energy of fibrin networks, a gradient-enhanced energy, and a stretch-induced energy of fibrin molecules, respectively, into the total free energy density function. An energy-based damage model is developed to predict the damage and fracture of blood clots, and an evolving regularization parameter is proposed to limit the damage zone bandwidth. The proposed model is implemented into finite element code by writing subroutines and is experimentally validated using single-edge cracked clot specimens with different constituents. The fracture of blood clots subject to different loading conditions is simulated, and the mechanisms of clot fracture are systematically analyzed. Computational results show that this model can accurately capture the experimentally measured deformation and fracture. The viscoelasticity and fluid transport play essential roles in the fracture of blood clots under physiological loading.

Three-dimensional particulate volume fraction reconstruction in the fluid based on the Lambert–Beer physics information neural network

The measurement of particle volume fraction in flow fields is of great significance in scientific research and engineering applications. As one of the particle detection techniques, the light extinction method is widely used in measuring nano-particles volume fraction in flow fields due to its simplicity and non-contact nature. In particular, in complex reactive flow fields like combustion reactions, the volume fraction of soot particulate and other particles can be accurately measured and reconstructed via the light extinction method that based on the Beer–Lambert law. This is crucial for exploring combustion phenomena, understanding their internal mechanisms, and reducing pollutant emissions. However, due to the enormous computational burden, current algebra reconstruction techniques struggle to achieve high-precision three-dimensional (3D) reconstruction of particles. Therefore, this paper originally proposes a 3D reconstruction algorithm based on the Beer–Lambert law physical information neural networks (LB-PINNs). By incorporating physical information as constraints into the particle reconstruction process, it is possible to achieve high-precision 3D reconstruction of particles in complex flow field environments with low computational cost. Meanwhile, to address the trade-off issues of reconstruction accuracy and smooth noise resistance in previous reconstruction algorithms, i.e., Tikhonov regularization, this paper employs dynamically adjusted regularization parameters in the LB-PINN algorithm. This approach ensures smooth noise-resistant processing while maintaining reconstruction accuracy, significantly reducing computation time and resource consumption. According to the experimental results, LB-PINNs demonstrate superior performance compared to previous reconstruction algorithms when reconstructing the soot volume fraction in complex reacting flow fields, i.e., combustion flame scenarios.

A Fast iterative shrinkage and threshold algorithm-based least-squares deconvolution technique with application to bearing fault signals

Bearings are commonly subjected to load, transmission, and impact during operation, resulting in bearing failure that ultimately leads to mechanical breakdown. The low energy level of fault-induced impulsive signals, however, renders it vulnerable to being masked by environmental noise and other disturbances present in the vibration signal. To cope with this issue, this paper investigates the application of the finite impulse response (FIR) filter for bearing fault detection. In this work, the estimation of the FIR is formulated as a least squares optimization problem, aiming to minimize the l 2 norm of the FIR and effectively suppress noise and interference sources. The solutions of the least squares optimization problem are obtained using the fast iterative shrinkage–thresholding (FISTA) algorithm. Furthermore, to better select the regularization parameter in our method, an improved gray wolf optimizer (I–GWO) is employed to conduct the parameter selection. Based on these improvements, the proposed FISTA–LSD technique enables rapid and precise detection of fault characteristics upon their occurrence, thereby mitigating the propagation of faults and preventing accidents from happening. Furthermore, with this automated selection process, engineers can save valuable time that would otherwise be spent on trial-and-error approaches. Through its rapid response capabilities and precise fault detection abilities enabled by the FISTA–LSD technique along with I–GWO integration for automatic parameter selection; this FISTA–LSD technique greatly enhances its practical value. The proposed technique is validated through the utilization of both numerical and experimental data.

Balance Sparse Decomposition Method with Nonconvex Regularization for Gearbox Fault Diagnosis

As an important part of rotating machinery, gearboxes often fail due to their complex working conditions and harsh working environment. Therefore, it is very necessary to effectively extract the fault features of the gearboxes. Gearbox fault signals usually contain multiple characteristic components and are accompanied by strong noise interference. Traditional sparse modeling methods are based on synthesis models, and there are few studies on analysis and balance models. In this paper, a balance nonconvex regularized sparse decomposition method is proposed, which based on a balance model and an arctangent nonconvex penalty function. The sparse dictionary is constructed by using Tunable Q-Factor Wavelet Transform (TQWT) that satisfies the tight frame condition, which can achieve efficient and fast solution. It is optimized and solved by alternating direction method of multipliers (ADMM) algorithm, and the non-convex regularized sparse decomposition algorithm of synthetic and analytical models are given. Through simulation experiments, the determination methods of regularization parameters and balance parameters are given, and compared with the L1 norm regularization sparse decomposition method under the three models. Simulation analysis and engineering experimental signal analysis verify the effectiveness and superiority of the proposed method.

Two-Phase Incompressible Flow with Dynamic Capillary Pressure in a Heterogeneous Porous Media

We prove the existence of weak solutions of a two-incompressible immiscible wetting and non-wetting fluids phase flow model in porous media with dynamic capillary pressure. This model is a coupled system which includes a nonlinear parabolic saturation equation and an elliptic pressure–velocity equation. In the regularized case, the existence and uniqueness of the weak solution are obtained. We let the regularization parameter be η→0 to prove the existence of weak solutions.