Regularization Parameter Research Articles

Network hub gene detection using the entire solution path information

Abstract Gene co-expression networks typically comprise modules and their associated hub genes, which are regulating numerous downstream interactions within the network. Methods for hub screening, as well as data-driven estimation of hub co-expression networks using graphical models, can serve as useful tools for identifying these hubs. Graphical model-based penalization methods typically have one or multiple regularization terms, each of which encourages some favorable characteristics (e.g., sparsity, hubs, power-law) to the estimated complex gene network. It is common practice to find a single optimal graphical model corresponding to a specific value of the regularization parameter(s). However, instead of doing this, one could aggregate information across several graphical models, all of which depend on the same data set, along the solution path in the hub gene detection process. We propose a novel method for detecting hub genes that utilizes the information available in the solution path. Our procedure is related to stability selection, but we replace resampling with a simple statistic. This procedure amalgamates information from each node of the data-driven graphical models into a single influence statistic, similar to Cook’s distance. We call this statistic the Mean Degree Squared Distance (MDSD). Our simulation and empirical studies demonstrate that the MDSD statistic maintains a good balance between false positive and true positive hubs. An R package MDSD is publicly available on GitHub under the General Public License https://github.com/markkukuismin/MDSD.

Predicting Prefecture-Level Well-Being Indicators in Japan Using Search Volumes in Internet Search Engines: Infodemiology Study

Background In recent years, the adoption of well-being indicators by national governments and international organizations has emerged as an important tool for evaluating state governance and societal progress. Traditionally, well-being has been gauged primarily through economic metrics such as gross domestic product, which fall short of capturing multifaceted well-being, including socioeconomic inequalities, life satisfaction, and health status. Current well-being indicators, including both subjective and objective measures, offer a broader evaluation but face challenges such as high survey costs and difficulties in evaluating at regional levels within countries. The emergence of web log data as an alternative source of well-being indicators offers the potential for more cost-effective, timely, and less biased assessments. Objective This study aimed to develop a model using internet search data to predict well-being indicators at the regional level in Japan, providing policy makers with a more accessible and cost-effective tool for assessing public well-being and making informed decisions. Methods This study used the Regional Well-Being Index (RWI) for Japan, which evaluates prefectural well-being across 47 prefectures for the years 2010, 2013, 2016, and 2019, as the outcome variable. The RWI includes a comprehensive approach integrating both subjective and objective indicators across 11 domains, including income, job, and life satisfaction. Predictor variables included z score–normalized relative search volume (RSV) data from Google Trends for words relevant to each domain. Unrelated words were excluded from the analysis to ensure relevance. The Elastic Net methodology was applied to predict RWI using RSVs, with α balancing ridge and lasso effects and λ regulating their strengths. The model was optimized by cross-validation, determining the best mix and strength of regularization parameters to minimize prediction error. Root mean square errors (RMSE) and coefficients of determination (R2) were used to assess the model’s predictive accuracy and fit. Results An analysis of Google Trends data yielded 275 words related to the RWI domains, and RSVs were collected for 211 words after filtering out irrelevant terms. The mean search frequencies for these words during 2010, 2013, 2016, and 2019 ranged from −1.587 to 3.902, with SDs between 3.025 and 0.053. The best Elastic Net model (α=0.1, λ=0.906, RMSE=1.290, and R2=0.904) was built using 2010-2016 training data and 2-13 variables per domain. Applied to 2019 test data, it yielded an RMSE of 2.328 and R2 of 0.665. Conclusions This study demonstrates the effectiveness of using internet search log data through the Elastic Net machine learning method to predict the RWI in Japanese prefectures with high accuracy, offering a rapid and cost-efficient alternative to traditional survey approaches. This study highlights the potential of this methodology to provide foundational data for evidence-based policy making aimed at enhancing well-being.

The Role of Uterocervical Angle on Preterm Birth Incidence

Background: Preterm birth (PB) is associated with significant morbidities in surviving infants. Accurate prediction of PB is essential for effective prevention and management. Continuous monitoring of cervical parameters has shown utility in several studies. Objectives: This study aimed to investigate the role of the anterior uterocervical angle (UCA) in predicting the incidence of PB. Methods: A prospective, descriptive-analytic study was conducted with 165 pregnant women referred to the prenatal care clinic at Yas Hospital Complex, Tehran University of Medical Sciences, Tehran, Iran, in 2021 - 2022. Participants were selected through convenience sampling. Cervical parameters, including UCA, length, and width, were measured via vaginal ultrasound during each trimester. All women were followed until delivery to determine PB incidence. Data were analyzed using descriptive and analytical statistical tests, including the chi-square or Fisher’s exact test and the independent samples t-test or Mann-Whitney U test. Results: Among the participants, 12 (7.3%) experienced preterm delivery. Cervical length was significantly shorter in women with PB during the second trimester (32.58 ± 4.77 mm vs. 34.68 ± 3.80 mm, P = 0.042) and third trimester (30.00 ± 4.74 mm vs. 32.77 ± 3.88 mm, P = 0.022). The mean UCA in women with preterm delivery was higher than in those with term delivery during both the second (90.58 ± 17.21° vs. 88.66 ± 16.76°) and third (100.25 ± 14.56° vs. 98.89 ± 17.78°) trimesters (P > 0.05). A UCA greater than 105° in the second trimester had a sensitivity of 16.7% and specificity of 81.5% for predicting PB. In the third trimester, a UCA greater than 105° showed a sensitivity of 58.3% and specificity of 60.7% for predicting PB. Conclusions: This study emphasizes the importance of regular cervical parameter measurements throughout pregnancy. A UCA greater than 105° in the third trimester appears to be a potential predictor of PB.

Refining penalized ridge regression: a novel method for optimizing the regularization parameter in genomic prediction

Abstract The popularity of genomic selection as an efficient and cost-effective approach to estimate breeding values continues to increase, due in part to the significant saving in phenotyping. Ridge regression is one of the most popular methods used for genomic prediction; however, its efficiency (in terms of prediction performance) depends on the appropriate tunning of the penalization parameter. In this paper we propose a novel, more efficient method to select the optimal penalization parameter for Ridge regression. We compared the proposed method with the conventional method to select the penalization parameter in 14 real data sets and we found that in 13 of these, the proposed method outperformed the conventional method and across data sets the gains in prediction accuracy in terms of Pearson's correlation was of 56.15%, with not-gains observed in terms of normalized mean square error. Finally, our results show evidence of the potential of the proposed method, and we encourage its adoption to improve the selection of candidate lines in the context of plant breeding.

Simultaneous multi-slice cardiac real-time MRI at 0.55T.

To determine the feasibility of simultaneous multi-slice (SMS) real-time MRI (RT-MRI) at 0.55T for the evaluation of cardiac function. Cardiac CINE MRI is routinely used to evaluate left-ventricular (LV) function. The standard is sequential multi-slice balanced SSFP (bSSFP) over a stack of short-axis slices using electrocardiogram (ECG) gating and breath-holds. SMS has been used in CINE imaging to reduce the number of breath-holds by a factor of 2-4 at 1.5T, 3T, and recently at 0.55T. This work aims to determine if SMS is similarly effective in the RT-MRI evaluation of cardiac function. We used an SMS bSSFP pulse sequence with golden-angle spirals at 0.55T with an SMS factor of three. We cover the LV with three acquisitions for SMS, and nine for single-band (SB). Imaging was performed on 9 healthy volunteers and 1 patient with myocardial fibrosis and sternal wires. A spatio-temporal constrained reconstruction is used, with regularization parameters selected by a board-certified cardiologist. Images were quantitatively analyzed with a normalized contrast and an Edge Sharpness (ES) score. There was a statistically significant 2-fold difference in contrast between SMS and SB and no significant difference in ES score. The contrast for SMS and SB were 13.38/29.05 at mid-diastole and 10.79/22.26 at end-systole; the ES scores for SMS and SB were 1.77/1.83 at mid-diastole and 1.50/1.72 at end-systole. SMS cardiac RT-MRI at 0.55T is feasible and provides sufficient blood-myocardium contrast to evaluate LV function in three slices simultaneously without any gating or periodic motion assumptions.

Improved Bacterial Foraging Optimization Algorithm with Machine Learning-Driven Short-Term Electricity Load Forecasting: A Case Study in Peninsular Malaysia

Accurate electricity demand forecasting is crucial for ensuring the sustainability and reliability of power systems. Least square support vector machines (LSSVM) are well suited to handle complex non-linear power load series. However, the less optimal regularization parameter and the Gaussian kernel function in the LSSVM model have contributed to flawed forecasting accuracy and random generalization ability. Thus, these parameters of LSSVM need to be chosen appropriately using intelligent optimization algorithms. This study proposes a new hybrid model based on the LSSVM optimized by the improved bacterial foraging optimization algorithm (IBFOA) for forecasting the short-term daily electricity load in Peninsular Malaysia. The IBFOA based on the sine cosine equation addresses the limitations of fixed chemotaxis constants in the original bacterial foraging optimization algorithm (BFOA), enhancing its exploration and exploitation capabilities. Finally, the load forecasting model based on LSSVM-IBFOA is constructed using mean absolute percentage error (MAPE) as the objective function. The comparative analysis demonstrates the model, achieving the highest determination coefficient (R2) of 0.9880 and significantly reducing the average MAPE value by 28.36%, 27.72%, and 5.47% compared to the deep neural network (DNN), LSSVM, and LSSVM-BFOA, respectively. Additionally, IBFOA exhibits faster convergence times compared to BFOA, highlighting the practicality of LSSVM-IBFOA for short-term load forecasting.

Prediction of building HVAC energy consumption based on least squares support vector machines

Air conditioning, as an essential appliance in daily life, has the function of ensuring comfortable room temperature, but it is also accompanied by a large amount of power consumption. Consequently, the study suggests an energy consumption prediction model based on improved genetic algorithm—least squares support vector machine—to accurately predict the energy consumption of building heating, ventilation, and air conditioning. This model uses the improved genetic algorithm for regularization parameter and kernel parameter optimization to prevent overfitting and underfitting issues. According to the testing results, the least squares support vector machine, an upgraded genetic algorithm, may accomplish convergence faster than other algorithms, taking only 0.2 milliseconds to finish. In addition, the average relative error of the improved genetic algorithm- least squares support vector machine did not exceed 0.6%. In the energy consumption prediction for the whole year of 2022, the average error of the improved genetic algorithm-least squares support vector machine was only 2.0 × 106 kWh, and the prediction accuracy could reach up to 97.2%. The above outcomes revealed that the energy consumption prediction model can accurately predict the air conditioning energy consumption, which provides a strong support for the control and optimization of the air conditioning system.

Refinement of the volcanic slope rating approach for determining slope stability in volcanic rocks of the Canary Islands

AbstractDue to their genesis, volcanic rocks present some singularities that make their geotechnical characteristics significantly different from other, more common types of rock masses, such as sedimentary and metamorphic rocks. The formation mechanisms of volcanic rocks are varied, rapid and generally high energy. These processes give this type of rock geotechnical behaviour and geomechanical properties that are completely different from those of nonvolcanic materials, due to their high heterogeneity and anisotropy. In volcanic rock slopes, applying widely used geomechanical classifications to assess the quality of any rock mass present several challenges due to their distinctive and singular characteristics, which detract from their validity. For this reason, the Center for Studies and Experimentation of Public Works (CEDEX) of the Ministry of Public Works, under the agreement signed with the Ministry of Public Works and Transport of the Government of the Canary Islands, carried out during 2017 and 2018 an exhaustive analysis and reviews of existing geomechanical classifications and their applications to volcanic terrains. They developed a new geomechanical classification called VSR (volcanic slope rating), based on data obtained from 85 slopes across the Canary Islands, which allowed for the estimation of the stability of volcanic rock slopes by evaluating seven geotechnical parameters. This article presents the work conducted to review and calibrate this tool, based on the study of 94 slopes of different lithotypes throughout the Canary Islands (lavas, pyroclasts, heterogeneous), culminating in the proposal of a new classification for pyroclastic massifs (with new parameters and different weightings) and the adjustment of the current classification for the other slopes (regarding the Surface Regularity parameter). Therefore, the VSR geomechanical classification was applied to assess the stability of these 94 slopes, revealing a strong fit for Types A and C (R2 > 0.97), while the tool overestimated stability in Type B slopes, prompting a new classification proposal for this category.

Flexible Krylov methods for group sparsity regularization

Abstract This paper introduces new solvers for efficiently computing solutions to large-scale inverse problems with group sparsity regularization, including both non-overlapping and overlapping groups. Group sparsity regularization refers to a type of structured sparsity regularization, where the goal is to impose additional structure in the regularization process by assigning variables to predefined groups that may represent graph or network structures. Special cases of group sparsity regularization include ℓ 1 and isotropic total variation regularization. In this work, we develop hybrid projection methods based on flexible Krylov subspaces, where we first recast the group sparsity regularization term as a sequence of 2-norm penalization terms using adaptive regularization matrices in an iterative reweighted norm fashion. Then we exploit flexible preconditioning techniques to efficiently incorporate the weight updates. The main advantages of these methods are that they are computationally efficient (leveraging the advantages of flexible methods), they are general (and therefore very easily adaptable to new regularization term choices), and they are able to select the regularization parameters automatically and adaptively (exploiting the advantages of hybrid methods). Extensions to multiple regularization terms and solution decomposition frameworks (e.g. for anomaly detection) are described, and a variety of numerical examples demonstrate both the efficiency and accuracy of the proposed approaches compared to existing solvers.

Investigating the Impact of the Regularization Parameter on EEG Resting-State Source Reconstruction and Functional Connectivity Using Real and Simulated Data

Accurate EEG source localization is crucial for mapping resting-state network dynamics and it plays a key role in estimating source-level functional connectivity. However, EEG source estimation techniques encounter numerous methodological challenges, with a key one being the selection of the regularization parameter in minimum norm estimation. This choice is particularly intricate because the optimal amount of regularization for EEG source estimation may not align with the requirements of EEG connectivityanalysis, highlighting a nuanced trade-off. In this study, we employed a methodological approach to determine the optimal regularization coefficient that yields the most effective reconstruction outcomes across all simulations involving varying signal-to-noise ratios for synthetic EEG signals. To this aim, we considered three resting state networks: the Motor Network, the Visual Network, and the Dorsal Attention Network. The performance was assessed using three metrics, at different regularization parameters: the Region Localization Error, source extension, and source fragmentation. The results were validated using real functional connectivity data. We show that the best estimate of functional connectivity is obtained using 10−2, while 10−1 has to be preferred when source localization only is at target.

Sensitivity-Aware Density Estimation in Multiple Dimensions.

We formulate an optimization problem to estimate probability densities in the context of multidimensional problems that are sampled with uneven probability. It considers detector sensitivity as an heterogeneous density and takes advantage of the computational speed and flexible boundary conditions offered by splines on a grid. We choose to regularize the Hessian of the spline via the nuclear norm to promote sparsity. As a result, the method is spatially adaptive and stable against the choice of the regularization parameter, which plays the role of the bandwidth. We test our computational pipeline on standard densities and provide software. We also present a new approach to PET rebinning as an application of our framework.

Dynamically regularized Lagrange multiplier schemes with energy dissipation for the incompressible Navier-Stokes equations

In this paper, we present efficient numerical schemes based on the Lagrange multiplier approach for the Navier-Stokes equations. By introducing a dynamic equation (involving the kinetic energy, the Lagrange multiplier, and a regularization parameter), we form a new system which incorporates the energy evolution process but is still equivalent to the original equations. Such nonlinear system is then discretized in time based on the backward differentiation formulas, resulting in a dynamically regularized Lagrange multiplier (DRLM) method. First- and second-order DRLM schemes are derived and shown to be unconditionally energy stable with respect to the original variables. The proposed schemes require only the solutions of two linear Stokes systems and a scalar quadratic equation at each time step. Moreover, with the introduction of the regularization parameter, the Lagrange multiplier can be uniquely determined from the quadratic equation, even with large time step sizes, without affecting accuracy and stability of the numerical solutions. Fully discrete energy stability is also proved with the Marker-and-Cell (MAC) discretization in space. Various numerical experiments in two and three dimensions verify the convergence and energy dissipation as well as demonstrate the accuracy and robustness of the proposed DRLM schemes.

On learning the optimal regularization parameter in inverse problems

Abstract Selecting the best regularization parameter in inverse problems is a classical and yet challenging problem. Recently, data-driven approaches have become popular to tackle this challenge. These approaches are appealing since they do require less a priori knowledge, but their theoretical analysis is limited. In this paper, we propose and study a statistical machine learning approach, based on empirical risk minimization. Our main contribution is a theoretical analysis, showing that, provided with enough data, this approach can reach sharp rates while being essentially adaptive to the noise and smoothness of the problem. Numerical simulations corroborate and illustrate the theoretical findings. Our results are a step towards grounding theoretically data-driven approaches to inverse problems.

A Particle Method for the Multispecies Landau Equation.

The multispecies Landau collision operator describes the two-particle, small scattering angle or grazing collisions in a plasma made up of different species of particles such as electrons and ions. Recently, a structure preserving deterministic particle method (Carrillo et al. in J. Comput. Phys. 7:100066, 2020) has been developed for the single species spatially homogeneous Landau equation. This method relies on a regularization of the Landau collision operator so that an approximate solution, which is a linear combination of Dirac delta distributions, is well-defined. Based on a weak form of the regularized Landau equation, the time dependent locations of the Dirac delta functions satisfy a system of ordinary differential equations. In this work, we extend this particle method to the multispecies case, and examine its conservation of mass, momentum, and energy, and decay of entropy properties. We show that the equilibrium distribution of the regularized multispecies Landau equation is a Maxwellian distribution, and state a critical condition on the regularization parameters that guarantees a species independent equilibrium temperature. A convergence study comparing an exact multispecies Bobylev-Krook-Wu (BKW) solution to the particle solution shows approximately 2nd order accuracy. Important physical properties such as conservation, decay of entropy, and equilibrium distribution of the particle method are demonstrated with several numerical examples.

Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization

AbstractThis paper deals with a second order dynamical system with a Tikhonov regularization term in connection to the minimization problem of a convex Fréchet differentiable function. The fact that beside the asymptotically vanishing damping we also consider an implicit Hessian driven damping in the dynamical system under study allows us, via straightforward explicit discretization, to obtain inertial algorithms of gradient type. We show that the value of the objective function in a generated trajectory converges rapidly to the global minimum of the objective function and depending the Tikhonov regularization parameter the generated trajectory converges weakly to a minimizer of the objective function or the generated trajectory converges strongly to the element of minimal norm from the $$\mathop {\text {argmin}}$$ argmin set of the objective function. We also obtain the fast convergence of the velocities towards zero and some integral estimates. Our analysis reveals that the Tikhonov regularization parameter and the damping parameters are strongly correlated, there is a setting of the parameters that separates the cases when weak convergence of the trajectories to a minimizer and strong convergence of the trajectories to the minimal norm minimizer can be obtained.

Characterization of transport optimizers via graphs and applications to Stackelberg–Cournot–Nash equilibria

AbstractWe introduce graphs associated to transport problems between discrete marginals, that allow to characterize the set of all optimizers given one primal optimizer. In particular, we establish that connectivity of those graphs is a necessary and sufficient condition for uniqueness of the dual optimizers. Moreover, we provide an algorithm that can efficiently compute the dual optimizer that is the limit, as the regularization parameter goes to zero, of the dual entropic optimizers. Our results find an application in a Stackelberg–Cournot–Nash game, for which we obtain existence and characterization of the equilibria.

A Fractional Tikhonov Regularization Method for Identifying a Time-Independent Source in the Fractional Rayleigh–Stokes Equation

The aim of this paper is to identify a time-independent source term in the Rayleigh–Stokes equation with a fractional derivative where additional data are considered at a fixed time point. This inverse problem is proved to be ill-posed in the sense of Hadamard. By using a fractional Tikhonov regularization method, we construct a regularized solution. Then, according to a priori and a posteriori regularization parameter selection rules, we prove the convergence estimates of the regularization method. Finally, we provide some numerical examples to prove the effectiveness of the proposed method.

A revisit of the development of viscoplastic flow in pipes and channels

This study revisits the development of viscoplastic flow in pipes and channels, focusing on the flow of a Bingham plastic. Using finite element simulations and the Papanastasiou regularisation, results are obtained across a range of Reynolds and Bingham numbers. The novel contributions of this work include: (a) investigating a definition of the development length based on wall shear stress, a critical parameter in numerous applications; (b) considering alternative definitions of the Reynolds number in an effort to collapse the development length curves onto a single master curve, independent of the Bingham number; (c) examining the patterns of yielded and unyielded regions within the flow domain; and (d) assessing the impact of the regularisation parameter on the accuracy of the results. The findings enhance the existing literature, providing a more comprehensive understanding of this classic flow problem.

Fan noise radial mode detection based on [formula omitted] regularization method

Radial mode detection has been widely employed in fan noise research. Many radial mode detection algorithms by conducting the convex L1 norm optimization have been proposed by taking advantage of the fact that the fan noise sound field is usually dominated by a limited set of radial modes. Nevertheless, L1 norm optimization has been shown to be suboptimal and cannot enforce further sparsity. A fan noise radial mode detection method based on L1/2 regularization algorithm is proposed in this paper to improve its robustness and dynamic range. This method employs the L1/2 regularization to directly solve the in-duct sound propagation model. Synthesized simulations indicates that the L1/2 regularization method can detect the correct radial mode order and amplitude under low signal-to-noise ratio. Moreover, the new method is insensitive to regularization parameter. An experimental system of radial rake is implemented to validate the new method. It is shown that the L-curve method, which do not require any prior information of the acoustic mode distribution or contaminating measurement noise, is an appropriate choice of the regularization parameter determination method for the L1/2 regularizer. It is demonstrated that the new L1/2 regularization method can enhance the mode sparsity, especially under low signal-to-noise ratio. Moreover, the resolution and dynamic range can be improved by the L1/2 regularization method, which makes it an effective and robust fan noise radial mode detection technique in practical applications.

High on sparsity: Interbin compensation of cardiac motion for improved assessment of left-ventricular function using 5D whole-heart MRI.

Cardiac magnetic resonance is the gold standard for evaluating left-ventricular ejection fraction (LVEF). Standard protocols, however, can be inefficient, facing challenges due to significant operator and patient involvement. Although the free-running framework (FRF) addresses these challenges, the potential of the extensive data it collects remains underutilized. Therefore, we propose to leverage the large amount of data collected by incorporating interbin cardiac motion compensation into FRF (FRF-MC) to improve both image quality and LVEF measurement accuracy, while reducing the sensitivity to user-defined regularization parameters. FRF-MC consists of several steps: data acquisition, self-gating signal extraction, deformation field estimations, and motion-resolved reconstruction with interbin cardiac motion compensation. FRF-MC was compared with the original 5D-FRF method using LVEF and several image-quality metrics. The cardiac regularization weight ( ) was optimized for both methods by maximizing image quality without compromising LVEF measurement accuracy. Evaluations were performed in numerical simulations and in 9 healthy participants. In vivo images were assessed by blinded expert reviewers and compared with reference standard 2D-cine images. Both in silico and in vivo results revealed that FRF-MC outperformed FRF in terms of image quality and LVEF accuracy. FRF-MC reduced temporal blurring, preserving detailed anatomy even at higher cardiac regularization weights, and led to more accurate LVEF measurements. Optimized produced accurate LVEF for both methods compared with the 2D-cine reference (FRF-MC: 0.59% [-7.2%, 6.0%], p = 0.47; FRF: 0.86% [-8.5%, 6.7%], p = 0.36), but FRF-MC resulted in superior image quality (FRF-MC: 2.89 ± 0.58, FRF: 2.11 ± 0.47; p < 10-3). Incorporating interbin cardiac motion compensation significantly improved image quality, supported higher cardiac regularization weights without compromising LVEF measurement accuracy, and reduced sensitivity to user-defined regularization parameters.