Lasso Approach Research Articles

Hyperparameter recommendation via automated meta-feature selection embedded with kernel group Lasso learning

Hyperparameter recommendation via meta-learning relies on the characterization and quality of meta-features. These meta-features provide critical information about the underlying datasets but are often selected manually based on the practitioner’s experience and preference, which can be inefficient and ineffective in many applications. In this paper, we propose a novel hyperparameter recommendation approach that integrates with a Lasso-based multivariate kernel group (KGLasso) model. The developed KGLasso model automatically identifies primary meta-features through model training. By selecting the most explanatory meta-features for a specific meta-learning task, the recommendation performance becomes much more effective. Our KGLasso model builds on a group-wise generalized multivariate Lasso approach. Within this framework, we establish a minimization algorithm using a corresponding auxiliary function, which is mathematically proven to be convergent and robust. As an application, we develop a hyperparameter recommendation system using our built KGLasso model on 120 UCI datasets for the well-known support vector machine (SVM) algorithm. This system efficiently provides competent hyperparameter recommendations for new tasks. Extensive experiments, including comparisons with popular meta-learning baselines and search algorithms, demonstrate the superiority of our proposed approach. Our results highlight the benefits of integrating model learning and feature selection to construct an automated meta-learner for hyperparameter recommendation in meta-learning.

Generalized Fused Lasso for Treatment Pooling in Network Meta-Analysis.

This work develops a generalized fused lasso (GFL) approach to fitting contrast-based network meta-analysis (NMA) models. The GFL method penalizes all pairwise differences between treatment effects, resulting in the pooling of treatments that are not sufficiently different. This approach offers an intriguing avenue for potentially mitigating biases in treatment rankings and reducing sparsity in networks. To fit contrast-based NMA models within the GFL framework, we formulate the models as generalized least squares problems, where the precision matrix depends on the standard error in the data, the estimated between-study heterogeneity and the correlation between contrasts in multi-arm studies. By utilizing a Cholesky decomposition of the precision matrix, we linearly transform the data vector and design matrix to frame NMA within the GFL framework. We demonstrate how to construct the GFL penalty such that every pairwise difference is penalized similarly. The model is straightforward to implement in R via the "genlasso" package, and runs instantaneously, contrary to other regularization approaches that are Bayesian. A two-step GFL-NMA approach is recommended to obtain measures of uncertainty associated with the (pooled) relative treatment effects. Two simulation studies confirm the GFL approach's ability to pool treatments that have the same (or similar) effects while also revealing when incorrect pooling may occur, and its potential benefits against alternative methods. The novel GFL-NMA method is successfully applied to a real-world dataset on diabetes where the standard NMA model was not favored compared to the best-fitting GFL-NMA model with AICc selection of the tuning parameter ( .

Bone Scintigraphy in Cardiac Transthyretin-Related Amyloidosis: A Novel Time-Saving Tool for Semiquantitative Analysis, with Good Potential for Predicting Different Etiologies

(1) Background: The visual and semiquantitative analysis of Technetium-99metastable-3,3-diphospono-1,2-propanodicarboxylic acid (99mTc-DPD) bone scintigraphy is promising for diagnosing cardiac amyloidosis but time-consuming. We validated a faster method, the geometric mean (GM) method with a semi-automated workflow, for heart–whole body (WB) ratio (H/WBr), heart retention (Hr), and WB retention (WBr) calculations compared to the classic method (CM) established in the literature. The capability of semiquantitative scintigraphy indexes to differentiate the etiology in transthyretin-related cardiac amyloidosis (cATTR) patients was investigated. (2) Methods: H/WBr, Hr, and WBr were calculated by extracting counts for WB, kidneys, bladder, and heart on early and late planar image scans and applying background, scan-time, and decay corrections, using CM and GM both on a referring workstation and on a semi-automated workflow in external software. The comparison between CM and GM was assessed with Pearson’s correlation, Lin’s Concordance Correlation Coefficient (CCC), and Bland–Altman analysis. H/WBr, Hr, and WBr and several clinical variables were used to implement LASSO, Random Forest (RF), and Neural Network (NN) models to predict mutated and wild-type ATTR etiologies. ROC curves and AUC were calculated. (3) Results: Hr, WBr, and H/WBr using CM and GM were highly correlated. Bland–Altman analysis between CM and GM showed biases of 0.12% [CI:0.04%;0.19%] for H/WBr, 0.07% [CI: 0.01%; 0.13%] for Hr, and -0.50% [CI: −1.22%; 0.22%] for WBr. LASSO and NN models had good performance in predicting etiologies with AUC values of 87.3% and 73.6%, respectively. The RF model showed a poorer AUC of 55.8%. (4) Conclusions: The GM in the assisted workflow was validated against the CM. LASSO and NN approaches allowed a good prediction performance to be obtained for patient etiology.

Predictors for psychosocial consequences of screening for liver diseases in Germany: a LASSO approach

Abstract Background Medical screening is extensively used across various disciplines for early disease detection and prevention of severe progression aiming to reduce disease-specific burden, improve quality of life and promote health. However, screening is debated due to unclear evidence of its overall benefits from a public health perspective. Screening is associated with improved health behavior and cost-effectiveness. However, studies identified adverse effects of screening like psychosocial burden. Most results originate from cancer screening and it is unclear whether they are transferable to other contexts. Within a research program, a systematic screening procedure to detect patients with liver cirrhosis at early stage in an asymptomatic general population was evaluated. In this study, we aimed to identify patient characteristics that predict and explain variance in psychosocial outcomes of screened patients. Methods This study analyzed data from 523 participants who underwent a systematic liver disease screening in Germany from Jan 2018 to Feb 2021. We used bootstrapped LASSO regression with 10-fold validation to evaluate the influence of various predictors on the outcomes measured by the Psychological Consequences of Screening Questionnaire (PCQ). The constructs measured by the PCQ are social, physical and emotional dysfunction associated with screening. Results We identified lower severity of comorbidities, higher subjective social status, stronger social support, older age, better critical health communication skills and higher health literacy as relevant predictors of lower psychosocial dysfunction, indicating their protective role in preventing psychosocial burden of screening. Conclusions From a public health perspective, medical screening offers should take into account patients’ individual context regarding their subjective social status, health impairments, personal social networks and health literacy skills to reduce post-screening psychosocial burden. Key messages • Social support, age, health literacy and higher subjective social status were positively associated with lower psychosocial consequences of liver screening. • The findings indicate that liver screening should be carried out, taking into account the individual’s situation and resources available.

Utilizing an integrated bioinformatics and machine learning approach to uncover biomarkers linking ulcerative colitis to purine metabolism-related genes

BackgroundUlcerative colitis (UC) is an increasing incidence of inflammatory disorder in the colon mucosa. One of the current research focuses is the alteration of metabolic networks in UC. One of the important aspects of this metabolic shift is the expression of purine metabolism genes (PMGs) vital for nucleic acid synthesis. Nevertheless, the precise function of PMGs in the pathophysiology of UC is not yet fully known. MethodsTo this end, this study used state-of-the-art bioinformatics tools and approaches to discover and confirm the PMGs involved in UC. All the 114 candidate PMGs were compared for their expression levels. GSEA and GSVA were applied to define the functional and pathway implications of these PMGs. Lasso regression and SVM-RFE approaches were used for the identification of hub genes and to assess the diagnostic potential of eight PMGs in UC classification. The relationship between these critical PMGs and clinical features was also systematically evaluated as well. The expression levels of these eight PMGs were validated using datasets GSE206285 and GSE179285. ResultsUsing bioinformatics and machine learning, this work seeks to establish the involvement of PMGs in UC. From the LASSO and SVM models, 114 DE PMGs were selected and investigated to build a stable predictive model. Based on these studies, the following genes: IMPDH1, GUK1, POLE3, ADCY3, ADCY4, PDE6B, PNPT1 and PDE4D were suggested as potential biomarkers of UC. Gene ontology enrichment analysis revealed that these genes are implicated in the biological processes of particular relevance to immune and inflammatory responses. The study also provided a lot of information on the interaction between immune cells and PMGs indicating that these genes may control some immune-related pathways in UC. Moreover, drug-gene interaction analysis presents potential therapeutic opportunities for potential drug targets which were further confirmed through molecular docking. Mendelian randomization analysis revealed that ADCY4 and PDAZN are involved in PMG-related processes, thus opening new possibilities for treatment. ConclusionsThis work reveals eight PMGs closely related to UC and provides new perspectives on possible markers of this inflammatory disease. These findings not only increase the understanding of the pathogenesis of UC but also offer potential for improving the surveillance of disease and its progression.

Identifying individuals at risk for surgical supravalvar aortic stenosis by polygenic risk score with graded phenotyping.

In a previous pathway-based, extreme phenotype study, we identified 1064 variants associated with supravalvar aortic stenosis (SVAS) severity in people with Williams syndrome (WS) and either no SVAS or surgical SVAS. Here, we use those variants to develop and test polygenic risk scores (PRS). We used the clumping and thresholding (CT) approach on the full 1064 variants and a 427-variant subset that was part of 13 biologically relevant pathways identified in the previous study. We also used a lasso approach on the full set. We were able to achieve an area under the curve (AUC) of >0.99 for the two CT PRS methods, using only 622 and 320 variants respectively when 2/3 of the initial 217 participants data were used for training and 1/3 for testing. The lasso performed less well. We then evaluated the performance of those PRS variant sets on an additional group of 138 patients with WS with intermediate severity SVAS and found a misclassification rate of <10% between the surgical and intermediate groups, suggesting potential for clinical utility of the score.

Model-Based Prediction for Small Domains Using Covariates: A Comparison of Four Methods

Abstract We consider methods for model-based small area estimation when the number of areas with sampled data is a small fraction of the total areas for which estimates are required. Abundant auxiliary information is available from the survey for all the sampled areas. Further, through an external source, there is information for all areas. The goal is to use auxiliary variables to predict the outcome of interest for all areas. We compare areal-level random forests and LASSO approaches to a frequentist forward variable selection approach and a Bayesian shrinkage method using a horseshoe prior. Further, to measure the uncertainty of estimates obtained from random forests and the LASSO, we propose a modification of the split conformal procedure that relaxes the assumption of exchangeable data. We show that the proposed method yields intervals with the correct coverage rate and this is confirmed through a simulation study. This work is motivated by Ghanaian data available from the sixth Ghana Living Standards Survey (GLSS) and the 2010 Population and Housing Census, in the Greater Accra Metropolitan Area (GAMA) region, which comprises eight districts that are further divided into enumeration areas (EAs). We estimate the areal mean household log consumption using both datasets. The outcome variable is measured only in the GLSS for 3 percent of all the EAs (136 out of 5019) and 174 potential covariates are available in both datasets. In the application, among the four modeling methods considered, the Bayesian shrinkage performed the best in terms of bias, mean squared error (MSE), and prediction interval coverages and scores, as assessed through a cross-validation study. We find substantial between-area variation with the estimated log consumption showing a 1.3-fold variation across the GAMA region. The western areas are the poorest while the Accra Metropolitan Area district has the richest areas.

The Effect of Multicollinearity on Feature Selection

Objectives: To provide a new LASSO-based feature selection technique that aids in selecting important variables for predicting the response variables in case of multicollinearity. Methods: LASSO is a type of regression method employed to select important covariates for predicting a dependent variable. The traditional LASSO method uses the conventional Ordinary Least Square (OLS) method for this purpose. The Use of the OLS based LASSO approach gives unreliable results if the data deviates from normality. Thus, this study recommends using, a Redescending M-estimator-based LASSO approach. The efficacy of this new method is checked against the ordinary LASSO method using a real dataset and also a simulation study with various levels of sample size (N=100,200,1000), different numbers of predictors (p=10,15,20), and varying degrees of correlation (ρ = 0.96, 0.98, 0.999). Findings: The usual OLS-based LASSO finds it difficult to select important variables when the independent variables are correlated. The Redescending M-estimator-based LASSO addresses at tackling the pitfalls faced by Conventional LASSO methodology. Among other things, the proposed method is far better than the old-fashioned LASSO since it helps to pick out significant factors more effectively, particularly in the presence of multicollinearity. Novelty: The conventional OLS-based LASSO approach selects a greater number of non-significant variables in the presence of multicollinearity. The proposed Redescending M-estimator-based LASSO approach selects the important variables in the presence of multicollinearity. Keywords: Feature Selection, LASSO, MDAE, VIF, Variable Selection

Inference of human contact networks based on epidemic data

A key challenge in epidemic modeling is the lack of adequate data on population interactions (such as traffic flow or mobility patterns) which result in the spread of infectious diseases. Knowledge of social contact patterns is crucial for public health professionals to devise effective non-pharmaceutical interventions to control epidemics. This paper focuses on inferring social contact rates from reported infection counts during the spread of an infectious disease, addressing the increased difficulty that arises when dealing with “sparse” contact networks where only a small subset of the edges have non-zero weights. Specifically, a new geographically constrained lasso approach for network reconstruction for non-homogeneous mixing Susceptible-Infected-Removed (SIR) disease spread models is presented. The new network reconstruction method can explicitly account for the spatial proximity of network nodes in estimating the disease transmission rates and predicting the future evolution of the epidemic dynamics. Extensive numerical experiments are presented to show the proposed method outperforms existing approaches in terms of accuracy of contact identification under various graph topologies. A case study based on real data from the COVID-19 pandemic is presented to demonstrate the application of the approach for inferring contact structures and a counterfactual scenario analysis to assess effectiveness of containment strategies.

Spatio-temporal clustering using generalized lasso to identify the spread of Covid-19 in Indonesia according to provincial flight route-based connections

Indonesia is a country that has been greatly affected by the Covid-19 pandemic. In the almost three years that the pandemic has been going on, the spread of Covid-19 has penetrated almost all regions of Indonesia. One of the causes of the rapid spread of Covid-19 confirmed cases in Indonesia is the existence of domestic flights between regions within the archipelago. This research is aimed to identify patterns of Covid-19 transmission cases between provinces in Indonesia using spatio-temporal clustering. The method used a generalized lasso approach based on flight connections and proximity between provinces. The results suggested that clustering based on flight connections between provinces obtained more reasonable results, namely that there were three clusters of provinces formed with different patterns of spread of Covid-19 cases over time.

Distributed learning for kernel mode–based regression

AbstractWe propose a parametric kernel mode–based regression built on the mode value, which provides robust and efficient estimators for datasets containing outliers or heavy‐tailed distributions. To address the challenges posed by massive datasets, we integrate this regression method with distributed statistical learning techniques, which greatly reduces the required amount of primary memory and simultaneously accommodates heterogeneity in the estimation process. By approximating the local kernel objective function with a least squares format, we are able to preserve compact statistics for each worker machine, facilitating the reconstruction of estimates for the entire dataset with minimal asymptotic approximation error. Additionally, we explore shrinkage estimation through local quadratic approximation, showcasing that the resulting estimator possesses the oracle property through an adaptive LASSO approach. The finite‐sample performance of the developed method is illustrated using simulations and real data analysis.

Tail dependence, risk contagion and industry systemic risk: Based on method of Lasso-Expectile

Using third-level industry data of CSI 300, we introduce the Lasso approach into the high-dimensional CARE to model the tail dependence network among the financial sector and real sectors and measure the industry systemic risk. We find that the realized systemic risks between industries are affected by individual risk exposures and marginal systemic risk contributions. Real estate’s marginal systemic risk contribution is the highest in real sectors, and the financial sector’s integrated finance industry shows vital marginal systemic risk contribution and network centrality. Auto parts, construction materials, metals and mining, construction engineering, electrical equipment, and means of transportation in real sectors strongly correlate with other industries. The capital markets industry has a tail dependence with 20 different industries.

A non-linear Lasso and explainable LSTM approach for estimating tail risk interconnectedness

ABSTRACT Tail risk inter-connectivity is a significant aspect and a risk indicator that should be focused on. Many of the previous works have shown potential non-linearity in tail risk contagion. With the recent advancements in deep learning, Long-Short Term Memory (LSTM) networks have played an important role in sequential data prediction. We experiment with LASSO-based neural networks and interpretative LSTM model along with other machine learning approaches for investigating tail risk interconnectedness among the public banks of Japan. We also investigate the risk reception from large overseas banks in United States finding that medium-sized banks are more likely to receive international risks. Our studies show that LSTM-based model is an excellent fit for the scenario and total connectedness goes up during an economic crisis. The banks having larger market capitalization are more prone to emission and reception of tail risks. This is accompanied by exhibiting the impact of some major economic distresses on Japanese banking system. These results provide important information to regulators and policy makers.

Effective monitoring of Noyyal River surface water quality using remote sensing and machine learning and GIS techniques

This study utilizes Geographic Information System (GIS) and remote sensing techniques to predict water quality metrics in the Noyyal River. Satellite data is employed to construct statistical models, with preprocessing involving adjustments from Landsat 8 images. The hybrid LASSO model demonstrates superior performance in predicting water quality parameters, supported by key performance metrics such as RMSE, R-squared, and ANOVA results. The study focuses on assessing the suitability of the hybrid LASSO model for predicting the Water Quality Index (WQI) in the Noyyal region, highlighting its ability to handle high-dimensional data and provide interpretable results. Prediction models employing the LASSO approach yield promising results, with R-squared values exceeding 0.87 for temperature and pH. Incorporating spectral indices significantly enhances model performance, with an average R-squared of 0.8. These models offer cost-effective options for monitoring water quality, revealing poor conditions in the Noyyal River and projecting deterioration for 2023. WQI ratings indicate poor conditions, particularly during July and August 2022 and April 2023, providing valuable insights for local pollution regulation enforcement.

Preoperative Prediction of Muscle Invasiveness in Bladder Cancer: The Role of 3D Volumetric Radiomics Using Diffusion-Weighted MRI, the VI-RADS Score, or a Combination of Both.

Bladder cancer treatment decisions hinge on detecting muscle invasion. The 2018 "Vesical Imaging Reporting and Data System" (VI-RADS) standardizes multiparametric MRI (mp-MRI) use. Radiomics, an analysis framework, provides more insightful information than conventional methods. To determine how well MIBC (Muscle Invasive Bladder Cancer) and NMIBC (Non-Muscle Invasive Bladder Cancer) can be distinguished using mp-MRI radiomics features. We conducted a study with 73 bladder cancer patients diagnosed pathologically, who underwent preoperative mp-MRI from January 2020 to July 2022. Utilizing 3D Slicer (version 4.8.1) and Pyradiomics, we manually extracted radiomic features from apparent diffusion coefficient (ADC) maps created from diffusion-weighted imaging. The LASSO approach identified optimal features, and we addressed sample imbalance using SMOTE. We developed a classification model using textural features alone or combined with VI-RADS, employing a random forest classifier with 10-fold cross-validation. Diagnostic performance was assessed using the area under the ROC curve analysis. Among 73 patients (63 men, 10 women; median age: 63 years), 41 had muscle-invasive and 32 had superficial bladder cancer. Muscle invasion was observed in 25 of 41 patients with VI-RADS 4 and 5 scores and 12 of 32 patients with VI-RADS 1, 2, and 3 scores (accuracy: 77.5%, sensitivity: 67.7%, specificity: 88.8%). The combined VI-RADS score and radiomics model (AUC = 0.92±0.12) outperformed the single radiomics model using ADC MRI (AUC = 0.83±0.22 with 10-fold cross-validation) in this dataset. Before undergoing surgery, bladder cancer invasion in muscle might potentially be predicted using a radiomics signature based on mp-MRI.

Double debiased transfer learning for adaptive Huber regression

AbstractThrough exploiting information from the source data to improve the fit performance on the target data, transfer learning estimations for high‐dimensional linear regression models have drawn much attention recently, but few studies focus on statistical inference and robust learning in the presence of heavy‐tailed/asymmetric errors. Using adaptive Huber regression (AHR) to achieve the bias and robustness tradeoff, in this paper we propose a robust transfer learning algorithm with high‐dimensional covariates, then construct valid confidence intervals and hypothesis tests based on the debiased lasso approach. When the transferable sources are known, a two‐step ‐penalized transfer AHR estimator is firstly proposed and the error bounds are established. To correct the biases caused by the lasso penalty, a unified debiasing framework based on the decorrelated score equations is considered to establish asymptotic normality of the debiased lasso transfer AHR estimator. Confidence intervals and hypothesis tests for each component can be constructed. When the transferable sources are unknown, a data‐driven source detection algorithm is proposed with theoretical guarantee. Numerical studies verify the performance of our proposed estimator and confidence intervals, and an application to Genotype‐Tissue Expression data is also presented.

Feature selection for high-dimensional neural network potentials with the adaptive group lasso

Neural network potentials are a powerful tool for atomistic simulations, allowing to accurately reproduce ab initio potential energy surfaces with computational performance approaching classical force fields. A central component of such potentials is the transformation of atomic positions into a set of atomic features in a most efficient and informative way. In this work, a feature selection method is introduced for high dimensional neural network potentials, based on the adaptive group lasso (AGL) approach. It is shown that the use of an embedded method, taking into account the interplay between features and their action in the estimator, is necessary to optimize the number of features. The method’s efficiency is tested on three different monoatomic systems, including Lennard–Jones as a simple test case, Aluminium as a system characterized by predominantly radial interactions, and Boron as representative of a system with strongly directional components in the interactions. The AGL is compared with unsupervised filter methods and found to perform consistently better in reducing the number of features needed to reproduce the reference simulation data at a similar level of accuracy as the starting feature set. In particular, our results show the importance of taking into account model predictions in feature selection for interatomic potentials.

Unveiling key determinants of higher education expenditure in Vietnam: an adaptive LASSO approach in Tobit regression analysis

PurposeThis paper investigates the key determinants affecting household spending for university degree in Vietnam. This paper serves as empirical evidence for policymakers to select appropriate factors to estimate financial needs for university students in Vietnam.Design/methodology/approachWe employ an innovative variable selection approach with adaptive LASSO for Tobit Regression Model.FindingsThe results suggest income, region and ethnicity play significant roles in determining higher education expenditure at Vietnamese households. Gender-related indicators (such as gender of household head, student’s gender), distance to school, occupations, etc. are empirically insignificant.Originality/valueThis study proposes a data-driven method by interfering regulation and Tobit regression to understand the divergence in higher education spending. Especially, adaptive LASSO was first introduced to identify key determinants of higher education expenditure at household level in Vietnam. It hopes to tackle over-fitting problems of traditional OLS regression in previous literature.

ESG factors and the cross-section of expected stock returns: A LASSO-based approach

We analyze high-dimensional factor data in the U.S. market to examine whether the ESG (environmental, social, and governance) factors help explain the cross-section of expected stock returns. To avoid omitted variable biases, we use the double-selection LASSO approach with more than 160 risk factors. ESG and environmental factors potentially explain the cross-section of stock returns and can also affect investors’ marginal utility.

Tweedie regularization model and application

This article proposes a new regular model called the Tweedie regular regression model. The Tweedie distributions are parameterized by their mean, dispersion and power parameter ξ. For each ξ, we obtain a specific regular model. Tweedie Regression model includes several distributions. In particular, for ξ = 0, we obtain the Gaussian case; the estimation of the parameters through the classical point of view has been done using Least Absolute Shrinkage and Selection Operator (Lasso), Ridge and Elastic methods. Therefore, the proposed model is appropriate for modeling equiover and underdispersed data. The parameters estimation, through the classical point of view, has been performed using the methods of Lasso, Ridge and Elastic approaches. The Tweedie regular regression model will be calibrated by choosing one of the Lasso, Ridge and Elastic approaches. To evaluate our suggested method, a detailed comparison study through a simulation and real data was conducted to verify the proposed models. It was carried out to examine the fitness of the estimated parameters. Compared with counterparts, the results demonstrate the superiority of models Ridge Tweedie Regression (RTR), Lasso Tweedie Regression (LTR), and Elastic Tweedie Regression (ETR) using MAE (Mean absolute error), RMSE (Root Mean Squared Error) and R-squared (Coefficient of determination)