Adaptive Lasso Research Articles

Quantile adaptive lasso: variable selection for quantile treatment effect estimation

Quantile treatment effects (QTEs) characterize different causal effects of treatment on outcome distribution. Propensity score (PS) methods are commonly employed for estimating QTEs in non-randomized studies. Previous studies have shown that insufficient and unnecessary adjustment for covariates in PS models leads to bias and efficiency loss of treatment effects. Confounders may vary across different quantiles of outcome distribution. To allow for the selection of covariates related to outcome at interesting quantiles corresponding to QTE, we proposed quantile adaptive lasso (QAL) to select covariates that can provide unbiased and efficient QTE estimation. QAL utilized linear quantile regression models to construct penalty weights. Theoretical analyses showed that QAL enjoys several asymptotic properties. Simulations showed the superiority of QAL over existing methods in variable selection and QTE estimation. We applied QAL to datasets from the China Health and Retirement Longitudinal Study to explore the impact of smoking on the severity of depression.

Interactive association of metals and Life’s Essential 8 with mortality in U.S. adults: a prospective cohort study from the NHANES dataset

BackgroundLife’s Essential 8 (LE8) is a novel assessment of cardiovascular health (CVH) by evaluating lifestyle, and reports of the associations between LE8 and urinary metals on mortality have been very limited. This study aimed to conduct a prospective cohort study and investigate the combined effects of metals and LE8 on mortality in U.S. adults.MethodsThis study enrolled participants with complete information on urinary metals, LE8, mortality status, and confounders from the National Health and Nutrition Examination Survey (2005–2018). The Cox regression model, adaptive lasso penalized regression, and restricted cubic spline were used to analyze the individual effects of metals and LE8 on all-cause mortality. The additive and multiplicative interaction scales and quantile g-computation were used to evaluate the interaction and combined effects. Stratified analyses were performed to clarify whether metals and LE8 interacted with other variables to influence all-cause mortality.ResultsA total of 8017 participants were included in this study. The concentrations of cadmium, cobalt, lead, antimony, and thorium were greater in the low CVH group than in the high CVH group [median (µg/L): 0.29 vs. 0.19, 0.36 vs. 0.35, 0.48 vs. 0.39, 0.05 vs. 0.04, and 0.07 vs. 0.06]. The interaction between cadmium and LE8 was statistically significant, with a synergy index of 1.169 (95% CI: 1.004, 1.361). The stratified analyses showed that the interaction between age and LE8 had an impact on all-cause mortality (P for interaction = 0.004).ConclusionsIn this representative sample of the U.S. population, we found that the combined effect of cadmium, lead, thallium, and LE8 was positively associated with all-cause mortality. Furthermore, the interaction between cadmium and LE8 influenced all-cause mortality. So people should adopt healthy behaviors and reduce heavy metal exposure to minimize the risk of adverse health outcomes.

Distributed Least Product Relative Error estimation for semi-parametric multiplicative regression with massive data

Distributed systems has been widely used for massive data analysis, but few studies focus on multiplicative regression models. We consider a communication-efficient surrogate likelihood method using the Least Product Relative Error criterion for semi-parametric multiplicative models on massive datasets. The non-parametric component is efficiently handled via B-spline approximation. We derive the asymptotic properties for both parametric and non-parametric components, while the SCAD and adaptive Lasso penalty functions are developed and their oracle properties for variable selection are validated. Simulation studies and an application to an energy prediction dataset are used to demonstrate the effectiveness and practical utility of the proposed method.

On using the penalized regression estimators to solve the multicollinearity problem

The paper compares coefficient parameter estimation efficiency using penalized regression approaches. Five estimators are employed: Ridge Regression, LASSO regression, Elastic Net (ENET) Regression, Adaptive Lasso (ALASSO) regression, and Adaptive Elastic Net (AENET) regression methods. The study uses a multiple linear regression model to address multicollinearity issues. The comparison is based on average mean square errors (MSE) using simulated data with varying sizes, numbers of independent variables, and correlation coefficients. The results are expected to be useful and will be applied to real data to determine the best-performing estimator.

Gene Regulatory Network inference based on Modified Adaptive Lasso

Gene Regulatory Network inference based on Modified Adaptive Lasso

HIGHLY ADAPTIVE LASSO: MACHINE LEARNING THAT PROVIDES VALID NONPARAMETRIC INFERENCE IN REALISTIC MODELS.

Understanding treatment effects on health-related outcomes using real-world data requires defining a causal parameter and imposing relevant identification assumptions to translate it into a statistical estimand. Semiparametric methods, like the targeted maximum likelihood estimator (TMLE), have been developed to construct asymptotically linear estimators of these parameters. To further establish the asymptotic efficiency of these estimators, two conditions must be met: 1) the relevant components of the data likelihood must fall within a Donsker class, and 2) the estimates of nuisance parameters must converge to their true values at a rate faster than . The Highly Adaptive LASSO (HAL) satisfies these criteria by acting as an empirical risk minimizer within a class of càdlàg functions with a bounded sectional variation norm, which is known to be Donsker. HAL achieves the desired rate of convergence, thereby guaranteeing the estimators' asymptotic efficiency. The function class over which HAL minimizes its risk is flexible enough to capture realistic functions while maintaining the conditions for establishing efficiency. Additionally, HAL enables robust inference for non-pathwise differentiable parameters, such as the conditional average treatment effect (CATE) and causal dose-response curve, which are important in precision health. While these parameters are often considered in machine learning literature, these applications typically lack proper statistical inference. HAL addresses this gap by providing reliable statistical uncertainty quantification that is essential for informed decision-making in health research.

Application of artificial intelligence for feature engineering in education sector and learning science

This study investigates the utilization of artificial intelligence (AI) for feature engineering in the education sector, highlighting its potential to enhance individualized learning and improve academic outcomes. The correlation analysis, performed using a correlation matrix of the feature set, indicated that specific pairings of characteristics exhibit a strong association, resulting in the ineffectiveness of conventional models. In order to tackle this issue, we utilized three sophisticated machine learning methodologies: Adaptive Lasso (ALasso), Artificial Neural Networks (ANN), and Support Vector Regression (SVR). The ALasso model discovered several influential characteristics, namely Gender (X5), Education (X1), Hours of Work (X4), and Marital Status (X6), that significantly affect salaries. Subsequently, a comparative evaluation of these methods was conducted using Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). The results demonstrated that SVR outperformed the other techniques, with the most optimal RMSE of 0.595 and MAE of 0.423. These findings emphasize the significance of using data-driven strategies in policymaking and propose further investigation into the use of AI methods in various educational contexts to improve the identification of features and the performance of models.

A Flexible Adaptive Lasso Cox Frailty Model Based on the Full Likelihood.

In this work, a method to regularize Cox frailty models is proposed that accommodates time-varying covariates and time-varying coefficients and is based on the full likelihood instead of the partial likelihood. A particular advantage of this framework is that the baseline hazard can be explicitly modeled in a smooth, semiparametric way, for example, via P-splines. Regularization for variable selection is performed via a lasso penalty and via group lasso for categorical variables while a second penalty regularizes wiggliness of smooth estimates of time-varying coefficients and the baseline hazard. Additionally, adaptive weights are included to stabilize the estimation. The method is implemented in the R function coxlasso, which is now integrated into the package PenCoxFrail, and will be compared to other packages for regularized Coxregression.

Hyperarousal dynamics reveal an overnight increase boosted by insomnia

Hyperarousal is a key symptom of anxiety, stress-related disorders, and insomnia. However, it has been conceptualized in many different ways, ranging from various physiological markers (e.g. cortisol levels, high-frequency EEG activity) to personality traits, or state assessments of subjective anxiety and tension. This approach resulted in partly inconsistent evidence, complicating unified interpretations. Crucially, no previous studies addressed the likely variability of hyperarousal within and across days, nor the relationship of such variability in hyperarousal with the night-by-night variability in sleep quality characteristic of insomnia.Here, we present a novel data-driven approach to understanding dynamics of state hyperarousal in insomnia. Using ecological momentary assessment, we tracked fluctuations in a wide range of emotions across 9 days in 169 people with insomnia disorders and 38 controls without sleep problems. Exploratory factor analysis identified a hyperarousal factor, comprised of items describing tension and distress. People with insomnia scored significantly higher on this factor than controls at all timepoints. In both groups, the hyperarousal factor score peaked in the morning and waned throughout the day, pointing to a potential contributing role of sleep or other circadian processes. Importantly, the overnight increase in hyperarousal was stronger in people with in insomnia than in controls. Subsequent adaptive LASSO regression analysis revealed a stronger overnight increase in hyperarousal across nights of worse subjective sleep quality.These findings demonstrate the relationship between subjective sleep quality and overnight modulations of hyperarousal. Disorders in which hyperarousal is a predominant complaint might therefore benefit from interventions focused on improving sleep quality.

Bayesian adaptive lasso quantile regression with non-ignorable missing responses

Bayesian adaptive lasso quantile regression with non-ignorable missing responses

Distributed learning for kernel mode–based regression

AbstractWe propose a parametric kernel mode–based regression built on the mode value, which provides robust and efficient estimators for datasets containing outliers or heavy‐tailed distributions. To address the challenges posed by massive datasets, we integrate this regression method with distributed statistical learning techniques, which greatly reduces the required amount of primary memory and simultaneously accommodates heterogeneity in the estimation process. By approximating the local kernel objective function with a least squares format, we are able to preserve compact statistics for each worker machine, facilitating the reconstruction of estimates for the entire dataset with minimal asymptotic approximation error. Additionally, we explore shrinkage estimation through local quadratic approximation, showcasing that the resulting estimator possesses the oracle property through an adaptive LASSO approach. The finite‐sample performance of the developed method is illustrated using simulations and real data analysis.

Selection of Tuning Parameter and Comparison of Lasso and Adaptive Lasso on ZYZ Condition: A Monte Carlo Study

One of the fundamental objectives of statistics is to achieve accurate predictions. In high-dimensional settings (where the number of variables, p, exceeds the number of observations, n), the performance of ordinary least squares (OLS) is often suboptimal due to its high variance, which leads to lower prediction accuracy. Shrinking the variables is a promising approach, and methods such as ridge regression, elastic net, lasso, and adaptive lasso are well-known techniques for this purpose. While variable shrinkage introduces a small bias, it significantly reduces the variance compared to OLS. The effectiveness of shrinkage methods largely depends on the selection of the tuning parameter. Cross-validation and the Bayesian Information Criterion (BIC) are commonly used for this purpose, and an improved version of BIC has shown impressive results.???????? = (5.6,5.6,5.6,0), ???????? = (3,1.5,0,0,2,0,0,0), ???????? = (0.85,0.85,0.85,0) are the multiple regression models which are compared.

Claim Prediction and Premium Pricing for Telematics Auto Insurance Data Using Poisson Regression with Lasso Regularisation

We leverage telematics data on driving behavior variables to assess driver risk and predict future insurance claims in a case study utilising a representative telematics sample. In the study, we aim to categorise drivers according to their driving habits and establish premiums that accurately reflect their driving risk. To accomplish our goal, we employ the two-stage Poisson model, the Poisson mixture model, and the Zero-Inflated Poisson model to analyse the telematics data. These models are further enhanced by incorporating regularisation techniques such as lasso, adaptive lasso, elastic net, and adaptive elastic net. Our empirical findings demonstrate that the Poisson mixture model with the adaptive lasso regularisation outperforms other models. Based on predicted claim frequencies and drivers’ risk groups, we introduce a novel usage-based experience rating premium pricing method. This method enables more frequent premium updates based on recent driving behaviour, providing instant rewards and incentivising responsible driving practices. Consequently, it helps to alleviate cross-subsidization among risky drivers and improves the accuracy of loss reserving for auto insurance companies.

Automating the discovery of partial differential equations in dynamical systems

Abstract Identifying partial differential equations (PDEs) from data is crucial for understanding the governing mechanisms of natural phenomena, yet it remains a challenging task. We present an extension to the ARGOS framework, ARGOS-RAL, which leverages sparse regression with the recurrent adaptive lasso to identify PDEs from limited prior knowledge automatically. Our method automates calculating partial derivatives, constructing a candidate library, and estimating a sparse model. We rigorously evaluate the performance of ARGOS-RAL in identifying canonical PDEs under various noise levels and sample sizes, demonstrating its robustness in handling noisy and non-uniformly distributed data. We also test the algorithm’s performance on datasets consisting solely of random noise to simulate scenarios with severely compromised data quality. Our results show that ARGOS-RAL effectively and reliably identifies the underlying PDEs from data, outperforming the sequential threshold ridge regression method in most cases. We highlight the potential of combining statistical methods, machine learning, and dynamical systems theory to automatically discover governing equations from collected data, streamlining the scientific modeling process.

Bayesian Adaptive Lasso for Detecting Item-Trait Relationship and Differential Item Functioning in Multidimensional Item Response Theory Models.

In multidimensional tests, the identification of latent traits measured by each item is crucial. In addition to item-trait relationship, differential item functioning (DIF) is routinely evaluated to ensure valid comparison among different groups. The two problems are investigated separately in the literature. This paper uses a unified framework for detecting item-trait relationship and DIF in multidimensional item response theory (MIRT) models. By incorporating DIF effects in MIRT models, these problems can be considered as variable selection for latent/observed variables and their interactions. A Bayesian adaptive Lasso procedure is developed for variable selection, in which item-trait relationship and DIF effects can be obtained simultaneously. Simulation studies show the performance of our method for parameter estimation, the recovery of item-trait relationship and the detection of DIF effects. An application is presented using data from the Eysenck Personality Questionnaire.

ANALYSIS OF FACTORS AFFECTING LIFE EXPECTANCY IN NORTH SUMATRA PROVINCE USING LASSO AND ADAPTIVE LASSO

Life expectancy is a measure used to evaluate the government's effectiveness in enhancing the overall well-being and health of the population. The life expectancy of a town is impacted by the educational attainment of its residents. LASSO regression is a variant of linear regression that incorporates an extra penalty term into the cost function. adjustable LASSO Regression is a modified version of this technique that incorporates adjustable weighting variables for each coefficient, enhancing its effectiveness.Multicollinearity, a condition of strong correlation between predictor variables, can complicate the identification of independent variables that have a significant impact. The LASSO and Adaptive LASSO are variable selection strategies used in multiple linear regression situations. The LASSO and Adaptive LASSO techniques are effective for identifying important variables and establishing the link between independent and dependent variables

Metabolomic Prediction of Breast Cancer Treatment-Induced Neurologic and Metabolic Toxicities.

Long-term treatment-related toxicities, such as neurologic and metabolic toxicities, are major issues in breast cancer. We investigated the interest of metabolomic profiling to predict toxicities. Untargeted high-resolution metabolomic profiles of 992 patients with estrogen receptor (ER)+/HER2- breast cancer from the prospective CANTO cohort were acquired (n = 1935 metabolites). A residual-based modeling strategy with discovery and validation cohorts was used to benchmark machine learning algorithms, taking into account confounding variables. Adaptive Least Absolute Shrinkage and Selection (adaptive LASSO) has a good predictive performance, has limited optimism bias, and allows the selection of metabolites of interest for future translational research. The addition of low-frequency metabolites and nonannotated metabolites increases the predictive power. Metabolomics adds extra performance to clinical variables to predict various neurologic and metabolic toxicity profiles. Untargeted high-resolution metabolomics allows better toxicity prediction by considering environmental exposure, metabolites linked to microbiota, and low-frequency metabolites.

Smoothed empirical likelihood estimation and automatic variable selection for an expectile high-dimensional model

We consider a linear model which can have a large number of explanatory variables, the errors with an asymmetric distribution or the values of the explained variable are missing at random. In order to take in account these several situations, we consider the non parametric empirical likelihood (EL) estimation method. Because a constraint in EL contains an indicator function then a smoothed function instead of the indicator will be considered. Two smoothed expectile maximum EL methods are proposed, one of which will automatically select the explanatory variables. For each of the methods we obtain the convergence rate of the estimators and their asymptotic normality. The smoothed expectile empirical log-likelihood ratio process follow asymptotically a chi-square distribution and moreover the adaptive LASSO smoothed expectile maximum EL estimator satisfies the sparsity property which guarantees the automatic selection of zero model coefficients. In order to implement these methods, we propose four algorithms.

Multi-modality risk prediction of cardiovascular diseases for breast cancer cohort in the All of Us Research Program.

This study leverages the rich diversity of the All of Us Research Program (All of Us)'s dataset to devise a predictive model for cardiovascular disease (CVD) in breast cancer (BC) survivors. Central to this endeavor is the creation of a robust data integration pipeline that synthesizes electronic health records (EHRs), patient surveys, and genomic data, while upholding fairness across demographic variables. We have developed a universal data wrangling pipeline to process and merge heterogeneous data sources of the All of Us dataset, address missingness and variance in data, and align disparate data modalities into a coherent framework for analysis. Utilizing a composite feature set including EHR, lifestyle, and social determinants of health (SDoH) data, we then employed Adaptive Lasso and Random Forest regression models to predict 6 CVD outcomes. The models were evaluated using the c-index and time-dependent Area Under the Receiver Operating Characteristic Curve over a 10-year period. The Adaptive Lasso model showed consistent performance across most CVD outcomes, while the Random Forest model excelled particularly in predicting outcomes like transient ischemic attack when incorporating the full multi-model feature set. Feature importance analysis revealed age and previous coronary events as dominant predictors across CVD outcomes, with SDoH clustering labels highlighting the nuanced impact of social factors. The development of both Cox-based predictive model and Random Forest Regression model represents the extensive application of the All of Us, in integrating EHR and patient surveys to enhance precision medicine. And the inclusion of SDoH clustering labels revealed the significant impact of sociobehavioral factors on patient outcomes, emphasizing the importance of comprehensive health determinants in predictive models. Despite these advancements, limitations include the exclusion of genetic data, broad categorization of CVD conditions, and the need for fairness analyses to ensure equitable model performance across diverse populations. Future work should refine clinical and social variable measurements, incorporate advanced imputation techniques, and explore additional predictive algorithms to enhance model precision and fairness. This study demonstrates the liability of the All of Us's diverse dataset in developing a multi-modality predictive model for CVD in BC survivors risk stratification in oncological survivorship. The data integration pipeline and subsequent predictive models establish a methodological foundation for future research into personalized healthcare.

When MIDAS Meets LASSO: The Power of Low-Frequency Variables in Forecasting Value-at-Risk and Expected Shortfall

Abstract We propose a new framework for the joint estimation and forecasting of Value-at-Risk (VaR) and Expected Shortfall (ES) that integrates low-frequency variables. By maximizing the Asymmetric Laplace likelihood function with an Adaptive Lasso penalty, the most informative variables are selected on a rolling-window basis. In the empirical analysis, realized volatility, term spread, and housing starts serve as the strongest predictors of future tail risk. The out-of-sample backtesting results demonstrate that our method significantly outperforms other benchmarks, and achieves minimum loss in the joint forecasting of both the one-day-ahead and multi-day-ahead extreme S&P500 VaR and ES.