Penalized Regression Framework Research Articles

SPLENDID incorporates continuous genetic ancestry in biobank-scale data to improve polygenic risk prediction across diverse populations.

Polygenic risk scores are widely used in disease risk stratification, but their accuracy varies across diverse populations. Recent methods large-scale leverage multi-ancestry data to improve accuracy in under-represented populations but require labelling individuals by ancestry for prediction. This poses challenges for practical use, as clinical practices are typically not based on ancestry. We propose SPLENDID, a novel penalized regression framework for diverse biobank-scale data. Our method utilizes ancestry principal component interactions to model genetic ancestry as a continuum within a single prediction model for all ancestries, eliminating the need for discrete labels. In extensive simulations and analyses of 9 traits from the All of Us Research Program (N=224,364) and UK Biobank (N=340,140), SPLENDID significantly outperformed existing methods in prediction accuracy and model sparsity. By directly incorporating continuous genetic ancestry in model training, SPLENDID stands as a valuable tool for robust risk prediction across diverse populations and fairer clinical implementation.

The predictive utility of the in utero exposome for childhood adiposity in independent and integrated frameworks.

To assess the predictive potential of the in utero exposome in relation to childhood adiposity as indicated by body mass index z-scores (BMIz) and the fourth versus first quartile of % fat mass (FM) at median age of 4.6 years. We leveraged data on clinical risk factors for childhood obesity during the perinatal period, along with cord blood per/polyfluoroalkyl substances (PFAS) and cord blood DNA methylation, in 268 mother-offspring pairs. We used the sparsity ranked LASSO penalized regression framework for each outcome and assessed model performance based on % variability explained for BMIz and area under the receiver operating characteristic curve (AUC) for the fourth versus first quartile of %FM. We employed cross-validation for model tuning and split-sample validation for model evaluation. Mean ± SD BMIz was 0.01 ± 1.1, %FM was 19.8 ± 6.34%. The optimal model for predicting BMIz explained 19.1% of the variability in the validation set and included only clinical characteristics: maternal pre-pregnancy BMI, paternal BMI, gestational weight gain, physical activity during pregnancy and child race/ethnicity. The optimal model for fourth versus first quartiles of %FM achieved an AUC of 0.82 ± 0.01 in the validation set, with the clinical features again emerging as the strongest predictors. In this study sample, perinatal chemical exposures and the epigenome have low utility in predicting childhood adiposity, beyond known clinical risk factors.

Robust MAVE through nonconvex penalized regression

High dimensionality has been a significant feature in modern statistical modeling. Sufficient dimension reduction (SDR) as an efficient tool aims at reducing the original high dimensional predictors without losing any regression information. Minimum average variance estimation (MAVE) is a popular approach in SDR among others. However, it is not robust to outliers in the response due to the use of least squares. A robust estimation through regularization with case-specific parameters is proposed to achieve robust estimation and outlier detection simultaneously. Under the nonconvex penalized regression framework, two efficient computational strategies are introduced. Simulation studies and a real data application show the efficacy of the proposed approach. Compared with existing methods, the proposed approach is less sensitive to the choice of initial estimators.

Sign-based Shrinkage Based on an Asymmetric LASSO Penalty.

Penalized regression provides an automated approach to preform simultaneous variable selection and parameter estimation and is a popular method to analyze high-dimensional data. Since the conception of the LASSO in the mid-to-late 1990s, extensive research has been done to improve penalized regression. The LASSO, and several of its variations, performs penalization symmetrically around zero. Thus, variables with the same magnitude are shrunk the same regardless of the direction of effect. To the best of our knowledge, sign-based shrinkage, preferential shrinkage based on the sign of the coefficients, has yet to be explored under the LASSO framework. We propose a generalization to the LASSO, asymmetric LASSO, that performs sign-based shrinkage. Our method is motivated by placing an asymmetric Laplace prior on the regression coefficients, rather than a symmetric Laplace prior. This corresponds to an asymmetric ℓ 1 penalty under the penalized regression framework. In doing so, preferential shrinkage can be performed through an auxiliary tuning parameter that controls the degree of asymmetry. Our numerical studies indicate that the asymmetric LASSO performs better than the LASSO when effect sizes are sign skewed. Furthermore, in the presence of positively-skewed effects, the asymmetric LASSO is comparable to the non-negative LASSO without the need to place an a priori constraint on the effect estimates and outperforms the non-negative LASSO when negative effects are also present in the model. A real data example using the breast cancer gene expression data from The Cancer Genome Atlas is also provided, where the asymmetric LASSO identifies two potentially novel gene expressions that are associated with BRCA1 with a minor improvement in prediction performance over the LASSO and non-negative LASSO.

A neuroimaging marker for predicting longitudinal changes in pain intensity of subacute back pain based on large-scale brain network interactions

Identification of predictive neuroimaging markers of pain intensity changes is a crucial issue to better understand macroscopic neural mechanisms of pain. Although a single connection between the medial prefrontal cortex and nucleus accumbens has been suggested as a powerful marker, how the complex interactions on a large-scale brain network can serve as the markers is underexplored. Here, we aimed to identify a set of functional connections predictive of longitudinal changes in pain intensity using large-scale brain networks. We re-analyzed previously published resting-state functional magnetic resonance imaging data of 49 subacute back pain (SBP) patients. We built a network-level model that predicts changes in pain intensity over one year by combining independent component analysis and a penalized regression framework. Connections involving top-down pain modulation, multisensory integration, and mesocorticolimbic circuits were identified as predictive markers for pain intensity changes. Pearson’s correlations between actual and predicted pain scores were r = 0.33–0.72, and group classification results between SBP patients with persisting pain and recovering patients, in terms of area under the curve (AUC), were 0.89/0.75/0.75 for visits four/three/two, thus outperforming the previous work (AUC 0.83/0.73/0.67). This study identified functional connections important for longitudinal changes in pain intensity in SBP patients, providing provisional markers to predict future pain using large-scale brain networks.

The impact of sentiment and attention measures on stock market volatility

We analyze the impact of sentiment and attention variables on the stock market volatility by using a novel and extensive dataset that combines social media, news articles, information consumption, and search engine data. We apply a state-of-the-art sentiment classification technique in order to investigate the question of whether sentiment and attention measures contain additional predictive power for realized volatility when controlling for a wide range of economic and financial predictors. Using a penalized regression framework, we identify the most relevant variables to be investors’ attention, as measured by the number of Google searches on financial keywords (e.g. “financial market” and “stock market”), and the daily volume of company-specific short messages posted on StockTwits. In addition, our study shows that attention and sentiment variables are able to improve volatility forecasts significantly, although the magnitudes of the improvements are relatively small from an economic point of view.

The Impact of Sentiment and Attention Measures on Stock Market Volatility

We analyze the impact of sentiment and attention variables on volatility by using a novel and extensive dataset that combines social media, news articles, information consumption, and search engine data. Applying a state-of-the-art sentiment classification technique, we investigate the question of whether sentiment and attention measures contain additional predictive power for realized volatility when controlling for a wide range of economic and predictors. Using a penalized regression framework, we identify investors' attention, as measured by the number of Google searches on keywords (e.g. financial market and stock market), and the daily volume of company-specific short messages posted on StockTwits to be the most relevant variables. In addition, our study shows that attention and sentiment variables are able to significantly improve volatility forecasts, although the improvements are of relatively small magnitude from an economic point of view.

Polygenic scores via penalized regression on summary statistics.

Polygenic scores (PGS) summarize the genetic contribution of a person's genotype to a disease or phenotype. They can be used to group participants into different risk categories for diseases, and are also used as covariates in epidemiological analyses. A number of possible ways of calculating PGS have been proposed, and recently there is much interest in methods that incorporate information available in published summary statistics. As there is no inherent information on linkage disequilibrium (LD) in summary statistics, a pertinent question is how we can use LD information available elsewhere to supplement such analyses. To answer this question, we propose a method for constructing PGS using summary statistics and a reference panel in a penalized regression framework, which we call lassosum. We also propose a general method for choosing the value of the tuning parameter in the absence of validation data. In our simulations, we showed that pseudovalidation often resulted in prediction accuracy that is comparable to using a dataset with validation phenotype and was clearly superior to the conservative option of setting the tuning parameter of lassosum to its lowest value. We also showed that lassosum achieved better prediction accuracy than simple clumping and P-value thresholding in almost all scenarios. It was also substantially faster and more accurate than the recently proposed LDpred.

Adaptive penalties for generalized Tikhonov regularization in statistical regression models with application to spectroscopy data.

Tikhonov regularization was proposed for multivariate calibration by Andries and Kalivas [1]. We use this framework for modeling the statistical association between spectroscopy data and a scalar outcome. In both the calibration and regression settings this regularization process has advantages over methods of spectral pre-processing and dimension-reduction approaches such as feature extraction or principal component regression. We propose an extension of this penalized regression framework by adaptively refining the penalty term to optimally focus the regularization process. We illustrate the approach using simulated spectra and compare it with other penalized regression models and with a two-step method that first pre-processes the spectra then fits a dimension-reduced model using the processed data. The methods are also applied to magnetic resonance spectroscopy data to identify brain metabolites that are associated with cognitive function.

JAM: A Scalable Bayesian Framework for Joint Analysis of Marginal SNP Effects.

ABSTRACTRecently, large scale genome‐wide association study (GWAS) meta‐analyses have boosted the number of known signals for some traits into the tens and hundreds. Typically, however, variants are only analysed one‐at‐a‐time. This complicates the ability of fine‐mapping to identify a small set of SNPs for further functional follow‐up. We describe a new and scalable algorithm, joint analysis of marginal summary statistics (JAM), for the re‐analysis of published marginal summary stactistics under joint multi‐SNP models. The correlation is accounted for according to estimates from a reference dataset, and models and SNPs that best explain the complete joint pattern of marginal effects are highlighted via an integrated Bayesian penalized regression framework. We provide both enumerated and Reversible Jump MCMC implementations of JAM and present some comparisons of performance. In a series of realistic simulation studies, JAM demonstrated identical performance to various alternatives designed for single region settings. In multi‐region settings, where the only multivariate alternative involves stepwise selection, JAM offered greater power and specificity. We also present an application to real published results from MAGIC (meta‐analysis of glucose and insulin related traits consortium) – a GWAS meta‐analysis of more than 15,000 people. We re‐analysed several genomic regions that produced multiple significant signals with glucose levels 2 hr after oral stimulation. Through joint multivariate modelling, JAM was able to formally rule out many SNPs, and for one gene, ADCY5, suggests that an additional SNP, which transpired to be more biologically plausible, should be followed up with equal priority to the reported index.

A 2-step penalized regression method for family-based next-generation sequencing association studies.

Large-scale genetic studies are often composed of related participants, and utilizing familial relationships can be cumbersome and computationally challenging. We present an approach to efficiently handle sequencing data from complex pedigrees that incorporates information from rare variants as well as common variants. Our method employs a 2-step procedure that sequentially regresses out correlation from familial relatedness and then uses the resulting phenotypic residuals in a penalized regression framework to test for associations with variants within genetic units. The operating characteristics of this approach are detailed using simulation data based on a large, multigenerational cohort.

Mixed Finite Elements for Spatial Regression with PDE Penalization

We study a class of models at the interface between statistics and numerical analysis. Specifically, we consider nonparametric regression models for the estimation of spatial fields from pointwise and noisy observations, which account for problem-specific prior information, described in terms of a partial differential equation governing the phenomenon under study. The prior information is incorporated in the model via a roughness term using a penalized regression framework. We prove the well-posedness of the estimation problem, and we resort to a mixed equal order finite element method for its discretization. Moreover, we prove the well-posedness and the optimal convergence rate of the proposed discretization method. Finally the smoothing technique is extended to the case of areal data, particularly interesting in many applications.

Bayesian endogeneity bias modeling

We propose to model endogeneity bias using prior distributions of moment conditions. The estimator can be obtained both as a method-of-moments estimator and in a Ridge penalized regression framework. We show the estimator’s relation to a Bayesian estimator.

EEG/MEG Source Reconstruction with Spatial-Temporal Two-Way Regularized Regression

In this work, we propose a spatial-temporal two-way regularized regression method for reconstructing neural source signals from EEG/MEG time course measurements. The proposed method estimates the dipole locations and amplitudes simultaneously through minimizing a single penalized least squares criterion. The novelty of our methodology is the simultaneous consideration of three desirable properties of the reconstructed source signals, that is, spatial focality, spatial smoothness, and temporal smoothness. The desirable properties are achieved by using three separate penalty functions in the penalized regression framework. Specifically, we impose a roughness penalty in the temporal domain for temporal smoothness, and a sparsity-inducing penalty and a graph Laplacian penalty in the spatial domain for spatial focality and smoothness. We develop a computational efficient multilevel block coordinate descent algorithm to implement the method. Using a simulation study with several settings of different spatial complexity and two real MEG examples, we show that the proposed method outperforms existing methods that use only a subset of the three penalty functions.

A Note on Bivariate Smoothing for Two-Dimensional Functional Data

In this paper we study a bivariate smoothing approach for estimating multiple functional parameters for functional data with a two-dimensional domain. We present a penalized regression framework for smoothing with the purpose to: (a) facilitate the estimation of the smooth overall bivariate mean function of two-dimensional functional data, (b) enable the estimation of the functional effect of a scalar covariate, (c) accommodate completely or incompletely sampled data, (d) implement the fitting approach using available statistical software, and (e) construct pointwise approximate confidence intervals for multiple bivariate functional parameters. Implementation results from simulation studies show that these methods perform very well in practice. We illustrate the usefulness of the bivariate smoothing approach to several real datasets, including applications to electricity demand.

Variable selection for optimal treatment decision

In decision-making on optimal treatment strategies, it is of great importance to identify variables that are involved in the decision rule, i.e. those interacting with the treatment. Effective variable selection helps to improve the prediction accuracy and enhance the interpretability of the decision rule. We propose a new penalized regression framework which can simultaneously estimate the optimal treatment strategy and identify important variables. The advantages of the new approach include: (i) it does not require the estimation of the baseline mean function of the response, which greatly improves the robustness of the estimator; (ii) the convenient loss-based framework makes it easier to adopt shrinkage methods for variable selection, which greatly facilitates implementation and statistical inferences for the estimator. The new procedure can be easily implemented by existing state-of-art software packages like LARS. Theoretical properties of the new estimator are studied. Its empirical performance is evaluated using simulation studies and further illustrated with an application to an AIDS clinical trial.

Penalized regression elucidates aberration hotspots mediating subtype-specific transcriptional responses in breast cancer

Copy number alterations (CNAs) associated with cancer are known to contribute to genomic instability and gene deregulation. Integrating CNAs with gene expression helps to elucidate the mechanisms by which CNAs act and to identify the transcriptional downstream targets of CNAs. Such analyses can help to sort functional driver events from the many accompanying passenger alterations. However, the way CNAs affect gene expression can vary in different cellular contexts, for example between different subtypes of the same cancer. Thus, it is important to develop computational approaches capable of inferring differential connectivity of regulatory networks in different cellular contexts. We propose a statistical deregulation model that integrates copy number and expression data of different disease subtypes to jointly model common and differential regulatory relationships. Our model not only identifies CNAs driving gene expression changes, but at the same time also predicts differences in regulation that distinguish one cancer subtype from the other. We implement our model in a penalized regression framework and demonstrate in a simulation study the feasibility and accuracy of our approach. Subsequently, we show that this model can identify both known and novel aspects of cross-talk between the ER and NOTCH pathways in ER-negative-specific deregulations, when compared with ER-positive breast cancer. This flexible model can be applied on other modalities such as methylation or microRNA and expression to disentangle cancer signaling pathways. The Bioconductor-compliant R package DANCE is available from www.markowetzlab.org/software/ yinyin.yuan@cancer.org.uk; florian.markowetz@cancer.org.uk.

Penalized regression for genome-wide association screening of sequence data.

Whole exome and whole genome sequencing are likely to be potent tools in the study of common diseases and complex traits. Despite this promise, some very difficult issues in data management and statistical analysis must be squarely faced. The number of rare variants identified by sequencing is apt to be much larger than the number of common variants encountered in current association studies. The low frequencies of rare variants alone will make association testing difficult. This article extends the penalized regression framework for model selection in genome-wide association data to sequencing data with both common and rare variants. Previous research has shown that lasso penalties discourage irrelevant predictors from entering a model. The Euclidean penalties dealt with here group variants by gene or pathway. Pertinent biological information can be incorporated by calibrating penalties by weights. The current paper examines some of the tradeoffs in using pure lasso penalties, pure group penalties, and mixtures of the two types of penalty. All of the computational and statistical advantages of lasso penalized estimation are retained in this richer setting. The overall strategy is implemented in the free statistical genetics analysis software MENDEL and illustrated on both simulated and real data.