Lasso-type Penalty Research Articles

Community influence analysis in social networks

Heterogeneous influence detection across network nodes is an important task in network analysis. A community influence model (CIM) is proposed to allow nodes to be classified into different communities (i.e., clusters or groups) such that the nodes within the same community share the common influence parameter. Employing the quasi-maximum likelihood approach, together with the fused lasso-type penalty, both the number of communities and the influence parameters can be estimated without imposing any specific distribution assumption on the error terms. The resulting estimators are shown to enjoy the oracle property; namely, they perform as well as if the true underlying network structure were known in advance. The proposed approach is also applicable for identifying influential nodes in a homogeneous setting. The performance of our method is illustrated via simulation studies and two empirical examples using stock data and coauthor citation data, respectively.

Revealing genomic heterogeneity and commonality: A penalized integrative analysis approach accounting for the adjacency structure of measurements.

Advancements in high-throughput genomic technologies have revolutionized the field of disease biomarker identification by providing large-scale genomic data. There is an increasing focus on understanding the relationships among diverse patient groups with distinct disease subtypes and characteristics. Complex diseases exhibit both heterogeneity and shared genomic factors, making it essential to investigate these patterns to accurately detect markers and comprehensively understand the diseases. Integrative analysis has emerged as a promising approach to address this challenge. However, existing studies have been limited by ignoring the adjacency structure of genomic measurements, such as single nucleotide polymorphisms (SNPs) and DNA methylations. In this study, we propose a structured integrative analysis method that incorporates a spline type penalty to accommodate this adjacency structure. We utilize a fused lasso type penalty to identify both heterogeneity and commonality across the groups. Extensive simulations demonstrate its superiority compared to several direct competing methods. The analysis of The Cancer Genome Atlas melanoma data with DNA methylation measurements and GENEVA diabetes data with SNP measurements exhibit that the proposed analysis lead to meaningful findings with better prediction performance and higher selection stability.

Homogeneity pursuit and variable selection in regression models for multivariate abundance data.

When building regression models for multivariate abundance data in ecology, it is important to allow for the fact that the species are correlated with each other. Moreover, there is often evidence species exhibit some degree of homogeneity in their responses to each environmental predictor, and that most species are informed by only a subset of predictors. We propose a generalized estimating equation(GEE) approach for simultaneous homogeneity pursuit (ie, grouping species with similar coefficient values while allowing differing groups for different covariates) and variable selection in regression models for multivariate abundance data. Using GEEs allows us to straightforwardly account for between-response correlations through a (reduced-rank) working correlation matrix. We augment the GEE with both adaptive fused lasso- and adaptive lasso-type penalties, which aim to cluster the species-specific coefficients within each covariate and encourage differing levels of sparsity across the covariates, respectively. Numerical studies demonstrate the strong finite sample performance of the proposed method relative to several existing approaches for modeling multivariate abundance data. Applying the proposed method to presence-absence records collected along the Great Barrier Reef in Australia reveals both a substantial degree of homogeneity and sparsity in species-environmental relationships. We show this leads to a more parsimonious model for understanding the environmental drivers of seabed biodiversity, and results in stronger out-of-sample predictive performance relative to methods that do not accommodate such features.

Robust low tubal rank tensor recovery via [formula omitted]E criterion

Tensor analysis has received enormous attention as the increasing prevalence of high-dimensional data in science research and engineering application. Tensor recovery is an important and meaningful problem for tensor analysis. It aims to complete a tensor from an observed subset of its entries disturbed by noise. However, classical methods either develop on the second-order statistics or Lasso-type penalty, leading to them not effectively dealing with gross or dense noise effectively. To address such issues, we propose a robust tensor recovery model for simultaneously completing a low tubal rank tensor with complex noise and missing data. Based on tensor–tensor product (t-product), we first develop a tensor factor Frobenius norm to exploit the low tubal rank property which is closely related to tensor nuclear norm and tubal rank. By utilizing the robust L2 criterion, we derive the nonconvex objective function to accommodate the low tubal rank tensor. An implementable alternating minimization algorithm has been developed to estimate the low tubal rank tensor. It is worth noting that our method is able to jointly estimate the precision parameter to capture the hidden complex noise pattern. Furthermore, some convergence properties of the proposed algorithm are presented. A series of numerical experiments are conducted on both synthetic and real-world data to demonstrate the effectiveness and robustness of the proposed approach in comparison with the state-of-the-art methods.

Spatial smoothing revisited: An application to rental data in Munich

Spatial smoothing makes use of spatial information to obtain better estimates in regression models. In particular flexible smoothing with B-splines and penalties, which has been propagated by Eilers and Marx (1996) , provides strong tools that can be used to include available spatial information. We consider alternative smoothing methods in spatial additive regression and employ them for analysing rental data in Munich. The first method applies tensor product P-splines to the geolocation of apartments, measured on a continuous scale through the centroid of the quarter where an apartment is. The alternative approach exploits the neighbourhood structure of districts on a discrete scale, where districts consist of a set of neighbouring quarters. The discrete modelling approach yields smooth estimates when using ridge-type penalties but can also enforce spatial clustering of districts with a homogeneous structure when using Lasso-type penalties.

High-dimensional sparse index tracking based on a multi-step convex optimization approach

Both convex and non-convex penalties have been widely proposed to tackle the sparse index tracking problem. Owing to their good property of generating sparse solutions, penalties based on the least absolute shrinkage and selection operator (LASSO) and its variations are often suggested in the stream of convex penalties. However, the LASSO-type penalty is often shown to have poor out-of-sample performance, due to the relatively large biases introduced in the estimates of tracking portfolio weights by shrinking the parameter estimates toward to zero. On the other hand, non-convex penalties could be used to improve the bias issue of LASSO-type penalty. However, the resulting problem is non-convex optimization and thus is computationally intensive, especially in high-dimensional settings. Aimed at ameliorating bias introduced by LASSO-type penalty while preserving computational efficiency, this paper proposes a multi-step convex optimization approach based on the multi-step weighted LASSO (MSW-LASSO) for sparse index tracking. Empirical results show that the proposed method can achieve smaller out-of-sample tracking errors than those based on LASSO-type penalties and have performance competitive to those based on non-convex penalties.

A practical approach to making use of uncertain species presence-only data in ecology: Reclassification, regularization methods and observer bias

Various statistical models and software platforms aim to produce species distribution models to better predict where species occur as a function of the environment. However, there are many practical challenges that arise with observations coming from opportunistic surveys. Such data may be of low quality with respect to accuracy and may also exhibit sampling bias. Here, we explore three main challenges. First, species identification can be misleading with the changes in taxonomy where the identification of species has changed for some genus, rendering older records confounded with respect to species identity. Second, the observers' sampled pattern may not reflect the true species distribution as some observers may favor some areas where the species is found. Furthermore, ecological knowledge of environmental drivers of a species distribution may be limited, which presents challenges in selecting appropriate covariates to include in species distribution models. In this paper, we extend two algorithms we recently developed which make use of misidentified observations in order to predict species distributions using spatial point processes. In particular, these algorithms incorporate sampling bias correction and address potential overfitting of the model via lasso-type penalties. We compare the performance of these algorithms to models which do not make use of the confounded species data, and explore the effects of the lasso penalty and bias correction on model performance. We apply the best performing methods to a real dataset of eastern Australian frogs for which taxonomy recently changed. Including confounded observations in the models is particularly relevant for informing management decisions regarding endangered species and species in remote areas.

Quantile regression shrinkage and selection via the Lqsso

ABSTRACT Quantile regression has recently received a considerable attention due to its remarkable development in enriching the variety of regression models. Many efforts have been made to blend different penalty and loss function to extend or develop novel regression models that are unique from different perspectives. Bearing in mind that the lasso quantile regression model ignores the randomness of the realizations in the penalty part, we propose a new penalty for the quantile regression models. Similar to the adaptive lasso quantile regression model, the proposed model simultaneously does estimation and variable selection tasks. We call the new model ‘lqsso-QR’, standing for the least quantile shrinkage and selection operator quantile regression. In this article, we present a sufficient and necessary condition for the variable selection of the lasso quantile regression to enjoy the consistent property. We show that the lqsso-QR follows oracle properties under some mild conditions. From computational perspective, we apply an efficient algorithm, originally developed for the lasso quantile regression. Using simulation studies, we elaborate on the superiority of the proposed model compared with other lasso-type penalties, especially regarding relative prediction error. Also, an application of our method to a real-life data; the rat eye data, is reported.

Forecasting Mortality Rates with a Two-Step LASSO Based Vector Autoregressive Model

This paper proposes a two-step LASSO based vector autoregressive (2-LVAR) model to forecast mortality rates. Within the VAR framework, recent studies have developed a spatial–temporal autoregressive (STAR) model, in which age-specific mortality rates are related to their own historical values (temporality) and the rates of the neighboring cohorts (spatiality). Despite its desirable age coherence property and the improved forecasting accuracy over the widely used Lee–Carter (LC) model, STAR employs a rather restrictive structure that only allows for non-zero cohort effects of the same cohorts and the neighboring cohorts. To address this limitation, the proposed 2-LVAR model adopts a data-driven principle, as in a sparse VAR (SVAR) model, to offer more flexibility in the parametric structure. A two-step estimation strategy is developed accordingly to resolve the challenging objective function of 2-LVAR, which consists of non-standard L2 and LASSO-type penalties with constraints. Using empirical data from Australia, the United Kingdom, France, and Switzerland, we show that the 2-LVAR model outperforms the LC, STAR, and SVAR models in most of our forecasting results. Further simulation studies confirm this outperformance, and analyses based on life expectancy at birth empirically support the existence of age coherence. The results of this paper will help researchers understand the mortality projections in the long run and improve the reserving/ratemaking accuracy for life insurers.

Penalized weighted proportional hazards model for robust variable selection and outlier detection.

Identifying exceptional responders or nonresponders is an area of increased research interest in precision medicine as these patients may have different biological or molecular features and therefore may respond differently to therapies. Our motivation stems from a real example from a clinical trial where we are interested in characterizing exceptional prostate cancer responders. We investigate the outlier detection and robust regression problem in the sparse proportional hazards model for censored survival outcomes. The main idea is to model the irregularity of each observation by assigning an individual weight to the hazard function. By applying a LASSO-type penalty on both the model parameters and the log transformation of the weight vector, our proposed method is able to perform variable selection and outlier detection simultaneously. The optimization problem can be transformed to a typical penalized maximum partial likelihood problem and thus it is easy to implement. We further extend the proposed method to deal with the potential outlier masking problem caused by censored outcomes. The performance of the proposed estimator is demonstrated with extensive simulation studies and real data analyses in low-dimensional and high-dimensional settings.

Clinical, Virological and Immunological Subphenotypes in a Cohort of Early Treated HIV-Infected Children.

BackgroundIdentifying subphenotypes within heterogeneous diseases may have an impact in terms of therapeutic options. In this study, we aim to assess different subphenotypes in children living with human immunodeficiency virus (HIV-1), according to the clinical, virological, and immunological characteristics.MethodsWe collected clinical and sociodemographic data, baseline viral load (VL), CD4 and CD8 count and percentage, age at initiation of ART, HIV DNA reservoir size in peripheral blood mononuclear cells (PBMCs), cell-associated RNA (CA-RNA), ultrasensitive VL, CD4 subsets (T effector CD25+, activated memory cells, Treg cells), humoral-specific HIV response (T-bet B cells), innate response (CD56dim natural killer (NK) cells, NKp46+, perforin), exhaustion markers (PD-1, PD-L1, DNAM), CD8 senescence, and biomarkers for T-lymphocyte thymic output (TREC) and endothelial activation (VCAM). The most informative variables were selected using an unsupervised lasso-type penalty selection for sparse clustering. Hierarchical clustering was performed using Pearson correlation as the distance metric and WARD.D2 as the clustering method. Internal validation was applied to select the best number of clusters. To compare the characteristics among clusters, boxplot and Kruskal Wallis test were assessed.ResultsThree subphenotypes were discovered (cluster1: n=18, 45%; cluster2: n=11, 27.5%; cluster3: n=11, 27.5%). Patients in cluster1 were treated earlier, had higher baseline %CD4, low HIV reservoir size, low western blot score, higher TREC values, and lower VCAM values than the patients in the other clusters. In contrast, cluster3 was the less favorable. Patients were treated later and presented poorer outcomes with lower %CD4, and higher reservoir size, along with a higher percentage of CD8 immunosenescent cells, lower TREC, higher VCAM cytokine, and a higher %CD4 PD-1. Cluster2 was intermediate. Patients were like those of cluster1, but had lower levels of t-bet expression and higher HIV DNA reservoir size.ConclusionsThree HIV pediatric subphenotypes with different virological and immunological features were identified. The most favorable cluster was characterized by a higher rate of immune reconstitution and a slower disease progression, and the less favorable with more senescence and high reservoir size. In the near future therapeutic interventions for a path of a cure might be guided or supported by the different subphenotypes.

ARZIMM: A Novel Analytic Platform for the Inference of Microbial Interactions and Community Stability from Longitudinal Microbiome Study.

Dynamic changes of microbiome communities may play important roles in human health and diseases. The recent rise in longitudinal microbiome studies calls for statistical methods that can model the temporal dynamic patterns and simultaneously quantify the microbial interactions and community stability. Here, we propose a novel autoregressive zero-inflated mixed-effects model (ARZIMM) to capture the sparse microbial interactions and estimate the community stability. ARZIMM employs a zero-inflated Poisson autoregressive model to model the excessive zero abundances and the non-zero abundances separately, a random effect to investigate the underlining dynamic pattern shared within the group, and a Lasso-type penalty to capture and estimate the sparse microbial interactions. Based on the estimated microbial interaction matrix, we further derive the estimate of community stability, and identify the core dynamic patterns through network inference. Through extensive simulation studies and real data analyses we evaluate ARZIMM in comparison with the other methods.

Searching for minimal optimal neural networks

Large neural network models have high predictive power but may suffer from overfitting if the training set is not large enough. Therefore, it is desirable to select an appropriate size for neural networks. The destructive approach, which starts with a large architecture and then reduces the size using a Lasso-type penalty, has been used extensively for this task. Despite its popularity, there is no theoretical guarantee for this technique. Based on the notion of minimal neural networks, we posit a rigorous mathematical framework for studying the asymptotic theory of the destructive technique. We prove that Adaptive group Lasso is consistent and can reconstruct the correct number of hidden nodes of one-hidden-layer feedforward networks with high probability. To the best of our knowledge, this is the first theoretical result establishing for the destructive technique.

Introducing LASSO-type penalisation to generalised joint regression modelling for count data

In this work, we propose an extension of the versatile joint regression framework for bivariate count responses of the R package GJRM by Marra and Radice (R package version 0.2-3, 2020) by incorporating an (adaptive) LASSO-type penalty. The underlying estimation algorithm is based on a quadratic approximation of the penalty. The method enables variable selection and the corresponding estimates guarantee shrinkage and sparsity. Hence, this approach is particularly useful in high-dimensional count response settings. The proposal’s empirical performance is investigated in a simulation study and an application on FIFA World Cup football data.

A Penalized Matrix Normal Mixture Model for Clustering Matrix Data.

Along with advances in technology, matrix data, such as medical/industrial images, have emerged in many practical fields. These data usually have high dimensions and are not easy to cluster due to their intrinsic correlated structure among rows and columns. Most approaches convert matrix data to multi dimensional vectors and apply conventional clustering methods to them, and thus, suffer from an extreme high-dimensionality problem as well as a lack of interpretability of the correlated structure among row/column variables. Recently, a regularized model was proposed for clustering matrix-valued data by imposing a sparsity structure for the mean signal of each cluster. We extend their approach by regularizing further on the covariance to cope better with the curse of dimensionality for large size images. A penalized matrix normal mixture model with lasso-type penalty terms in both mean and covariance matrices is proposed, and then an expectation maximization algorithm is developed to estimate the parameters. The proposed method has the competence of both parsimonious modeling and reflecting the proper conditional correlation structure. The estimators are consistent, and their limiting distributions are derived. We applied the proposed method to simulated data as well as real datasets and measured its clustering performance with the clustering accuracy (ACC) and the adjusted rand index (ARI). The experiment results show that the proposed method performed better with higher ACC and ARI than those of conventional methods.

Meta-Analyzing Multiple Omics Data With Robust Variable Selection.

High-throughput omics data are becoming more and more popular in various areas of science. Given that many publicly available datasets address the same questions, researchers have applied meta-analysis to synthesize multiple datasets to achieve more reliable results for model estimation and prediction. Due to the high dimensionality of omics data, it is also desirable to incorporate variable selection into meta-analysis. Existing meta-analyzing variable selection methods are often sensitive to the presence of outliers, and may lead to missed detections of relevant covariates, especially for lasso-type penalties. In this paper, we develop a robust variable selection algorithm for meta-analyzing high-dimensional datasets based on logistic regression. We first search an outlier-free subset from each dataset by borrowing information across the datasets with repeatedly use of the least trimmed squared estimates for the logistic model and together with a hierarchical bi-level variable selection technique. We then refine a reweighting step to further improve the efficiency after obtaining a reliable non-outlier subset. Simulation studies and real data analysis show that our new method can provide more reliable results than the existing meta-analysis methods in the presence of outliers.

Learning rates for partially linear support vector machine in high dimensions

This paper analyzes a new regularized learning scheme for high-dimensional partially linear support vector machine (SVM). The proposed approach consists of an empirical risk and the Lasso-type penalty for linear part, as well as the standard functional norm for nonlinear part. Here, the linear kernel is used for model interpretation and feature selection, while the nonlinear kernel is adopted to enhance algorithmic flexibility. In this paper, we develop a new technical analysis on the weighted empirical process, and establish the sharp learning rates for the semi-parametric estimator under the regularized conditions. Specially, our derived learning rates for semi-parametric SVM depend on not only the sample size and the functional complexity, but also the sparsity and the margin parameters.

Scalable network estimation with L 0 penalty.

With the advent of high-throughput sequencing, an efficient computing strategy is required to deal with large genomic data sets. The challenge of estimating a large precision matrix has garnered substantial research attention for its direct application to discriminant analyses and graphical models. Most existing methods either use a lasso-type penalty that may lead to biased estimators or are computationally intensive, which prevents their applications to very large graphs. We propose using an L 0 penalty to estimate an ultra-large precision matrix (scalnetL0). We apply scalnetL0 to RNA-seq data from breast cancer patients represented in The Cancer Genome Atlas and find improved accuracy of classifications for survival times. The estimated precision matrix provides information about a large-scale co-expression network in breast cancer. Simulation studies demonstrate that scalnetL0 provides more accurate and efficient estimators, yielding shorter CPU time and less Frobenius loss on sparse learning for large-scale precision matrix estimation.

Brain connectivity alteration detection via matrix-variate differential network model.

Brain functional connectivity reveals the synchronization of brain systems through correlations in neurophysiological measures of brain activities. Growing evidence now suggests that the brain connectivity network experiences alterations with the presence of numerous neurological disorders, thus differential brain network analysis may provide new insights into disease pathologies. The data from neurophysiological measurement are often multidimensional and in a matrix form, posing a challenge in brain connectivity analysis. Existing graphical model estimation methods either assume a vector normal distribution that in essence requires the columns of the matrix data to be independent or fail to address the estimation of differential networks across different populations. To tackle these issues, we propose an innovative matrix-variate differential network (MVDN) model. We exploit the D-trace loss function and a Lasso-type penalty to directly estimate the spatial differential partial correlation matrix and use an alternating direction method of multipliers algorithm for the optimization problem. Theoretical and simulation studies demonstrate that MVDN significantly outperforms other state-of-the-art methods in dynamic differential network analysis. We illustrate with a functional connectivity analysis of an attention deficit hyperactivity disorder dataset. The hub nodes and differential interaction patterns identified are consistent with existing experimentalstudies.

Personalized Policy Learning Using Longitudinal Mobile Health Data

Personalized policy represents a paradigm shift one decision rule for all users to an individualized decision rule for each user. Developing personalized policy in mobile health applications imposes challenges. First, for lack of adherence, data from each user are limited. Second, unmeasured contextual factors can potentially impact on decision making. Aiming to optimize immediate rewards, we propose using a generalized linear mixed modeling framework where population features and individual features are modeled as fixed and random effects, respectively, and synthesized to form the personalized policy. The group lasso type penalty is imposed to avoid overfitting of individual deviations from the population model. We examine the conditions under which the proposed method work in the presence of time-varying endogenous covariates, and provide conditional optimality and marginal consistency results of the expected immediate outcome under the estimated policies. We apply our method to develop personalized push (“prompt”) schedules in 294 app users, with the goal to maximize the prompt response rate given past app usage and other contextual factors. The proposed method compares favorably to existing estimation methods including using the R function “glmer” in a simulation study. Supplementary materials for this article are available online.