Adaptive Lasso Penalty Research Articles

Adaptive LASSO model selection in a multiphase quantile regression

We propose a general adaptive LASSO method for a quantile regression model. Our method is very interesting when we know nothing about the first two moments of the model error. We first prove that the obtained estimators satisfy the oracle properties, which involves the relevant variable selection without using hypothesis test. Next, we study the proposed method when the (multiphase) model changes to unknown observations called change-points. Convergence rates of the change-points and of the regression parameter estimators in each phase are found. The sparsity of the adaptive LASSO quantile estimators of the regression parameters is not affected by the change-points estimation. If the number of phases is unknown, a consistent criterion is proposed. Numerical studies by Monte Carlo simulations show the performance of the proposed method, compared to other existing methods in the literature, for models with a single phase or for multiphase models. The adaptive LASSO quantile method performs better than known variable selection methods, as the least squared method with adaptive LASSO penalty, -method with LASSO-type penalty and quantile method with SCAD penalty.

Sparse Sliced Inverse Quantile Regression

The current paper proposes the sliced inverse quantile regression method (SIQR). In addition to the latter this study proposes both the sparse sliced inverse quantile regression method with Lasso (LSIQR) and Adaptive Lasso (ALSIQR) penalties. This article introduces a comprehensive study of SIQR and sparse SIQR. The simulation and real data analysis have been employed to check the performance of the SIQR, LSIQR and ALSIQR. According to the results of median of mean squared error and the absolute correlation criteria, we can conclude that the SIQR, LSIQR and ALSIQR are the more advantageous approaches in practice.

Fine particles, genetic pathways, and markers of inflammation and endothelial dysfunction: Analysis on particulate species and sources.

Studies have found associations between PM2.5 and cardiovascular events. The role of different components of PM2.5 is not well understood. We used linear mixed-effects models with the adaptive LASSO penalty to select PM2.5 species and source(s), separately, that may be associated with markers of inflammation and endothelial dysfunction, with adjustment for age, obesity, smoking, statin use, diabetes mellitus, temperature, and season as fixed effects in a large longitudinal cohort of elderly men. We also analyzed these associations with source apportionment models and examined genetic pathway-air pollution interactions within three relevant pathways (oxidative stress, metal processing, and endothelial function). We found that independent of PM2.5 mass vanadium (V) was associated with intercellular adhesion molecule-1 (ICAM-1) and vascular cell adhesion molecule-1 (VCAM-1). An IQR increase (3.2 ng/m(3)) in 2-day moving average V was associated with a 2.5% (95% CI: 1.2-3.8%) change in ICAM-1 and a 3.9% (95% CI: 2.2-5.7%) change in VCAM-1, respectively. In addition, an oil combustion source rich in V was linked to these adhesion molecules. People with higher allelic risk profiles related to oxidative stress may have greater associations (P-value of interaction=0.11). Our findings suggest that particles derived from oil combustion may be associated with inflammation and endothelial dysfunction, and it is likely that oxidative stress plays a role in the associations.

Asymptotic properties of adaptive group Lasso for sparse reduced rank regression

This paper studies the asymptotic properties of the penalized least squares estimator using an adaptive group Lasso penalty for the reduced rank regression. The group Lasso penalty is defined in the way that the regression coefficients corresponding to each predictor are treated as one group. It is shown that under certain regularity conditions, the estimator can achieve the minimax optimal rate of convergence. Moreover, the variable selection consistency can also be achieved, that is, the relevant predictors can be identified with probability approaching one. In the asymptotic theory, the number of response variables, the number of predictors and the rank number are allowed to grow to infinity with the sample size. Copyright © 2016 John Wiley & Sons, Ltd.

Bayesian analysis of penalized quantile regression for longitudinal data

This paper considers penalized quantile regression model for random effects longitudinal data from a Bayesian perspective. The introduction of a large number of individual random effects can significantly inflate the variability of estimates of other covariate effects. To modify this inflation effect a hierarchical Bayesian model is introduced to shrink the individual effects toward the common population values by using the Lasso and adaptive Lasso penalties in the quantile regression check function. A Gibbs sampling algorithm is developed to simulate the parameters from the posterior distributions. The simulation studies and real data analysis indicate that the proposed methods generally perform better in comparison to the other approaches.

Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models

In this paper, we study the model selection and structure specification for the generalised semi-varying coefficient models (GSVCMs), where the number of potential covariates is allowed to be larger than the sample size. We first propose a penalised likelihood method with the LASSO penalty function to obtain the preliminary estimates of the functional coefficients. Then, using the quadratic approximation for the local log-likelihood function and the adaptive group LASSO penalty (or the local linear approximation of the group SCAD penalty) with the help of the preliminary estimation of the functional coefficients, we introduce a novel penalised weighted least squares procedure to select the significant covariates and identify the constant coefficients among the coefficients of the selected covariates, which could thus specify the semiparametric modelling structure. The developed model selection and structure specification approach not only inherits many nice statistical properties from the local maximum likelihood estimation and nonconcave penalised likelihood method, but also computationally attractive thanks to the computational algorithm that is proposed to implement our method. Under some mild conditions, we establish the asymptotic properties for the proposed model selection and estimation procedure such as the sparsity and oracle property. We also conduct simulation studies to examine the finite sample performance of the proposed method, and finally apply the method to analyse a real data set, which leads to some interesting findings.

Regularized estimation for the least absolute relative error models with a diverging number of covariates

This paper considers the variable selection for the least absolute relative error (LARE) model, where the dimension of model, pn, is allowed to increase with the sample size n. Under some mild regular conditions, we establish the oracle properties, including the consistency of model selection and the asymptotic normality for the estimator of non-zero parameter. An adaptive weighting scheme is considered in the regularization, which admits the adaptive Lasso, SCAD and MCP penalties by linear approximation. The theoretical results allow the dimension diverging at the rate pn=o(n1/2) for the consistency and pn=o(n1/3) for the asymptotic normality. Furthermore, a practical variable selection procedure based on least squares approximation (LSA) is studied and its oracle property is also provided. Numerical studies are carried out to evaluate the performance of the proposed approaches.

Penalized estimation equation for an extended single-index model

The single-index model is a useful extension of the linear regression model. Cui et al. (Ann Stat 39:1658–1688, 2011) proposed an estimating function method for the estimation of index vector in an extended single-index model (ESIM). Nevertheless, how to conduct variable selection for ESIM has not been studied. To solve this problem, we penalize the estimating equation with some types of penalty, such as smoothly clipped absolute deviation penalty and adaptive lasso penalty. Under some regularity conditions, the oracle property is established, i.e., the resulting estimator can be as efficient as the oracle estimator, thus we improve the explanatory ability and accuracy of estimator for the ESIM. A novel algorithm is proposed to solve the penalized estimating equation by combining quasi-Fisher scoring type algorithm and MM algorithm. Simulation study and real data application demonstrate the excellent performance of the proposed estimators.

Use of the Adaptive LASSO Method to Identify PM2.5 Components Associated with Blood Pressure in Elderly Men: The Veterans Affairs Normative Aging Study.

BackgroundPM2.5 (particulate matter ≤ 2.5 μm) has been associated with adverse cardiovascular outcomes, but it is unclear whether specific PM2.5 components, particularly metals, may be responsible for cardiovascular effects.ObjectivesWe aimed to determine which PM2.5 components are associated with blood pressure in a longitudinal cohort.MethodsWe fit linear mixed-effects models with the adaptive LASSO penalty to longitudinal data from 718 elderly men in the Veterans Affairs Normative Aging Study, 1999–2010. We controlled for PM2.5 mass, age, body mass index, use of antihypertensive medication (ACE inhibitors, non-ophthalmic beta blockers, calcium channel blockers, diuretics, and angiotensin receptor antagonists), smoking status, alcohol intake, years of education, temperature, and season as fixed effects in the models, and additionally applied the adaptive LASSO method to select PM2.5 components associated with blood pressure. Final models were identified by the Bayesian Information Criterion (BIC).ResultsFor systolic blood pressure (SBP), nickel (Ni) and sodium (Na) were selected by the adaptive LASSO, whereas only Ni was selected for diastolic blood pressure (DBP). An interquartile range increase (2.5 ng/m3) in 7-day moving-average Ni was associated with 2.48-mmHg (95% CI: 1.45, 3.50 mmHg) increase in SBP and 2.22-mmHg (95% CI: 1.69, 2.75 mmHg) increase in DBP, respectively. Associations were comparable when the analysis was restricted to study visits with PM2.5 below the 75th percentile of the distribution (12 μg/m3).ConclusionsOur study suggested that exposure to ambient Ni was associated with increased blood pressure independent of PM2.5 mass in our study population of elderly men. Further research is needed to confirm our findings, assess generalizability to other populations, and identify potential mechanisms for Ni effects.CitationDai L, Koutrakis P, Coull BA, Sparrow D, Vokonas PS, Schwartz JD. 2016. Use of the adaptive LASSO method to identify PM2.5 components associated with blood pressure in elderly men: the Veterans Affairs Normative Aging Study. Environ Health Perspect 124:120–125; http://dx.doi.org/10.1289/ehp.1409021

Variable selection and estimation for semi-parametric multiple-index models

In this paper, we propose a novel method to select significant variables and estimate the corresponding coefficients in multiple-index models with a group structure. All existing approaches for single-index models cannot be extended directly to handle this issue with several indices. This method integrates a popularly used shrinkage penalty such as LASSO with the group-wise minimum average variance estimation. It is capable of simultaneous dimension reduction and variable selection, while incorporating the group structure in predictors. Interestingly, the proposed estimator with the LASSO penalty then behaves like an estimator with an adaptive LASSO penalty. The estimator achieves consistency of variable selection without sacrificing the root-$n$ consistency of basis estimation. Simulation studies and a real-data example illustrate the effectiveness and efficiency of the new method.

Parametric and semiparametric reduced-rank regression with flexible sparsity

We consider joint rank and variable selection in multivariate regression. Previously proposed joint rank and variable selection approaches assume that different responses are related to the same set of variables, which suggests using a group penalty on the rows of the coefficient matrix. However, this assumption may not hold in practice and motivates the usual lasso ( l 1 ) penalty on the coefficient matrix. We propose to use the gradient-proximal algorithm to solve this problem, which is a recent development in optimization. We also present some theoretical results for the proposed estimator with the l 1 penalty. We then consider several extensions including adaptive lasso penalty, sparse group penalty, and additive models. The proposed methodology thus offers a much more complete set of tools in high-dimensional multivariate regression. Finally, we present numerical illustrations based on simulated and real data sets.

Sparse Semiparametric Nonlinear Model With Application to Chromatographic Fingerprints

Traditional Chinese herbal medications (TCHMs) are composed of a multitude of compounds and the identification of their active composition is an important area of research. Chromatography provides a visual representation of a TCHM sample’s composition by outputting a curve characterized by spikes corresponding to compounds in the sample. Across different experimental conditions, the location of the spikes can be shifted, preventing direct comparison of curves and forcing compound identification to be possible only within each experiment. In this article, we propose a sparse semiparametric nonlinear modeling framework for the establishment of a standardized chromatographic fingerprint. Data-driven basis expansion is used to model the common shape of the curves, while a parametric time warping function registers across individual curves. Penalized weighted least-squares with the adaptive lasso penalty provides a unified criterion for registration, model selection, and estimation. Furthermore, the adaptive lasso estimators possess attractive sampling properties. A back-fitting algorithm is proposed for estimation. Performance is assessed through simulation and we apply the model to chromatographic data of rhubarb collected from different experimental conditions and establish a standardized fingerprint as a first step in TCHM research.

Robust variable selection in modal varying-coefficient models with longitudinal

In this article we present a robust and efficient variable selection procedure by using modal regression for varying-coefficient models with longitudinal data. The new method is proposed based on basis function approximations and a group version of the adaptive LASSO penalty, which can select significant variables and estimate the non-zero smooth coefficient functions simultaneously. Under suitable conditions, we establish the consistency in variable selection and the oracle property in estimation. A simulation study and two real data examples are undertaken to assess the finite sample performance of the proposed variable selection procedure.

Simultaneous variable selection and parametric estimation for quantile regression

In this paper, variable selection techniques in the linear quantile regression model are mainly considered. Based on the penalized quantile regression model, a one-step procedure that can simultaneously perform variable selection and coefficient estimation is proposed. The proposed procedure has three distinctive features: (1) By considering quantile regression, the set of relevant variables can vary across quantiles, thus making it more flexible to model heterogeneous data; (2) The one-step estimator has nice properties in both theory and practice. By applying SCAD penalty (Fan and Li, 2001) and Adaptive-LASSO penalty (Zou, 2006), we establish the oracle property for the sparse quantile regression under mild conditions. Computationally, the one-step estimator is fast, dramatically reducing the computation cost; (3) We suggest a BIC-like tuning parameter selector for the penalized quantile regression and demonstrate the consistency of this criterion. That is to say the true model can be identified consistently based on the BIC-like criterion, making our one-step estimator more reliable practically. Monte Carlo simulation studies are conducted to examine the finite-sample performance of this procedure. Finally, we conclude with a real data analysis. The results are promising.

Using graphical adaptive lasso approach to construct transcription factor and microRNA's combinatorial regulatory network in breast cancer.

Discovering the regulation of cancer-related gene is of great importance in cancer biology. Transcription factors and microRNAs are two kinds of crucial regulators in gene expression, and they compose a combinatorial regulatory network with their target genes. Revealing the structure of this network could improve the authors' understanding of gene regulation, and further explore the molecular pathway in cancer. In this article, the authors propose a novel approach graphical adaptive lasso (GALASSO) to construct the regulatory network in breast cancer. GALASSO use a Gaussian graphical model with adaptive lasso penalties to integrate the sequence information as well as gene expression profiles. The simulation study and the experimental profiles verify the accuracy of the authors' approach. The authors further reveal the structure of the regulatory network, and explore the role of feedforward loops in gene regulation. In addition, the authors discuss the combinatorial regulatory effect between transcription factors and microRNAs, and select miR-155 for detailed analysis of microRNA's role in cancer. The proposed GALASSO approach is an efficient method to construct the combinatorial regulatory network. It also provides a new way to integrate different data sources and could find more applications in meta-analysis problem.

Risk Factor Selection in Rate Making: EM Adaptive LASSO for Zero‐Inflated Poisson Regression Models

Risk factor selection is very important in the insurance industry, which helps precise rate making and studying the features of high-quality insureds. Zero-inflated data are common in insurance, such as the claim frequency data, and zero-inflation makes the selection of risk factors quite difficult. In this article, we propose a new risk factor selection approach, EM adaptive LASSO, for a zero-inflated Poisson regression model, which combines the EM algorithm and adaptive LASSO penalty. Under some regularity conditions, we show that, with probability approaching 1, important factors are selected and the redundant factors are excluded. We investigate the finite sample performance of the proposed method through a simulation study and the analysis of car insurance data from SAS Enterprise Miner database.

Resampling-based efficient shrinkage method for non-smooth minimands

In many regression models, the coefficients are typically estimated by optimising an objective function with a U-statistic structure. Under such a setting, we propose a simple and general method for simultaneous coefficient estimation and variable selection. It combines an efficient quadratic approximation of the objective function with the adaptive lasso penalty to yield a piecewise-linear regularisation path which can be easily obtained from the fast lars–lasso algorithm. Furthermore, the standard asymptotic oracle properties can be established under general conditions without requiring the covariance assumption (Wang, H., and Leng, C. (2007), ‘Unified Lasso Estimation by Least Squares Approximation’, Journal of the American Statistical Association, 102, 1039–1048). This approach applies to many semiparametric regression problems. Three examples are used to illustrate the practical utility of our proposal. Numerical results based on simulated and real data are provided.

Variable selection in high-dimensional quantile varying coefficient models

In this paper, we propose a two-stage variable selection procedure for high dimensional quantile varying coefficient models. The proposed method is based on basis function approximation and LASSO-type penalties. We show that the first stage penalized estimator with LASSO penalty reduces the model from ultra-high dimensional to a model that has size close to the true model, but contains the true model as a valid sub model. By applying adaptive LASSO penalty to the reduced model, the second stage excludes the remained irrelevant covariates, leading to an estimator consistent in variable selection. A simulation study and the analysis of a real data demonstrate that the proposed method performs quite well in finite samples, with regard to dimension reduction and variable selection.

Variable selection in semiparametric transformation models for right-censored data

We study variable selection in general transformation models for right-censored data. The models studied can incorporate external time-varying covariates, and they include the proportional hazards model and the proportional odds model as special cases. We propose an estimation method that involves minimizing a weighted negative partial loglikelihood function plus an adaptive lasso penalty, with the initial values obtained from nonparametric maximum likelihood estimation. The objective function is parametric and convex, so the minimization is easy to implement. We show that our selection has oracle properties and that the estimator is semiparametrically efficient. We demonstrate the small-sample performance of the proposed method via simulations, and we use the method to analyse data from the Atherosclerosis Risk in Communities Study. Copyright 2013, Oxford University Press.

Shrinkage Estimation of Varying Covariate Effects Based On Quantile Regression.

Varying covariate effects often manifest meaningful heterogeneity in covariate-response associations. In this paper, we adopt a quantile regression model that assumes linearity at a continuous range of quantile levels as a tool to explore such data dynamics. The consideration of potential non-constancy of covariate effects necessitates a new perspective for variable selection, which, under the assumed quantile regression model, is to retain variables that have effects on all quantiles of interest as well as those that influence only part of quantiles considered. Current work on l1-penalized quantile regression either does not concern varying covariate effects or may not produce consistent variable selection in the presence of covariates with partial effects, a practical scenario of interest. In this work, we propose a shrinkage approach by adopting a novel uniform adaptive LASSO penalty. The new approach enjoys easy implementation without requiring smoothing. Moreover, it can consistently identify the true model (uniformly across quantiles) and achieve the oracle estimation efficiency. We further extend the proposed shrinkage method to the case where responses are subject to random right censoring. Numerical studies confirm the theoretical results and support the utility of our proposals.