Adaptive Lasso Method Research Articles

Return Predictability from Industry Network Effects: Evidence from Rolling Window Adaptive Lasso

The literature argues that industry network effects imply the predictability of industry returns, while the return predictability can estimate the industry network in reverse. The paper employs rolling window adaptive lasso regressions to test the robustness of return predictions, showing the inherent model instability of the industry network and raising caveats for using the adaptive lasso method to estimate the industry network with a long time series. The out-of-sample predictability is examined by creating long-short trading portfolios based on the predictions. With non-parametric tests, the paper discovers that the out-of-sample predictability exists but is primarily due to the momentum effect. The adaptive lasso portfolios perform significantly worse than the momentum portfolio before the adjustment of risks. Based on the multifactor models, the trading portfolios also fail to continuously generate significant alphas for risk-averse investors.

Accounting for grouped predictor variables or pathways in high-dimensional penalized Cox regression models

BackgroundThe standard lasso penalty and its extensions are commonly used to develop a regularized regression model while selecting candidate predictor variables on a time-to-event outcome in high-dimensional data. However, these selection methods focus on a homogeneous set of variables and do not take into account the case of predictors belonging to functional groups; typically, genomic data can be grouped according to biological pathways or to different types of collected data. Another challenge is that the standard lasso penalisation is known to have a high false discovery rate.ResultsWe evaluated different penalizations in a Cox model to select grouped variables in order to further penalize variables that, in addition to having a low effect, belong to a group with a low overall effect; and to favor the selection of variables that, in addition to having a large effect, belong to a group with a large overall effect. We considered the case of prespecified and disjoint groups and proposed diverse weights for the adaptive lasso method. In particular we proposed the product Max Single Wald by Single Wald weighting (MSW*SW) which takes into account the information of the group to which it belongs and of this biomarker. Through simulations, we compared the selection and prediction ability of our approach with the standard lasso, the composite Minimax Concave Penalty (cMCP), the group exponential lasso (gel), the Integrative L1-Penalized Regression with Penalty Factors (IPF-Lasso), and the Sparse Group Lasso (SGL) methods. In addition, we illustrated the methods using gene expression data of 614 breast cancer patients.ConclusionsThe adaptive lasso with the MSW*SW weighting method incorporates both the information in the grouping structure and the individual variable. It outperformed the competitors by reducing the false discovery rate without severely increasing the false negative rate.

Statistical Inference on Semiparametric Spatial Additive Model

There has been a growing interest on using nonparametric and semiparametric modelling techniques for the analysis of spatial data because of their powerfulness in extracting the underlying local patterns in the data. In this study, stimulated by the Boston house price data, we apply a semiparametric spatial additive model to incorporation of spatial e ects in regression models. For this semiparametric model, we develop a linear hypothesis test of parametric coecients as well as a test for the existence of the spatial e ects. For the problem of variable selection, the adaptive Lasso method was applied. Monte Carlo simulation studies are conducted to illustrate the finite sample performance of the proposed inference procedures. Finally, an application in Boston housing data is studied.

BP Neural Network Calculus in Economic Growth Modelling of the Group of Seven

In this paper, the adaptive lasso method is used to screen variables, and different neural network models of seven countries are established by choosing variables. Gross domestic product (GDP) is a function of land area in the country, cultivated land, population, enrollment rate, total capital formation, exports of goods and services, and the general government’s final consumption of collateral and broad money. Based on the empirical analysis of the above factors from 1973 to 2016, the results show that the BP neural network model has better performance based on multiple summary statistics, without increasing the number of parameters and better predicting short-term GDP. In addition, the change and the error of the model are small and have a certain reference value.

Adaptive k-class estimation in high-dimensional linear models

In this paper, we explore the k-class estimator in high-dimensional linear models with potential endogeneity issues which are very common in empirical economics studies. K-class estimator has the advantage of incorporating many popular estimators such as the OLS estimator, two stage least squares (2SLS) estimator, limited information maximum likelihood (LIML) estimator, etc., as k takes on different values. Our main innovations are: 1) In the newly proposed high-dimensional k-class estimator, we allow the value of k (hence the level of endogeneity) to be determined by the data. Therefore, our method is very useful in empirical studies where the researcher do not know the severity of endogeneity or the importance of variables in a very large pool of candidate covariates. 2) In this paper, the adaptive LASSO method is incorporated into the generalized k-class estimation for variable selection and coefficient estimation in both the structural and reduced form equations. We show that the adaptive LASSO type k-class estimator has oracle properties. In simulation studies, we show that our new estimator can choose the optimal k value as well as achieving the minimum MSE among a set of popular estimators in finite samples where the number of potential endogenous variable is large.

Two-step approach for assessing the health effects of environmental chemical mixtures: application to simulated datasets and real data from the Navajo Birth Cohort Study

BackgroundThere is increasing interest in examining the consequences of simultaneous exposures to chemical mixtures. However, a consensus or recommendations on how to appropriately select the statistical approach analyzing the health effects of mixture exposures which best aligns with study goals has not been well established. We recognize the limitations that existing methods have in effectively reducing data dimension and detecting interaction effects when analyzing chemical mixture exposures collected in high dimensional datasets with varying degrees of variable intercorrelations. In this research, we aim to examine the performance of a two-step statistical approach in addressing the analytical challenges of chemical mixture exposures using two simulated data sets, and an existing data set from the Navajo Birth Cohort Study as a representative case study.MethodsWe propose to use a two-step approach: a robust variable selection step using the random forest approach followed by adaptive lasso methods that incorporate both dimensionality reduction and quantification of the degree of association between the chemical exposures and the outcome of interest, including interaction terms. We compared the proposed method with other approaches including (1) single step adaptive lasso; and (2) two-step Classification and regression trees (CART) followed by adaptive lasso method.ResultsUtilizing simulated data sets and applying the method to a real-life dataset from the Navajo Birth Cohort Study, we have demonstrated good performance of the proposed two-step approach. Results from the simulation datasets indicated the effectiveness of variable dimension reduction and reliable identification of a parsimonious model compared to other methods: single-step adaptive lasso or two-step CART followed by adaptive lasso method.ConclusionsOur proposed two-step approach provides a robust way of analyzing the effects of high-throughput chemical mixture exposures on health outcomes by combining the strengths of variable selection and adaptive shrinkage strategies.

A Note on the Adaptive LASSO for Zero-Inflated Poisson Regression

We consider the problem of modelling count data with excess zeros using Zero-Inflated Poisson (ZIP) regression. Recently, various regularization methods have been developed for variable selection in ZIP models. Among these, EM LASSO is a popular method for simultaneous variable selection and parameter estimation. However, EM LASSO suffers from estimation inefficiency and selection inconsistency. To remedy these problems, we propose a set of EM adaptive LASSO methods using a variety of data-adaptive weights. We show theoretically that the new methods are able to identify the true model consistently, and the resulting estimators can be as efficient as oracle. The methods are further evaluated through extensive synthetic experiments and applied to a German health care demand dataset.

Robust Adaptive Lasso method for parameter's estimation and variable selection in high-dimensional sparse models.

High dimensional data are commonly encountered in various scientific fields and pose great challenges to modern statistical analysis. To address this issue different penalized regression procedures have been introduced in the litrature, but these methods cannot cope with the problem of outliers and leverage points in the heavy tailed high dimensional data. For this purppose, a new Robust Adaptive Lasso (RAL) method is proposed which is based on pearson residuals weighting scheme. The weight function determines the compatibility of each observations and downweight it if they are inconsistent with the assumed model. It is observed that RAL estimator can correctly select the covariates with non-zero coefficients and can estimate parameters, simultaneously, not only in the presence of influential observations, but also in the presence of high multicolliearity. We also discuss the model selection oracle property and the asymptotic normality of the RAL. Simulations findings and real data examples also demonstrate the better performance of the proposed penalized regression approach.

Financial ratios and bankruptcy predictions: An international evidence

We study bankruptcy prediction over the international market using Compustat Global database. First, we apply a popular variable selection method, adaptive LASSO (least absolute shrinkage and selection operator), to select a parsimonious set of default predictor variables. When different market infrastructures are presented, our empirical study demonstrates an advantage in forecasting defaults with the adaptively selected default predictor variables. Second, with selected default predictor covariates, we apply the discrete hazard model to examine the performance for the international market with time varying panel data over different prediction horizons. Our empirical study shows that for Japan market, three predictor variables, including Retained Earning/Total Asset, Total Debt/Total Asset and Current Liability/Sales are selected by adaptive-LASSO method. Such selection results are strikingly consistent for different sampling periods over different prediction horizons. The model using those three financial ratios alone demonstrates strong predictability in forecasting corporate default. For Japan market, the model with adaptive-LASSO selected variables shows superior out-of-sample predictive power over the Altman's Z-score model. On the other hand, for some Europe countries including UK, Germany and France, the equity ratio variable, Equity/Total Liability, is consistently selected across different prediction horizons, whereas, the rest of selected variables are mixed.

Adaptive LASSO model selection in a multiphase quantile regression

We propose a general adaptive LASSO method for a quantile regression model. Our method is very interesting when we know nothing about the first two moments of the model error. We first prove that the obtained estimators satisfy the oracle properties, which involves the relevant variable selection without using hypothesis test. Next, we study the proposed method when the (multiphase) model changes to unknown observations called change-points. Convergence rates of the change-points and of the regression parameter estimators in each phase are found. The sparsity of the adaptive LASSO quantile estimators of the regression parameters is not affected by the change-points estimation. If the number of phases is unknown, a consistent criterion is proposed. Numerical studies by Monte Carlo simulations show the performance of the proposed method, compared to other existing methods in the literature, for models with a single phase or for multiphase models. The adaptive LASSO quantile method performs better than known variable selection methods, as the least squared method with adaptive LASSO penalty, -method with LASSO-type penalty and quantile method with SCAD penalty.

Nonnegative adaptive lasso for ultra-high dimensional regression models and a two-stage method applied in financial modeling

This paper proposes the nonnegative adaptive lasso method for variable selection both in the classical fixed p setting (OLS initial estimator) and the ultra-high dimensional setting (root-n-consistent initial estimator). This method is an extension of the adaptive lasso with nonnegative constraint on the coefficients. It is shown to have asymptotic unbiasedness, asymptotic normality and variable selection consistency and its mean squared error decays fast too. Comparing with other procedures, nonnegative adaptive lasso satisfies oracle properties and can select the true variables under fewer assumptions. To get the solution of the nonnegative adaptive lasso, we extend the multiplicative approach for computing. This algorithm is valid for the general framework where the number of regression parameters p is allowed to very large. Simulations are performed to illustrate above results.The constrained index tracking problem in the stock market without short sales is studied in the empirical part. A two-stage method, nonnegative adaptive lasso+nonnegative LS, is applied in the financial modeling. The tracking results indicate that nonnegative adaptive lasso and the two-stage method can both get small tracking error and is successful in assets selection.

Prediction of Rural Residents’ Consumption Expenditure Based on Lasso and Adaptive Lasso Methods

When the variable of model is large, the Lasso method and the Adaptive Lasso method can effectively select variables. This paper prediction the rural residents’ consumption expenditure in China, based on respectively using the Lasso method and the Adaptive Lasso method. The results showed that both can effectively and accurately choose the appropriate variable, but the Adaptive Lasso method is better than the Lasso method in prediction accuracy and prediction error. It shows that in variable selection and parameter estimation, Adaptive Lasso method is better than the Lasso method.

Use of the Adaptive LASSO Method to Identify PM2.5 Components Associated with Blood Pressure in Elderly Men: The Veterans Affairs Normative Aging Study.

BackgroundPM2.5 (particulate matter ≤ 2.5 μm) has been associated with adverse cardiovascular outcomes, but it is unclear whether specific PM2.5 components, particularly metals, may be responsible for cardiovascular effects.ObjectivesWe aimed to determine which PM2.5 components are associated with blood pressure in a longitudinal cohort.MethodsWe fit linear mixed-effects models with the adaptive LASSO penalty to longitudinal data from 718 elderly men in the Veterans Affairs Normative Aging Study, 1999–2010. We controlled for PM2.5 mass, age, body mass index, use of antihypertensive medication (ACE inhibitors, non-ophthalmic beta blockers, calcium channel blockers, diuretics, and angiotensin receptor antagonists), smoking status, alcohol intake, years of education, temperature, and season as fixed effects in the models, and additionally applied the adaptive LASSO method to select PM2.5 components associated with blood pressure. Final models were identified by the Bayesian Information Criterion (BIC).ResultsFor systolic blood pressure (SBP), nickel (Ni) and sodium (Na) were selected by the adaptive LASSO, whereas only Ni was selected for diastolic blood pressure (DBP). An interquartile range increase (2.5 ng/m3) in 7-day moving-average Ni was associated with 2.48-mmHg (95% CI: 1.45, 3.50 mmHg) increase in SBP and 2.22-mmHg (95% CI: 1.69, 2.75 mmHg) increase in DBP, respectively. Associations were comparable when the analysis was restricted to study visits with PM2.5 below the 75th percentile of the distribution (12 μg/m3).ConclusionsOur study suggested that exposure to ambient Ni was associated with increased blood pressure independent of PM2.5 mass in our study population of elderly men. Further research is needed to confirm our findings, assess generalizability to other populations, and identify potential mechanisms for Ni effects.CitationDai L, Koutrakis P, Coull BA, Sparrow D, Vokonas PS, Schwartz JD. 2016. Use of the adaptive LASSO method to identify PM2.5 components associated with blood pressure in elderly men: the Veterans Affairs Normative Aging Study. Environ Health Perspect 124:120–125; http://dx.doi.org/10.1289/ehp.1409021

Variable selection for inhomogeneous spatial point process models

AbstractIn this work, we consider variable selection when modelling the intensity and clustering of inhomogeneous spatial point processes, integrating well‐known procedures in the respective fields of variable selection and spatial point process modelling to introduce a simple procedure for variable selection in spatial point process modelling. Specifically, we consider modelling spatial point data with Poisson, pairwise interaction and Neyman–Scott cluster models, and incorporate LASSO, adaptive LASSO, and elastic net regularization methods into the generalized linear model framework for fitting these point models. We perform simulation studies to explore the effectiveness of using each of the three‐regularization methods in our procedure. We then use the procedure in two applications, modelling the intensity and clustering of rainforest trees with soil and geographical covariates using a Neyman–Scott model, and of fast food restaurant locations in New York City with Census variables and school locations using a pairwise interaction model. The Canadian Journal of Statistics 43: 288–305; 2015 © 2015 Statistical Society of Canada

测量误差模型的自适应LASSO变量选择方法研究

This paper focuses on variable selection and parameter estimation for measurement error models via adaptive LASSO method. Firstly, the adaptive LASSO estimator for linear models and partially linear models are proposed when the covariates are measured with error. Under some regular conditions the asymptotic properties of the estimators are investigated, it is proved that the adaptive lasso estimator has the oracle properties with proper choices of tuning parameter. Moreover, the algorithms and choices of penalty parameter and bandwidth are discussed. Finally, a Monte Carlo simulation study and a real data analysis are conducted to assess the finite sample performance of the proposed variable selection procedure. The results show that the adaptive LASSO estimator behaves well.

Biomedical Informatics and Computational Biology for High-Throughput Data Analysis

The last few years have witnessed the emergence of several high-throughput (HTP) platforms that are based on the evolution of omic science through transcriptomics, proteomics, and metabolomics. With the huge data accumulation, the informatics and computational methods have become essential to understand the complexity of biomedical data. We lunched the special issue to address the demand for statistical models, biological omic data analyses, and meta-analysis of biomarker for complex diseases. Statistical models are always important for the understanding of complex data; in this issue, Y. Liang et al. proposed a new adaptive L1/2 shooting regularization method for variable selection based on the Cox's proportional hazards mode. Simulation and the real gene expression dataset analysis showed that the method is more accurate for variable selection than Lasso and adaptive Lasso methods. L. Tian et al. employed a nonlinear model to analyze time course gene expression data. They firstly developed a method for estimating the parameters in the nonlinear model and then utilized the model to perform the significance analysis of individually differentially expressed genes and clustering analysis of a set of gene expression profiles. The simulation and real-life biological data analyses showed that their methods outperform some existing methods. For the application of bioinformatics tools, Z. E et al. analyzed the gene expression in rice leaf blades at different temperatures. They analyzed the next-generation sequencing (NGS) datasets and characterized the transcription profiles of the samples in rice seedling leaf blades at 25°C and 30°C. Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) databases were applied to study the effect of the temperature. Finally, they observed that temperature markedly regulates several superfamilies of transcription factors, including bZIP, MYB, and WRKY. Chronobiology is a field of biology that studies biological rhythms. R. Lopes et al. reviewed the bioinformatics or computational systems biology methods, tools, and databases which are very helpful to the understanding of the patterns and biological rhythms found in living organisms and the review will benefit the community for the application of bioinformatics in their future researches. For the understanding of protein structure and structure-function relationship, two papers are selected in this issue. M. Shambhu et al. studied the surface accessibility of hydrophobic residues by considering their conservation score and knowledge of flanking regions. The accuracy of prediction is therefore improved. In the other work, D. Lu et al. applied molecular dynamics to investigate the effects of X-linked agammaglobulinaemia associated with amino acid mutations in pleckstrin homology (PH) domain on the binding of the domain with inositol 1,3,4,5-tetrakisphosphate (Ins(1,3,4,5)P4). The mutations were then classified as “functional mutations”, and “folding mutations.” The result provided new insights into the biological function of the Btk-PH domain and related mutation-causing diseases. Finally, two works in this special issue are medical studies. The first is the work by B. Lv et al. They analyzed the 347 studies for the diagnostic value of tumor necrosis factor-α (TNF-α) and found that the northern hemisphere group in the TNF-α test has higher sensitivity and specificity than those of southern hemisphere group. The second medical study is an association study. Z. Liu et al. reported the significantly association of NCK2 with opiates addiction in African-origin men by a comprehensive analysis of a dataset from the Study of Addiction: Genetics and Environment (SAGE). They employed both SNP and gene based methods of analysis and identified a strong and significant association between a SNP in the NCK2 gene on chromosome 2 with opiates addiction in African-origin men. Bairong Shen Jian Ma Jiajun Wang Junbai Wang

Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap

Zou [J. Amer. Statist. Assoc. 101 (2006) 1418-1429] proposed the Adaptive LASSO (ALASSO) method for simultaneous variable selection and estimation of the regression parameters, and established its oracle property. In this paper, we investigate the rate of convergence of the ALASSO estimator to the oracle distribution when the dimension of the regression parameters may grow to infinity with the sample size. It is shown that the rate critically depends on the choices of the penalty parameter and the initial estimator, among other factors, and that confidence intervals (CIs) based on the oracle limit law often have poor coverage accuracy. As an alternative, we consider the residual bootstrap method for the ALASSO estimators that has been recently shown to be consistent; cf. Chatterjee and Lahiri [J. Amer. Statist. Assoc. 106 (2011a) 608-625]. We show that the bootstrap applied to a suitable studentized version of the ALASSO estimator achieves second-order correctness, even when the dimension of the regression parameters is unbounded. Results from a moderately large simulation study show marked improvement in coverage accuracy for the bootstrap CIs over the oracle based CIs.

AdaptiveL1/2Shooting Regularization Method for Survival Analysis Using Gene Expression Data

A new adaptive L1/2 shooting regularization method for variable selection based on the Cox's proportional hazards mode being proposed. This adaptive L1/2 shooting algorithm can be easily obtained by the optimization of a reweighed iterative series of L1 penalties and a shooting strategy of L1/2 penalty. Simulation results based on high dimensional artificial data show that the adaptive L1/2 shooting regularization method can be more accurate for variable selection than Lasso and adaptive Lasso methods. The results from real gene expression dataset (DLBCL) also indicate that the L1/2 regularization method performs competitively.

Model Selection for Cox Models with Time‐Varying Coefficients

Summary Cox models with time-varying coefficients offer great flexibility in capturing the temporal dynamics of covariate effects on right-censored failure times. Because not all covariate coefficients are time varying, model selection for such models presents an additional challenge, which is to distinguish covariates with time-varying coefficient from those with time-independent coefficient. We propose an adaptive group lasso method that not only selects important variables but also selects between time-independent and time-varying specifications of their presence in the model. Each covariate effect is partitioned into a time-independent part and a time-varying part, the latter of which is characterized by a group of coefficients of basis splines without intercept. Model selection and estimation are carried out through a fast, iterative group shooting algorithm. Our approach is shown to have good properties in a simulation study that mimics realistic situations with up to 20 variables. A real example illustrates the utility of the method.

Adaptive Lasso Variable Selection for the Accelerated Failure Models

This article considers the adaptive lasso procedure for the accelerated failure time model with multiple covariates based on weighted least squares method, which uses Kaplan-Meier weights to account for censoring. The adaptive lasso method can complete the variable selection and model estimation simultaneously. Under some mild conditions, the estimator is shown to have sparse and oracle properties. We use Bayesian Information Criterion (BIC) for tuning parameter selection, and a bootstrap variance approach for standard error. Simulation studies and two real data examples are carried out to investigate the performance of the proposed method.