Lasso Estimator Research Articles

Penalized Sparse Covariance Regression with High Dimensional Covariates

Covariance regression offers an effective way to model the large covariance matrix with the auxiliary similarity matrices. In this work, we propose a sparse covariance regression (SCR) approach to handle the potentially high-dimensional predictors (i.e., similarity matrices). Specifically, we use the penalization method to identify the informative predictors and estimate their associated coefficients simultaneously. We first investigate the Lasso estimator and subsequently consider the folded concave penalized estimation methods (e.g., SCAD and MCP). However, the theoretical analysis of the existing penalization methods is primarily based on i.i.d. data, which is not directly applicable to our scenario. To address this difficulty, we establish the non-asymptotic error bounds by exploiting the spectral properties of the covariance matrix and similarity matrices. Then, we derive the estimation error bound for the Lasso estimator and establish the desirable oracle property of the folded concave penalized estimator. Extensive simulation studies are conducted to corroborate our theoretical results. We also illustrate the usefulness of the proposed method by applying it to a Chinese stock market dataset.

Variable selection for uncertain regression models based on elastic net method

On one hand, determining the exact number of explanatory variables in uncertain regression analysis is often challenging, as real-world scenarios are interconnected. On the other hand, it’s worth noting that even when the number of variables is determined, some variables may have minimal impact on the response variable, and retaining them would only increase the complexity of the model. Therefore, variable selection methods can help eliminate variables with minimal impact to enhance the simplicity and interpretability of the model. In light of this, this paper first introduces a novel variable selection strategy based on the elastic net method to address the variable selection problem in uncertain regression models. This method not only offers the flexibility to simplify uncertain regression analysis into Lasso estimation or Ridge estimation but also exhibits extremely high efficiency in variable selection. Secondly, to address the issue of determining tuning parameters, we introduce the cross-validation method to ensure optimal model performance. Furthermore, we propose an uncertain hypothesis testing method based on imprecise observational data, which can help us assess the suitability of estimated parameters. Finally, through a series of numerical examples, we validate the feasibility and effectiveness of this method.

Multi-Task Forecasting of the Realized Volatilities of Agricultural Commodity Prices

Motivated by the comovement of realized volatilities (RVs) of agricultural commodity prices, we study whether multi-task forecasting algorithms improve the accuracy of out-of-sample forecasts of 15 agricultural commodities during the sample period from July 2015 to April 2023. We consider alternative multi-task stacking algorithms and variants of the multivariate Lasso estimator. We find evidence of in-sample predictability but scarce evidence that multi-task forecasting improves out-of-sample forecasts relative to a classic univariate heterogeneous autoregressive (HAR)-RV model. This lack of systematic evidence of out-of-sample forecasting gains is corroborated by extensive robustness checks, including an in-depth study of the quantiles of the distributions of the RVs and subsample periods that account for increases in the total spillovers among the RVs. We also study an extended model that features the RVs of energy commodities and precious metals, but our conclusions remain unaffected. Besides offering important lessons for future research, our results are interesting for financial market participants, who rely on accurate forecasts of RVs when solving portfolio optimization and derivatives pricing problems, and policymakers, who need accurate forecasts of RVs when designing policies to mitigate the potential adverse effects of a rise in the RVs of agricultural commodity prices and the concomitant economic and political uncertainty.

A machine learning analysis of the value-added intellectual coefficient’s effect on firm performance

Purpose Recently, machine learning (ML) methods gained popularity in finance and accounting research as alternatives to econometric analysis. Their success in high-dimensional settings is promising as a cure for the shortcomings of econometric analysis. The purpose of this study is to prove further the relationship between intellectual capital (IC) efficiency and firm performance using ML methods. Design/methodology/approach This study used the double selection, partialing-out and cross-fit partialing-out LASSO estimators to analyze the IC efficiency’s linear and nonlinear effects on firm performance using a sample of 2,581 North American firms from 1999 to 2021. The value-added intellectual capital (VAIC) and its components are used as indicators of IC efficiency. Firm performance is measured by return on equity, return on assets and market-to-book ratio. Findings The findings revealed significant connections between IC measures and firm performance. First, the VAIC, as an aggregate measure, significantly impacts both firm profitability and value. When the VAIC is decomposed into its breakdowns, it is revealed that structural capital efficiency substantially affects firm value, and capital employed efficiency has the same function for firm profitability. In contrast to the prevalent belief in the area, human capital efficiency’s impact is found to be less important than the others. Nonlinearities are also detected in the relationships. Originality/value As ML tools are most recently introduced to the IC literature, only a few studies have used them to expand the current knowledge. However, none of these studies investigated the role of IC as a determinant of firm performance. The present study fills this gap in the literature by investigating the effect of IC efficiency on firm performance using supervised ML methods. It also provides a novel approach by comparing the estimation results of three LASSO estimators. To the best of the author’s knowledge, this is the first study that has used LASSO in IC research.

On learning sparse linear models from cross samples

The sample complexity of a sparse linear model where samples are dependent is studied in this paper. We consider a specific dependency structure of the samples which arises in some experimental designs such as drug sensitivity studies, where two sets of objects (drugs and cells) are sampled independently, and after crossing (making all possible combinations of drugs and cells), the resulting output (efficacy of drugs) is measured. We call these types of samples as “cross samples”. The dependency among such samples is strong, and existing theoretical studies are either inapplicable or fail to provide realistic bounds. We aim at analyzing the performance of the Lasso estimator where the underlying distributions are mixtures of Gaussians and the data dependency arises from the crossing procedure. Our theoretical results show that the performance of the Lasso estimator in case of cross samples follows that of the i.i.d. samples with differences in constant factors. Through numerical results, we observe a phase transition: When datasets are too small, the error for cross samples is much larger than for i.i.d. samples, but once the size is large enough, cross samples are nearly as useful as i.i.d. samples. Our theoretical analysis suggests that the transition threshold is governed by the level of sparsity of the true parameter vector being estimated.

House dust metagenome and pulmonary function in a US farming population

BackgroundChronic exposure to microorganisms inside homes can impact respiratory health. Few studies have used advanced sequencing methods to examine adult respiratory outcomes, especially continuous measures. We aimed to identify metagenomic profiles in house dust related to the quantitative traits of pulmonary function and airway inflammation in adults. Microbial communities, 1264 species (389 genera), in vacuumed bedroom dust from 779 homes in a US cohort were characterized by whole metagenome shotgun sequencing. We examined two overall microbial diversity measures: richness (the number of individual microbial species) and Shannon index (reflecting both richness and relative abundance). To identify specific differentially abundant genera, we applied the Lasso estimator with high-dimensional inference methods, a novel framework for analyzing microbiome data in relation to continuous traits after accounting for all taxa examined together.ResultsPulmonary function measures (forced expiratory volume in one second (FEV1), forced vital capacity (FVC), and FEV1/FVC ratio) were not associated with overall dust microbial diversity. However, many individual microbial genera were differentially abundant (p-value < 0.05 controlling for all other microbial taxa examined) in relation to FEV1, FVC, or FEV1/FVC. Similarly, fractional exhaled nitric oxide (FeNO), a marker of airway inflammation, was unrelated to overall microbial diversity but associated with differential abundance for many individual genera. Several genera, including Limosilactobacillus, were associated with a pulmonary function measure and FeNO, while others, including Moraxella to FEV1/FVC and Stenotrophomonas to FeNO, were associated with a single trait.ConclusionsUsing state-of-the-art metagenomic sequencing, we identified specific microorganisms in indoor dust related to pulmonary function and airway inflammation. Some were previously associated with respiratory conditions; others were novel, suggesting specific environmental microbial components contribute to various respiratory outcomes. The methods used are applicable to studying microbiome in relation to other continuous outcomes.3hzq1LfdnyTZVYvBm7QT8RVideo

Marginalized LASSO in the low-dimensional difference-based partially linear model for variable selection

The difference-based partially linear model is an appropriate regression model when both linear and nonlinear predictors are present in the data. However, when we want to optimize the weights using the difference-based method, the problem of variable selection can be difficult since low-variance predictors present a challenge. Therefore, this study aims to establish a novel methodology based on marginal theory to tackle such mixed relationships successfully, emphasizing variable selection in low dimensions. We suggest using a marginalized LASSO estimator with a penalty term that is not as severe and related to the difference order. As part of our numerical analysis of small sample performance, we undertake comprehensive simulation experiments to numerically demonstrate the strength of our proposed technique in estimation and prediction compared to the LASSO under a low-dimensional setup. This is done so that we can numerically demonstrate the strength of our proposed method in estimation and prediction. The bootstrapped method is utilized to evaluate how well our proposed prediction method performs when examining the King House dataset.

Variable selection for semivarying coefficient models via local averaging

AbstractThis study aims to provide novel insights into variable selection in the semivarying coefficient model. We focus on the problem of variable selection and screening for the constant coefficient part. A common approach in the existing literature is to infer the constant coefficients by transforming the problem into a linear model scenario, utilizing a fine estimator of the varying coefficients. In this paper, we propose an approximation method for the varying coefficient functions using local averaging, which is characterized by its simplicity, rough and computational efficiency. Additionally, we introduce an adaptive lasso estimator and a forward regression algorithm specifically designed for semivarying coefficient models. Theoretical and experimental results highlight the effectiveness of the local averaging method in extending variable selection techniques from the linear model to the semivarying coefficient model. Our proposed approaches demonstrate a significant improvement in inference speed compared with baseline methods, with little loss of asymptotic efficiency.

TAVI PACER - a contemporary risk score for pacemaker implantation after TAVI

Abstract Background The need for permanent pacemaker implantation (PPM) remains one of the most frequent complications of transcatheter aortic valve implantation (TAVI). The aim of this study was to develop a novel risk score for the prediction of PPM after TAVI using contemporary transcatheter heart valves. Purpose The TAVI PACER score is designed to optimize valve selection and implantation technique based on individual patient factors to reduce the need for PPM. Methods Between January 2019 and December 2020, 1039 patients underwent TAVI at Charité Universitätsmedizin Berlin. Patients with prior pacemaker implantation or valve-in-valve procedure were excluded. We retrospectively evaluated clinical, electrocardiographic, echocardiographic, computed tomographic, and periprocedural data and applied logistic regression for predicting pacemaker implantation. For variable selection, we ran regression with a Lasso estimator for optimizing the predictive quality of the selected variables. Additionally, we used multiple imputation with 30 imputed sets to deal with missing data. Results We included 840 patients (mean age 80.2 ± 9.1 years, 50.5% female), of whom 140 patients (16.6%) needed pacemaker implantation within 30 days post TAVI. We identified 11 relevant risk factors that were included in the final score. In a first step the TAVI PACER score calculates an individual risk for pacemaker implantation using preprocedural characteristics such as grade of mitral insufficiency, nicotine use, preexisting AV Block I° and RBBB, EF, LVEDD and BMI. In a second step the influence of intraprocedural aspects are added to illustrate the extent to which the risk for pacemaker implantation can change, depending on which type of valve, implantation depth and valve perimeter are chosen. Pacemaker implantation can be predicted with a sensitivity of 60% and specificity of 82%. Conclusion The TAVI PACER Score is an intuitive and useful tool to predict the risk for pacemaker implantation after TAVI before the intervention and could thus help to reduce the rate of pacemakers implanted.

A dual symmetric Gauss-Seidel technique-based proximal ADMM for robust fused lasso estimation

A dual symmetric Gauss-Seidel technique-based proximal ADMM for robust fused lasso estimation

Network Inference With the Lasso

Calculating confidence intervals and p-values of edges in networks is useful to decide their presence or absence and it is a natural way to quantify uncertainty. Since lasso estimation is often used to obtain edges in a network, and the underlying distribution of lasso estimates is discontinuous and has probability one at zero when the estimate is zero, obtaining p-values and confidence intervals is problematic. It is also not always desirable to use the lasso to select the edges because there are assumptions required for correct identification of network edges that may not be warranted for the data at hand. Here, we review three methods that either use a modified lasso estimate (desparsified or debiased lasso) or a method that uses the lasso for selection and then determines p-values without the lasso. We compare these three methods with popular methods to estimate Gaussian Graphical Models in simulations and conclude that the desparsified lasso and its bootstrapped version appear to be the best choices for selection and quantifying uncertainty with confidence intervals and p-values.

Multivariate realized volatility: an analysis via shrinkage methods for Brazilian market data

IntroductionRealized volatility analysis of assets in the Brazilian market within a multivariate framework is the focus of this study. Despite the success of volatility models in univariate scenarios, challenges arise due to increasing dimensionality of covariance matrices and lower asset liquidity in emerging markets.MethodsIn this study, we utilize intraday stock trading data from the Brazilian Market to compute daily covariance matrices using various specifications. To mitigate dimensionality issues in covariance matrix estimation, we implement penalization restrictions on coefficients through regressions with shrinkage techniques using Ridge, LASSO, or Elastic Net estimators. These techniques are employed to capture the dynamics of covariance matrices.ResultsComparison of covariance construction models is performed using the Model Confidence Set (MCS) algorithm, which selects the best models based on their predictive performance. The findings indicate that the method used for estimating the covariance matrix significantly impacts the selection of the best models. Additionally, it is observed that more liquid sectors demonstrate greater intra-sectoral dynamics.DiscussionWhile the results benefit from shrinkage techniques, the high correlation between assets presents challenges in capturing stock or sector idiosyncrasies. This suggests the need for further exploration and refinement of methods to better capture the complexities of volatility dynamics in emerging markets like Brazil.

Should Humans Lie to Machines? The Incentive Compatibility of Lasso and GLM Structured Sparsity Estimators

We consider situations where a user feeds her attributes to a machine learning method that tries to predict her best option based on a random sample of other users. The predictor is incentive-compatible if the user has no incentive to misreport her covariates. Focusing on the popular Lasso estimation technique, we borrow tools from high-dimensional statistics to characterize sufficient conditions that ensure that Lasso is incentive compatible in the asymptotic case. We extend our results to a new nonlinear machine learning technique, Generalized Linear Model Structured Sparsity estimators. Our results show that incentive compatibility is achieved if the tuning parameter is kept above some threshold in the case of asymptotics.

On Lasso and Slope drift estimators for Lévy-driven Ornstein–Uhlenbeck processes

On Lasso and Slope drift estimators for Lévy-driven Ornstein–Uhlenbeck processes

On the adaptive Lasso estimator of AR(p) time series with applications to INAR(p) and Hawkes processes

We investigate the consistency and the rate of convergence of the adaptive Lasso estimator for the parameters of linear AR(p) time series with a white noise which is a strictly stationary and ergodic martingale difference. Roughly speaking, we prove that (i) If the white noise has a finite second moment, then the adaptive Lasso estimator is almost sure consistent (ii) If the white noise has a finite fourth moment, then the error estimate converges to zero with the same rate as the regularizing parameters of the adaptive Lasso estimator. Such theoretical findings are applied to estimate the parameters of INAR(p) time series and to estimate the fertility function of Hawkes processes. The results are validated by some numerical simulations, which show that the adaptive Lasso estimator allows for a better balancing between bias and variance with respect to the Conditional Least Square estimator and the classical Lasso estimator.

Collective Anomaly Detection in High-Dimensional Var Models

There is increasing interest in detecting collective anomalies: potentially short periods of time where the features of data change before reverting back to normal behaviour. We propose a new method for detecting a collective anomaly in VAR models. Our focus is on situations where the change in the VAR coefficient matrix at an anomaly is sparse, i.e. a small number of entries of the VAR coefficient matrix change. To tackle this problem, we propose a test statistic for a local segment that is built on the lasso estimator of the change in model parameters. This enables us to detect a sparse change more efficiently and our lasso-based approach becomes especially advantageous when the anomalous interval is short. We show that the new procedure controls Type 1 error and has asymptotic power tending to one. The practicality of our approach is demonstrated through simulations and two data examples, involving New York taxi trip data and EEG data.

Hierarchical false discovery rate control for high-dimensional survival analysis with interactions

With the development of data collection techniques, analysis with a survival response and high-dimensional covariates has become routine. Here we consider an interaction model, which includes a set of low-dimensional covariates, a set of high-dimensional covariates, and their interactions. This model has been motivated by gene-environment (G-E) interaction analysis, where the E variables have a low dimension, and the G variables have a high dimension. For such a model, there has been extensive research on estimation and variable selection. Comparatively, inference studies with a valid false discovery rate (FDR) control have been very limited. The existing high-dimensional inference tools cannot be directly applied to interaction models, as interactions and main effects are not “equal”. In this article, for high-dimensional survival analysis with interactions, we model survival using the Accelerated Failure Time (AFT) model and adopt a “weighted least squares + debiased Lasso” approach for estimation and selection. A hierarchical FDR control approach is developed for inference and respect of the “main effects, interactions” hierarchy. The asymptotic distribution properties of the debiased Lasso estimators are rigorously established. Simulation demonstrates the satisfactory performance of the proposed approach, and the analysis of a breast cancer dataset further establishes its practical utility.

An Efficient Method for Variable Selection Based on Diagnostic-Lasso Regression

In contemporary statistical methods, robust regression shrinkage and variable selection have gained paramount significance due to the prevalence of datasets characterized by contamination and an abundance of variables, often categorized as ‘high-dimensional data’. The Least Absolute Shrinkage and Selection Operator (Lasso) is frequently employed in this context for both the model and selecting variables. However, no one has attempted to apply regression diagnostic measures to Lasso regression, despite its power and widespread practical use. This work introduces a combined Lasso and diagnostic technique to enhance Lasso regression modeling for high-dimensional datasets with multicollinearity and outliers. We utilize a diagnostic Lasso estimator (D-Lasso). The breakdown point of the proposed method is also discussed. Finally, simulation examples and analyses of real data are provided to support the conclusions. The results of the numerical examples demonstrate that the D-Lasso approach performs as well as, if not better than, the robust Lasso method based on the MM-estimator.

An explainable fused lasso regression model for handling high-dimensional fuzzy data

In machine learning, the fused lasso is a regularization technique that is used to handle problems where the underlying signal has some kind of structure. In this paper, we extend fused lasso estimation for regression models characterized by fuzzy predictors and fuzzy responses. We present the first fused lasso regression model that is able to handle and analyze high-dimensional fuzzy data. The proposed model provides feature selection, leads to an improved predictive performance, and ensures interpretable and explainable models, all while being computationally efficient. Moreover, it can recover the true signal more accurately, find solutions that are structurally sparse, and is robust to noise. We conduct comprehensive comparative analysis and demonstrate the practical applicability of the presented fuzzy regression model through simulation and real-life applications.

Bayesian adaptive Lasso estimation of large graphical model based on modified Cholesky decomposition

In this paper, based on the modified Cholesky decomposition of the precision matrix, we propose Bayesian adaptive Lasso estimation and maximum adaptive posterior estimation for graphical model. We also recover the graph by minimizing the decoupled shrinkage and selection loss function.