Adaptive Lasso Penalty Research Articles

Distributed Least Product Relative Error estimation for semi-parametric multiplicative regression with massive data

Distributed systems has been widely used for massive data analysis, but few studies focus on multiplicative regression models. We consider a communication-efficient surrogate likelihood method using the Least Product Relative Error criterion for semi-parametric multiplicative models on massive datasets. The non-parametric component is efficiently handled via B-spline approximation. We derive the asymptotic properties for both parametric and non-parametric components, while the SCAD and adaptive Lasso penalty functions are developed and their oracle properties for variable selection are validated. Simulation studies and an application to an energy prediction dataset are used to demonstrate the effectiveness and practical utility of the proposed method.

When MIDAS Meets LASSO: The Power of Low-Frequency Variables in Forecasting Value-at-Risk and Expected Shortfall

Abstract We propose a new framework for the joint estimation and forecasting of Value-at-Risk (VaR) and Expected Shortfall (ES) that integrates low-frequency variables. By maximizing the Asymmetric Laplace likelihood function with an Adaptive Lasso penalty, the most informative variables are selected on a rolling-window basis. In the empirical analysis, realized volatility, term spread, and housing starts serve as the strongest predictors of future tail risk. The out-of-sample backtesting results demonstrate that our method significantly outperforms other benchmarks, and achieves minimum loss in the joint forecasting of both the one-day-ahead and multi-day-ahead extreme S&P500 VaR and ES.

Selecting the number of factors in approximate factor models using group variable regularization

. We propose a novel method for the estimation of the number of factors in approximate factor models. The model is based on a penalized maximum likelihood approach incorporating an adaptive hierarchical lasso penalty function that enables setting entire columns of the factor loadings matrix to zero, which corresponds to removing uninformative factors. Additionally, the model is capable of estimating weak factors by allowing for sparsity in the non zero columns. We prove that the proposed estimator consistently estimates the true number of factors. Simulation experiments reveal superior selection accuracies for our method in finite samples over existing approaches, especially for data with cross-sectional and serial correlations. In an empirical application on a large macroeconomic dataset, we show that the mean squared forecast errors of an approximate factor model are lower if the number of included factors is selected by our method.

On variable selection in a semiparametric AFT mixture cure model.

In clinical studies, one often encounters time-to-event data that are subject to right censoring and for which a fraction of the patients under study never experience the event of interest. Such data can be modeled using cure models in survival analysis. In the presence of cure fraction, the mixture cure model is popular, since it allows to model probability to be cured (called the incidence) and the survival function of the uncured individuals (called the latency). In this paper, we develop a variable selection procedure for the incidence and latency parts of a mixture cure model, consisting of a logistic model for the incidence and a semiparametric accelerated failure time model for the latency. We use a penalized likelihood approach, based on adaptive LASSO penalties for each part of the model, and we consider two algorithms for optimizing the criterion function. Extensive simulations are carried out to assess the accuracy of the proposed selection procedure. Finally, we employ the proposed method to a real dataset regarding heart failure patients with left ventricular systolic dysfunction.

Heterogeneous macroeconomic factors' effects on stocks across sizes, styles, and sectors in the South Korean market.

Knowledge of the key macroeconomic variables that influence stock volatility across capital sizes, styles, and sectors can provide clues for investment strategies and policy decisions. We use the GARCH-MIDAS model with feature selection to analyze Korean Benchmark Indices from 2009 to 2022. This study maximizes memory retention through an optimal fractional differentiated price series and uses an adaptive lasso penalty for feature selection. The housing price-sales index and realized volatility were consistently influential across most indices and sectors. The GARCH-MIDAS model, paired with these variables, significantly improves long-term stock volatility forecasts. This study underscores the need to monitor housing prices in South Korea because of their substantial effects on long-term stock volatility.

Robust Variable Selection with Exponential Squared Loss for the Spatial Error Model

With the widespread application of spatial data in fields like econometrics and geographic information science, the methods to enhance the robustness of spatial econometric model estimation and variable selection have become a central focus of research. In the context of the spatial error model (SEM), this paper introduces a variable selection method based on exponential square loss and the adaptive lasso penalty. Due to the non-convex and non-differentiable nature of this proposed method, convex programming is not applicable for its solution. We develop a block coordinate descent algorithm, decompose the exponential square component into the difference of two convex functions, and utilize the CCCP algorithm in combination with parabolic interpolation for optimizing problem-solving. Numerical simulations demonstrate that neglecting the spatial effects of error terms can lead to reduced accuracy in selecting zero coefficients in SEM. The proposed method demonstrates robustness even when noise is present in the observed values and when the spatial weights matrix is inaccurate. Finally, we apply the model to the Boston housing dataset.

Interpretation of a change point detection problem via adaptive lasso penalty and its application

Interpretation of a change point detection problem via adaptive lasso penalty and its application

Robust estimators of functional single index models for longitudinal data

Functional single index model is a powerful model for longitudinal data, which effectively reduces the dimension of covariates. Inspired by robust calibration of computer models, we propose a robust estimator based on Huber loss for functional single index model with fixed dimension parameter. Furthermore, we also consider the case of high dimension parameter, and proposed a robust L 1 estimator of the unknown parameter based on adaptive lasso penalty. Theoretical properties including the asymptotic and non asymptotic results have also been investigated. Numerical studies including simulated experiments and an application to AIDS data verify the validity of the proposed estimators.

Adaptive LASSO estimation for functional hidden dynamic geostatistical models

We propose a novel model selection algorithm based on a penalized maximum likelihood estimator (PMLE) for functional hidden dynamic geostatistical models (f-HDGM). These models employ a classic mixed-effect regression structure with embedded spatiotemporal dynamics to model georeferenced data observed in a functional domain. Thus, the regression coefficients are functions. The algorithm simultaneously selects the relevant spline basis functions and regressors that are used to model the fixed effects. In this way, it automatically shrinks to zero irrelevant parts of the functional coefficients or the entire function for an irrelevant regressor. The algorithm is based on an adaptive LASSO penalty function, with weights obtained by the unpenalised f-HDGM maximum likelihood estimators. The computational burden of maximisation is drastically reduced by a local quadratic approximation of the log-likelihood. A Monte Carlo simulation study provides insight in prediction ability and parameter estimate precision, considering increasing spatiotemporal dependence and cross-correlations among predictors. Further, the algorithm behaviour is investigated when modelling air quality functional data with several weather and land cover covariates. Within this application, we also explore some scalability properties of our algorithm. Both simulations and empirical results show that the prediction ability of the penalised estimates are equivalent to those provided by the maximum likelihood estimates. However, adopting the so-called one-standard-error rule, we obtain estimates closer to the real ones, as well as simpler and more interpretable models.

BAYESIAN ANALYSIS OF TOBIT QUANTILE REGRESSION WITH ADAPTIVE LASSO PENALTY IN HOUSEHOLD EXPENDITURE FOR CIGARETTE CONSUMPTION

Tobit Quantile Regression with Adaptive Lasso Penalty is a quantile regression model on censored data that adds Lasso's adaptive penalty to its parameter estimation. The estimation of the regression parameters is solved by Bayesian analysis. Parameters are assumed to follow a certain distribution called the prior distribution. Using the sample information along with the prior distribution, the conditional posterior distribution is searched using the Box-Tiao rule. Computational solutions are solved by the MCMC Gibbs Sampling algorithm. Gibbs Sampling can generate samples based on the conditional posterior distribution of each parameter in order to obtain a posterior joint distribution. Tobit Quantile Regression with Adaptive Lasso Penalty was applied to data on Household Expenditure for Cigarette Consumption in 2011. As a comparison for data analysis, Tobit Quantile Regression was used. The results of data analysis show that the Tobit Quantile Regression model with Adaptive Lasso Penalty is better than the Tobit Quantile Regression.

Partially functional linear quantile regression model and variable selection with censoring indicators MAR

In this paper, we study the quantile regression (QR) estimation for the partially functional linear model with the responses being right-censored and the censoring indicators being missing at random (MAR). Firstly, we construct the weighted QR estimators for both the infinite-dimensional slope function and the finite scalar parameters of the model by combining the methods of calibration, imputation and inverse probability weighting. Then, some asymptotic properties such as the convergence rate of the estimator for the slope function and the asymptotic distribution of the estimator for the finite scalar parameters are obtained respectively. Moreover, to select the scalar covariates in the model, we also give a variable selection procedure by the method of adaptive LASSO penalty and establish the oracle property of the proposed weighted penalized estimators simultaneously. Finally, some simulation studies and a real data analysis are carried out to show the performances of the proposed methods.

Composite quantile regression analysis of survival data with missing cause-of-failure information and its application to breast cancer clinical trial

The analysis of survival data can be challenging due to the presence of missing data. This paper proposes weighted composite quantile regression (CQR) for estimating a lot of quantile regression (QR) of survival data based on single-index coefficient model (SICM), which is a very general and flexible tool for exploring the relationship between response variable and a set of predictors. The statistical inference for SICM is considered when cause-of-failure information (censored or non-censored) is always observed. However, the cause-of-failure information may be missing at random (MAR) for various reasons. Regression calibration, imputation and inverse probability weighted approaches are applied to deal with the MAR assumption. The asymptotic normalities of the proposed estimators are established. Meanwhile, the oracle property of the variable selection based on adaptive LASSO penalty procedure is conducted. To assess the finite sample performance of the proposed estimators, simulation study with normal error and heavy-tail error are considered. As expected, the CQR estimators perform as good as the least-square estimators for normal error, and are more robust to heavy-tailed error. Finally, a breast cancer real data analysis is carried out to illustrate the proposed methodologies.

Penalty and Shrinkage Strategies Based on Local Polynomials for Right-Censored Partially Linear Regression.

This study aims to propose modified semiparametric estimators based on six different penalty and shrinkage strategies for the estimation of a right-censored semiparametric regression model. In this context, the methods used to obtain the estimators are ridge, lasso, adaptive lasso, SCAD, MCP, and elasticnet penalty functions. The most important contribution that distinguishes this article from its peers is that it uses the local polynomial method as a smoothing method. The theoretical estimation procedures for the obtained estimators are explained. In addition, a simulation study is performed to see the behavior of the estimators and make a detailed comparison, and hepatocellular carcinoma data are estimated as a real data example. As a result of the study, the estimators based on adaptive lasso and SCAD were more resistant to censorship and outperformed the other four estimators.

Sparse calibration based on adaptive lasso penalty for computer models

Computer model calibration is a method to identify the unknown parameters of computer models, which is attaining more and more attention now. Most of the existing articles develop the calibration procedure under the assumption that the sample size of the physical experiments is larger than the dimension of the calibration parameters, which would not be satisfied in practice. In this article, we propose a sparse estimator of the calibration parameters and its robust version based on adaptive lasso penalty with adapting to the sample size of the physical experiments and the dimension of the calibration parameters, and the proposed robust estimator can deal with the heavy-tailed error and outliers efficiently. Subsequently, we investigate the nonasymptotic properties of the proposed estimators and obtain an upper bound of error of the proposed estimators by the concentration inequalities. We conduct some numerical simulations and an application to composite fuselage simulation, which verify that the proposed estimators enjoy nice performance.

Classification of COVID19 Patients Using Robust Logistic Regression

Coronavirus disease 2019 (COVID19) has triggered a global pandemic affecting millions of people. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) causing the COVID-19 disease is hypothesized to gain entry into humans via the airway epithelium, where it initiates a host response. The expression levels of genes at the upper airway that interact with the SARS-CoV-2 could be a telltale sign of virus infection. However, gene expression data have been flagged as suspicious of containing different contamination errors via techniques for extracting such information, and clinical diagnosis may contain labelling errors due to the specificity and sensitivity of diagnostic tests. We propose to fit the regularized logistic regression model as a classifier for COVID-19 diagnosis, which simultaneously identifies genes related to the disease and predicts the COVID-19 cases based on the expression values of the selected genes. We apply a robust estimating methods based on the density power divergence to obtain stable results ignoring the effects of contamination or labelling errors in the data and compare its performance with respect to the classical maximum likelihood estimator with different penalties, including the LASSO and the general adaptive LASSO penalties.

Adaptive lasso variable selection method for semiparametric spatial autoregressive panel data model with random effects

This paper investigates variable selection in semiparametric spatial autoregressive panel data model with random effects. A penalized profile maximum-likelihood method is proposed with adaptive lasso penalty which achieves parameter estimation and variable selection at the same time. Under some regular conditions, we prove the theoretical properties of the estimators, including consistency and oracle property. In addition, we develop a feasible logarithm and carry out numerical simulations to examine the finite sample performance of this method. At last, a real data study about the investment influencing factors of the “Belt and Road” initiative is presented for illustration purpose.

Fast Lasso-type safe screening for Fine-Gray competing risks model with ultrahigh dimensional covariates.

The Fine-Gray proportional sub-distribution hazards (PSH) model is among the most popular regression model for competing risks time-to-event data. This article develops a fast safe feature elimination method, named PSH-SAFE, for fitting the penalized Fine-Gray PSH model with a Lasso (or adaptive Lasso) penalty. Our PSH-SAFE procedure is straightforward to implement, fast, and scales well to ultrahigh dimensional data. We also show that as a feature screening procedure, PSH-SAFE is safe in a sense that the eliminated features are guaranteed to be inactive features in the original Lasso (or adaptive Lasso) estimator for the penalized PSH model. We evaluate the performance of the PSH-SAFE procedure in terms of computational efficiency, screening efficiency and safety, run-time, and prediction accuracy on multiple simulated datasets and a real bladder cancer data. Our empirical results show that the PSH-SAFE procedure possesses desirable screening efficiency and safety properties and can offer substantially improved computational efficiency as well as similar or better prediction performance in comparison to their baseline competitors.

Exponential squared loss based robust variable selection of AR models

Time series analysis is widely used in the fields of economics, ecology and medicine. Robust variable selection procedures through penalized regression have been gaining increased attention. In our work, a robust penalized regression estimator based on exponential squared loss for autoregressive (AR) models is proposed and discussed. The objective model with adaptive Lasso penalty realizes variable selection and parameter estimation simultaneously. Under some regular conditions, we establish the asymptotic and “Oracle” properties of the proposed estimator. In particular, the induced non-convex and non-differentiable mathematical programming problem offers challenges for solving algorithms. To solve this problem efficiently, we specially design a block coordinate descent (BCD) algorithm equipped with concave-convex process (CCCP) and provide a convergence guarantee. Numerical simulation studies are carried out to show that the proposed method is particularly robust and applicable compared with some recent methods when there are different types of noise or different intensity of noise. Furthermore, an application on a dataset of daily minimum temperature in Melbourne over 1981–1990 is performed.

Identifying the Determinants of Crude Oil Market Volatility by the Multivariate GARCH-MIDAS Model

Many macro-level variables have been used in forecasting crude oil price volatility. This article aims to identify which variables have the greatest impact and give more accurate predictions. The GARCH-MIDAS model with variable selection enables us to incorporate many variables in a single model. By combining the log-likelihood function with adaptive lasso penalty, three most informative determinants have been identified, namely, macroeconomic uncertainty, financial uncertainty and default yield spread. Out-of-sample results show that using these three variables significantly improves prediction accuracy compared to baseline models. However, the variables widely studied by other scholars, such as the supply and demand of crude oil, industrial production index, etc., were not selected, indicating that the impact of these variables may be overestimated. When studying crude oil price volatility, macroeconomic and financial market uncertainties can be used as effective predictors for investors and market analysts. Crude oil market participants should focus on macroeconomic and financial market uncertainties to make risk management more efficient.

Logistic regression with adaptive sparse group lasso penalty and its application in acute leukemia diagnosis

Cancer diagnosis based on gene expression profile data has attracted extensive attention in computational biology and medicine. It suffers from three challenges in practical applications: noise, gene grouping, and adaptive gene selection. This paper aims to solve the above problems by developing the logistic regression with adaptive sparse group lasso penalty (LR-ASGL). A noise information processing method for cancer gene expression profile data is first presented via robust principal component analysis. Genes are then divided into groups by performing weighted gene co-expression network analysis on the clean matrix. By approximating the relative value of the noise size, gene reliability criterion and robust evaluation criterion are proposed. Finally, LR-ASGL is presented for simultaneous cancer diagnosis and adaptive gene selection. The performance of the proposed method is compared with the other four methods in three simulation settings: Gaussian noise, uniformly distributed noise, and mixed noise. The acute leukemia data are adopted as an experimental example to demonstrate the advantages of LR-ASGL in prediction and gene selection.