Lasso-type Penalty Research Articles

Quantile LASSO in arbitrage-free option markets

The option price function and the implied volatility surface are both key tools for the derivative pricing strategies and the financial market analysis. Modern and sophisticated methods are used but their credibility suffered due to the financial crisis in 2007–2010. Instead, a method based on a standard semiparametric smoothing is proposed and the overall complexity and robustness (with respect to various anomalies, such as bid-ask spreads, discrete ticks in price, non-synchronous trading, or even heavy tailed error distributions) is achieved by using the conditional quantile estimation. The overestimation and the sparsity principle are adopted to introduce additional flexibility and the LASSO-type penalty and the set of well-defined linear constraints are employed to produce the final estimate which complies with the arbitrage-free criteria dictated by the financial theory. The theoretical results of the model are discussed, finite sample properties are investigated via a simulation study and a practical application of the proposed method is illustrated for the Apple Inc. (AAPL) call options.

The LASSO on latent indices for regression modeling with ordinal categorical predictors

Many applications of regression models involve ordinal categorical predictors. Two common approaches for handling ordinal predictors are to form a set of dummy variables, or employ a two stage approach where dimension reduction is first applied and then the response is regressed against the predicted latent indices. Both approaches have drawbacks, with the former running into a high-dimensional problem especially if interactions are considered, while the latter separates the prediction of the latent indices from the construction of the regression model. To overcome these challenges, a new approach called the LASSO on Latent Indices (LoLI) for handling ordinal predictors in regression is proposed, which involves jointly constructing latent indices for each or for groups of ordinal predictors and modeling the response directly as a function of these. LoLI borrows strength from the response to more accurately predict the latent indices, leading to better estimation of the corresponding effects. Furthermore, LoLI incorporates a LASSO type penalty to perform hierarchical selection, with interaction terms selected only if both parent main effects are included. Simulations show that LoLI can outperform the dummy variable and two stage approaches in selection and prediction performance. Applying LoLI to an Australian household-based panel identified three dimensions of psychosocial workplace quality (job demands, stress, and security) which affect an individual’s mental health in an additive and pairwise interactive manner.

Robust and sparse k-means clustering for high-dimensional data

In real-world application scenarios, the identification of groups poses a significant challenge due to possibly occurring outliers and existing noise variables. Therefore, there is a need for a clustering method which is capable of revealing the group structure in data containing both outliers and noise variables without any pre-knowledge. In this paper, we propose a k-means-based algorithm incorporating a weighting function which leads to an automatic weight assignment for each observation. In order to cope with noise variables, a lasso-type penalty is used in an objective function adjusted by observation weights. We finally introduce a framework for selecting both the number of clusters and variables based on a modified gap statistic. The conducted experiments on simulated and real-world data demonstrate the advantage of the method to identify groups, outliers, and informative variables simultaneously.

Structure learning of sparse directed acyclic graphs incorporating the scale-free property

Directed acyclic graphs have been widely used to model the causal relationships among variables. Many existing works focus on $$l_1$$ based methods to induce sparsity. However, in addition to sparsity, studies on networks show that many real networks are scale-free, that is, the degree of the network follows a power-law. To capture the scale-free property, in this paper we propose a novel penalized likelihood method by employing a log 1-norm group penalty which is the composite of the well-known log-type and lasso-type penalty functions. We then design an efficient coordinate descent algorithm to solve the resulting nonconvex problem. Moreover, we establish the estimation consistency of the estimator under the setting where the error variances are fixed at an identical constant. Numerical studies are also conducted to demonstrate the merits of our method.

Order selection and sparsity in latent variable models via the ordered factor LASSO.

Generalized linear latent variable models (GLLVMs) offer a general framework for flexibly analyzing data involving multiple responses. When fitting such models, two of the major challenges are selecting the order, that is, the number of factors, and an appropriate structure for the loading matrix, typically a sparse structure. Motivated by the application of GLLVMs to study marine species assemblages in the Southern Ocean, we propose the Ordered Factor LASSO or OFAL penalty for order selection and achieving sparsity in GLLVMs. The OFAL penalty is the first penalty developed specifically for order selection in latent variable models, and achieves this by using a hierarchically structured group LASSO type penalty to shrink entire columns of the loading matrix to zero, while ensuring that non-zero loadings are concentrated on the lower-order factors. Simultaneously, individual element sparsity is achieved through the use of an adaptive LASSO. In conjunction with using an information criterion which promotes aggressive shrinkage, simulation shows that the OFAL penalty performs strongly compared with standard methods and penalties for order selection, achieving sparsity, and prediction in GLLVMs. Applying the OFAL penalty to the Southern Ocean marine species dataset suggests the available environmental predictors explain roughly half of the total covariation between species, thus leading to a smaller number of latent variables and increased sparsity in the loading matrix compared to a model without any covariates.

Sparse group lasso for multiclass functional logistic regression models

ABSTRACTSparsity-inducing penalties are useful tools for variable selection and are also effective for regression problems where the data are functions. We consider the problem of selecting not only variables but also decision boundaries in multiclass logistic regression models for functional data, using sparse regularization. The parameters of the functional logistic regression model are estimated in the framework of the penalized likelihood method with the sparse group lasso-type penalty, and then tuning parameters for the model are selected using the model selection criterion. The effectiveness of the proposed method is investigated through simulation studies and the analysis of a gene expression data set.

Identifying drug-pathway association pairs based on L2,1-integrative penalized matrix decomposition

BackgroundTraditional drug identification methods follow the “one drug-one target” thought. But those methods ignore the natural characters of human diseases. To overcome this limitation, many identification methods of drug-pathway association pairs have been developed, such as the integrative penalized matrix decomposition (iPaD) method. The iPaD method imposes the L1-norm penalty on the regularization term. However, lasso-type penalties have an obvious disadvantage, that is, the sparsity produced by them is too dispersive.ResultsTherefore, to improve the performance of the iPaD method, we propose a novel method named L2,1-iPaD to identify paired drug-pathway associations. In the L2,1-iPaD model, we use the L2,1-norm penalty to replace the L1-norm penalty since the L2,1-norm penalty can produce row sparsity.ConclusionsBy applying the L2,1-iPaD method to the CCLE and NCI-60 datasets, we demonstrate that the performance of L2,1-iPaD method is superior to existing methods. And the proposed method can achieve better enrichment in terms of discovering validated drug-pathway association pairs than the iPaD method by performing permutation test. The results on the two real datasets prove that our method is effective.

Systematic investigations of gene effects on both topologies and supports: An Echinococcus illustration.

In this paper, we propose a high performance computing toolbox implementing efficient statistical methods for the study of phylogenies. This toolbox, which implements logit models and LASSO-type penalties, gives a way to better understand, measure, and compare the impact of each gene on a global phylogeny. As an application, we study the Echinococcus phylogeny, which is often considered as a particularly difficult example. Mitochondrial and nuclear genomes (19 coding sequences) of nine Echinococcus species are considered in order to investigate the molecular phylogeny of this genus. First, we check that the 19 gene trees lead to 19 totally different unsupported topologies (a topology is the sister relationship when both branch lengths and supports are ignored in a phylogenetic tree), while using the 19 genes as a whole are not sufficient for estimating the phylogeny. In order to circumvent this issue and understand the impact of the genes, we computed 43,796 trees using combinations ranging from 13 to 19 genes. By doing so, 15 topologies are obtained. Four particular topologies, appearing more robust and frequent, are then selected for more precise investigation. Refining further our statistical analysis, a particularly robust topology is extracted. We also carefully demonstrate the influence of nuclear genes on the likelihood of the phylogeny.

Identifying drug-pathway association pairs based on L1L2,1-integrative penalized matrix decomposition.

The traditional methods of drug discovery follow the “one drug-one target” approach, which ignores the cellular and physiological environment of the action mechanism of drugs. However, pathway-based drug discovery methods can overcome this limitation. This kind of method, such as the Integrative Penalized Matrix Decomposition (iPaD) method, identifies the drug-pathway associations by taking the lasso-type penalty on the regularization term. Moreover, instead of imposing the L1-norm regularization, the L2,1-Integrative Penalized Matrix Decomposition (L2,1-iPaD) method imposes the L2,1-norm penalty on the regularization term. In this paper, based on the iPaD and L2,1-iPaD methods, we propose a novel method named L1L2,1-iPaD (L1L2,1-Integrative Penalized Matrix Decomposition), which takes the sum of the L1-norm and L2,1-norm penalties on the regularization term. Besides, we perform permutation test to assess the significance of the identified drug-pathway association pairs and compute the P-values. Compared with the existing methods, our method can identify more drug-pathway association pairs which have been validated in the CancerResource database. In order to identify drug-pathway associations which are not validated in the CancerResource database, we retrieve published papers to prove these associations. The results on two real datasets prove that our method can achieve better enrichment for identified association pairs than the iPaD and L2,1-iPaD methods.

Lasso-type estimation for covariate-adjusted linear model

ABSTRACTLasso is popularly used for variable selection in recent years. In this paper, lasso-type penalty functions including lasso and adaptive lasso are employed in simultaneously variable selection and parameter estimation for covariate-adjusted linear model, where the predictors and response cannot be observed directly and distorted by some observable covariate through some unknown multiplicative smooth functions. Estimation procedures are proposed and some asymptotic properties are obtained under some mild conditions. It deserves noting that under appropriate conditions, the adaptive lasso estimator correctly select covariates with nonzero coefficients with probability converging to one and that the estimators of nonzero coefficients have the same asymptotic distribution that they would have if the zero coefficients were known in advance, i.e. the adaptive lasso estimator has the oracle property in the sense of Fan and Li [6]. Simulation studies are carried out to examine its performance in finite sample situations and the Boston Housing data is analyzed for illustration.

A relative error-based approach for variable selection

The accelerated failure time model or the multiplicative regression model is well-suited to analyze data with positive responses. For the multiplicative regression model, the authors investigate an adaptive variable selection method via a relative error-based criterion and Lasso-type penalty with desired theoretical properties and computational convenience. With fixed or diverging number of variables in regression model, the resultant estimator achieves the oracle property. An alternating direction method of multipliers algorithm is proposed for computing the regularization paths effectively. A data-driven procedure based on the Bayesian information criterion is used to choose the tuning parameter. The finite-sample performance of the proposed method is examined via simulation studies. An application is illustrated with an analysis of one period of stock returns in Hong Kong Stock Exchange.

Bayesian model selection in logistic regression for the detection of adverse drug reactions.

Spontaneous adverse event reports have a high potential for detecting adverse drug reactions. However, due to their dimension, the analysis of such databases requires statistical methods. In this context, disproportionality measures can be used. Their main idea is to project the data onto contingency tables in order to measure the strength of associations between drugs and adverse events. However, due to the data projection, these methods are sensitive to the problem of coprescriptions and masking effects. Recently, logistic regressions have been used with a Lasso type penalty to perform the detection of associations between drugs and adverse events. On different examples, this approach limits the drawbacks of the disproportionality methods, but the choice of the penalty value is open to criticism while it strongly influences the results. In this paper, we propose to use a logistic regression whose sparsity is viewed as a model selection challenge. Since the model space is huge, a Metropolis-Hastings algorithm carries out the model selection by maximizing the BIC criterion. Thus, we avoid the calibration of penalty or threshold. During our application on the French pharmacovigilance database, the proposed method is compared to well-established approaches on a reference dataset, and obtains better rates of positive and negative controls. However, many signals (i.e., specific drug-event associations) are not detected by the proposed method. So, we conclude that this method should be used in parallel to existing measures in pharmacovigilance. Code implementing the proposed method is available at the following url: https://github.com/masedki/MHTrajectoryR.

Sparse factor regression via penalized maximum likelihood estimation

In factor regression model, the maximum likelihood estimation suffers from three disadvantages: (i) the maximum likelihood estimates are unavailable when the number of variables exceeds the number of observations, (ii) the rotation technique based on maximum likelihood estimates produces an insufficiently sparse loading matrix, and (iii) multicollinearity can occur when the estimates of unique variances (specific variances) are small because the regression coefficients are sensitive to the inverse of unique variances. To handle these problems, we propose a penalized maximum likelihood procedure. Specifically, we impose a lasso-type penalty on the factor loadings to improve the sparseness of the solution. We also introduce a penalty on unique variances, which (given the factor scores) corresponds to the ridge penalty on the regression coefficient. Theoretical properties from a prediction viewpoint of our procedure are discussed. The effectiveness of the procedure is investigated through Monte Carlo simulations. The utility of our procedure is demonstrated on real data collected by an online questionnaire.

Adaptive LASSO model selection in a multiphase quantile regression

We propose a general adaptive LASSO method for a quantile regression model. Our method is very interesting when we know nothing about the first two moments of the model error. We first prove that the obtained estimators satisfy the oracle properties, which involves the relevant variable selection without using hypothesis test. Next, we study the proposed method when the (multiphase) model changes to unknown observations called change-points. Convergence rates of the change-points and of the regression parameter estimators in each phase are found. The sparsity of the adaptive LASSO quantile estimators of the regression parameters is not affected by the change-points estimation. If the number of phases is unknown, a consistent criterion is proposed. Numerical studies by Monte Carlo simulations show the performance of the proposed method, compared to other existing methods in the literature, for models with a single phase or for multiphase models. The adaptive LASSO quantile method performs better than known variable selection methods, as the least squared method with adaptive LASSO penalty, -method with LASSO-type penalty and quantile method with SCAD penalty.

Variable selection in discrete survival models including heterogeneity.

Several variable selection procedures are available for continuous time-to-event data. However, if time is measured in a discrete way and therefore many ties occur models for continuous time are inadequate. We propose penalized likelihood methods that perform efficient variable selection in discrete survival modeling with explicit modeling of the heterogeneity in the population. The method is based on a combination of ridge and lasso type penalties that are tailored to the case of discrete survival. The performance is studied in simulation studies and an application to the birth of the first child.

CGMM LASSO-type estimator for the process of Ornstein–Uhlenbeck type

In this paper, we study the LASSO-type penalized CGMM (GMM with continuum of moment method) estimator for the process of Ornstein–Uhlenbeck type. This LASSO-type estimator is obtained by minimizing the summation of the CGMM object function and a LASSO-type penalty, which is included for model selection. In the proposed method, model selection and estimation are done simultaneously. Under some regularity conditions, the proposed estimator asymptotically follows a non-standard normal distribution (Caner, 2009). Simulation study shows that the proposed estimator correctly selects the true model much more frequently than the commonly used Bayesian Information Criterion (BIC) and Akaike Information Criterion (AIC).

Improved Variable Selection Algorithm Using a LASSO-Type Penalty, with an Application to Assessing Hepatitis B Infection Relevant Factors in Community Residents.

ObjectivesIn epidemiological studies, it is important to identify independent associations between collective exposures and a health outcome. The current stepwise selection technique ignores stochastic errors and suffers from a lack of stability. The alternative LASSO-penalized regression model can be applied to detect significant predictors from a pool of candidate variables. However, this technique is prone to false positives and tends to create excessive biases. It remains challenging to develop robust variable selection methods and enhance predictability.Material and methodsTwo improved algorithms denoted the two-stage hybrid and bootstrap ranking procedures, both using a LASSO-type penalty, were developed for epidemiological association analysis. The performance of the proposed procedures and other methods including conventional LASSO, Bolasso, stepwise and stability selection models were evaluated using intensive simulation. In addition, methods were compared by using an empirical analysis based on large-scale survey data of hepatitis B infection-relevant factors among Guangdong residents.ResultsThe proposed procedures produced comparable or less biased selection results when compared to conventional variable selection models. In total, the two newly proposed procedures were stable with respect to various scenarios of simulation, demonstrating a higher power and a lower false positive rate during variable selection than the compared methods. In empirical analysis, the proposed procedures yielding a sparse set of hepatitis B infection-relevant factors gave the best predictive performance and showed that the procedures were able to select a more stringent set of factors. The individual history of hepatitis B vaccination, family and individual history of hepatitis B infection were associated with hepatitis B infection in the studied residents according to the proposed procedures.ConclusionsThe newly proposed procedures improve the identification of significant variables and enable us to derive a new insight into epidemiological association analysis.

Functional Principal Component Analysis and Randomized Sparse Clustering Algorithm for Medical Image Analysis.

Due to the advancement in sensor technology, the growing large medical image data have the ability to visualize the anatomical changes in biological tissues. As a consequence, the medical images have the potential to enhance the diagnosis of disease, the prediction of clinical outcomes and the characterization of disease progression. But in the meantime, the growing data dimensions pose great methodological and computational challenges for the representation and selection of features in image cluster analysis. To address these challenges, we first extend the functional principal component analysis (FPCA) from one dimension to two dimensions to fully capture the space variation of image the signals. The image signals contain a large number of redundant features which provide no additional information for clustering analysis. The widely used methods for removing the irrelevant features are sparse clustering algorithms using a lasso-type penalty to select the features. However, the accuracy of clustering using a lasso-type penalty depends on the selection of the penalty parameters and the threshold value. In practice, they are difficult to determine. Recently, randomized algorithms have received a great deal of attentions in big data analysis. This paper presents a randomized algorithm for accurate feature selection in image clustering analysis. The proposed method is applied to both the liver and kidney cancer histology image data from the TCGA database. The results demonstrate that the randomized feature selection method coupled with functional principal component analysis substantially outperforms the current sparse clustering algorithms in image cluster analysis.

Wavelet-Based Weighted LASSO and Screening Approaches in Functional Linear Regression

One useful approach for fitting linear models with scalar outcomes and functional predictors involves transforming the functional data to wavelet domain and converting the data-fitting problem to a variable selection problem. Applying the LASSO procedure in this situation has been shown to be efficient and powerful. In this article, we explore two potential directions for improvements to this method: techniques for prescreening and methods for weighting the LASSO-type penalty. We consider several strategies for each of these directions which have never been investigated, either numerically or theoretically, in a functional linear regression context. We compare the finite-sample performance of the proposed methods through both simulations and real-data applications with both 1D signals and 2D image predictors. We also discuss asymptotic aspects. We show that applying these procedures can lead to improved estimation and prediction as well as better stability. Supplementary materials for this article are available online.

Nearest neighbor ensembles for functional data with interpretable feature selection

Functional data becomes increasingly common in many fields of application. Although much research has been done on functional regression and clustering approaches for chemometric data, so far few classification methods exist. This paper introduces an ensemble method for classification that inherently provides automatic and interpretable feature selection. It is designed for single as well as multiple functional (and non-functional) covariates. The ensemble members are posterior probability estimates that are based on a k-nearest-neighbor approach. The ensemble allows for feature selection by including members that are calculated from various semi-metrics used in the k-nearest-neighbor approach, where a particular semi-metric represents a specific curve feature. Each ensemble member, and thus each curve feature, is weighted by an unknown coefficient. These coefficients are estimated using a proper scoring rule with implicit Lasso-type penalty, such that some coefficients can be estimated to be exactly zero. Thus, the ensemble automatically provides feature selection, and also, in the case of multiple functional (and non-functional) covariates, variable selection. The selection performance and the interpretability of the coefficients are investigated in simulation studies. Data of a cell chip used for water quality monitoring experiments is examined. Here, the relevance of especially the feature selection aspect of the ensemble is illustrated.