Group Lasso Method Research Articles

Study of GSO-GL based secondary source configuration optimization method for broadband sound field reproduction

Sound field reproduction is a method that uses multiple secondary sound sources to reproduce a predefined desired sound field in a listening area. The configuration of the secondary sound sources is a key factor affecting the reproduction performances. To optimize the secondary source configuration, this study proposes a new method that combines the Gram-Schmidt orthogonalization method with the group Lasso method, which is called GSO-GL method. Firstly, it selects a certain number of the secondary sources from all the alternative secondary sources using the Gram-Schmidt orthogonalization method. The excitation sources are then further selected from the secondary sources selected in the previous step using the group Lasso method. The sound field simulation from the 2D room model show that the GSO-GL method outperforms the Gram-Schmidt orthogonalization method or the group Lasso method alone, combining the high system stability of the Gram-Schmidt orthogonalization method with the high reproduction accuracy of the group Lasso method.

Identification of essential plasma protein using manifold regularized sparse group-lasso for prediction of Alzheimer’s disease

Accurate diagnose of Alzheimer’s disease (AD), especially in its early stage, is very important for the possible delay and early treatment of the disease. Many researches indicate that the change of the expression level of plasma proteins will lead to the emergence of AD, which creates a new focus on finding essential proteins that affects AD. Nearly all the existing models were constructed on the independent hypothesis, where each candidate protein is treated independently. However, proteomics studies suggest that the essential proteins in cells do not act independently under illness conditions. Instead, they perform biological functions in groups, where each group is composed of proteins with similar functions. Accordingly, we propose a manifold regularized sparse group-lasso (i.e., MSGL) method to identify essential proteins that will incur AD. Specifically, we firstly propose a novel method to calculate the similarity between pairs of proteins with the aid of Gene Ontology, and then use the spectral clustering algorithm to divide the proteins into groups. Next, we adopt a sparse group-lasso model to select proteins according to both intra-group sparsity and inter-group sparsity, which ensures only a small number of proteins in a few of groups will be selected. In addition, we also introduce a manifold-based Laplacian regularizer to preserve the data distribution information. Finally, support vector machine (SVM) is used to classify AD patients. The experimental results show that our method can achieve a classification accuracy of 97.5%, a sensitivity of 97.6%, and specificity of 97.4%, demonstrating great potential in classification of AD patients.

A Group MCP Approach for Structure Identification in Non-Parametric Accelerated Failure Time Additive Regression Model

In biomedical research, identifying genes associated with diseases is of paramount importance. However, only a small fraction of genes are related to specific diseases among the multitude of genes. Therefore, gene selection and estimation are necessary, and the accelerated failure time model is often used to address such issues. Hence, this article presents a method for structural identification and parameter estimation based on a non-parametric additive accelerated failure time model for censored data. Regularized estimation and variable selection are achieved using the Group MCP penalty method. The non-parametric component of the model is approximated using B-spline basis functions, and a group coordinate descent algorithm is employed for model solving. This approach effectively identifies both linear and nonlinear factors in the model. The Group MCP penalty estimation exhibits consistency and oracle properties under regularization conditions, meaning that the selected variable set tends to have a probability of approaching 1 and asymptotically includes the actual predictive factors. Numerical simulations and a lung cancer data analysis demonstrate that the Group MCP method outperforms the Group Lasso method in terms of predictive performance, with the proposed algorithm showing faster convergence rates.

Oracle inequalities for weighted group Lasso in high-dimensional Poisson regression model

This article considers the problem of estimating the high-dimensional Poisson regression model with group sparsity in the parameter vector using the weighted group Lasso method. Non asymptotic oracle inequalities measuring the convergence rate of the estimation and prediction errors are given, provided that the true regression coefficients and the covariates satisfy suitable conditions. The non asymptotic oracle inequalities imply that the weighted group lasso estimator is consistent. A comparison of the performance of our proposed weights with other adaptive weights in simulated data shows that the weighted group Lasso estimator provides a good sparse approximation to the true parameters.

A calibrated SVM based on weighted smooth [formula omitted] for Alzheimer’s disease prediction

Alzheimer’s disease (AD) is currently one of the mainstream senile diseases in the world. It is a key problem predicting the early stage of AD. Low accuracy recognition of AD and high redundancy brain lesions are the main obstacles. Traditionally, Group Lasso method can achieve good sparseness. But, redundancy inside group is ignored. This paper proposes an improved smooth classification framework which combines the weighted smooth GL1/2 (wSGL1/2) as feature selection method and a calibrated support vector machine (cSVM) as the classifier. wSGL1/2 can make intra-group and inner-group features sparse, in which the group weights can further improve the efficiency of the model. cSVM can enhance the speed and stability of model by adding calibrated hinge function. Before feature selecting, an anatomical boundary-based clustering, called as ac-SLIC-AAL, is designed to make adjacent similar voxels into one group for accommodating the overall differences of all data. The cSVM model is fast convergence speed, high accuracy and good interpretability on AD classification, AD early diagnosis and MCI transition prediction. In experiments, all steps are tested respectively, including classifiers’ comparison, feature selection verification, generalization verification and comparing with state-of-the-art methods. The results are supportive and satisfactory. The superior of the proposed model are verified globally. At the same time, the algorithm can point out the important brain areas in the MRI, which has important reference value for the doctor’s predictive work. The source code and data is available at http://github.com/Hu-s-h/c-SVMForMRI.

Study of Bayesian variable selection method on mixed linear regression models.

Variable selection has always been an important issue in statistics. When a linear regression model is used to fit data, selecting appropriate explanatory variables that strongly impact the response variables has a significant effect on the model prediction accuracy and interpretation effect. redThis study introduces the Bayesian adaptive group Lasso method to solve the variable selection problem under a mixed linear regression model with a hidden state and explanatory variables with a grouping structure. First, the definition of the implicit state mixed linear regression model is presented. Thereafter, the Bayesian adaptive group Lasso method is used to determine the penalty function and parameters, after which each parameter's specific form of the fully conditional posterior distribution is calculated. Moreover, the Gibbs algorithm design is outlined. Simulation experiments are conducted to compare the variable selection and parameter estimation effects in different states. Finally, a dataset of Alzheimer's Disease is used for application analysis. The results demonstrate that the proposed method can identify the observation from different hidden states, but the results of the variable selection in different states are obviously different.

Data-Driven Sensor Selection for Signal Estimation of Vertical Wheel Forces in Vehicles

Abstract Sensor selection is one of the key factors that dictate the performance of estimating vertical wheel forces in vehicle durability design. To select K most relevant sensors among S candidate ones that best fit the response of one vertical wheel force, it has (SK) possible choices to evaluate, which is not practical unless K or S is small. In order to tackle this issue, this paper proposes a data-driven method based on maximizing the marginal likelihood of the data of the vertical wheel force without knowing the dynamics of vehicle systems. Although the resulting optimization problem is a mixed-integer programming problem, it is relaxed to a convex problem with continuous variables and linear constraints. The proposed sensor selection method is flexible and easy to implement, and the hyper-parameters do not need to be tuned using additional validation data sets. The feasibility and effectiveness of the proposed method are verified using numerical examples and experimental data. In the results of different data sizes and model orders, the proposed method has better fitting performance than that of the group lasso method in the sense of the 2-norm based metric. Also, the computational time of the proposed method is much less than that of the enumeration-based method.

Sparse group selection and analysis of function-related residue for protein-state recognition.

Machine learning methods have helped to advance wide range of scientific and technological field in recent years, including computational chemistry. As the chemical systems could become complex with high dimension, feature selection could be critical but challenging to develop reliable machine learning based prediction models, especially for proteins as bio-macromolecules. In this study, we applied sparse group lasso (SGL) method as a general feature selection method to develop classification model for an allosteric protein in different functional states. This results into a much improved model with comparable accuracy (Acc) and only 28 selected features comparing to 289 selected features from a previous study. The Acc achieves 91.50% with 1936 selected feature, which is far higher than that of baseline methods. In addition, grouping protein amino acids into secondary structures provides additional interpretability of the selected features. The selected features are verified as associated with key allosteric residues through comparison with both experimental and computational works about the model protein, and demonstrate the effectiveness and necessity of applying rigorous feature selection and evaluation methods on complex chemical systems.

FUNCTION GROUP SELECTION OF SEMBUNG LEAVES (BLUMEA BALSAMIFERA) SIGNIFICANT TO ANTIOXIDANTS USING OVERLAPPING GROUP LASSO

Functional groups of sembung leaf metabolites can be detected using FTIR spectrometry by looking at the spectrum's shape from specific peaks that indicate the type of functional group of a compound. There were 35 observations and 1866 explanatory variables (wavelength) in this study. The number of explanatory variables more than the number of observations is high-dimensional data. One method that can be used to analyze high-dimensional data is penalized regression. The overlapping group lasso method is a development of the group-based penalized regression method that can solve the problem of selecting variable groups and members of overlapping groups of variables. The results of selecting the variable groups using the overlapping group lasso method found that the functional groups that were significant for the antioxidants of sembung leaves were C=C Unstructured, CN amide, Polyphenol, Sio2.

Time-varying Group Lasso Granger Causality Graph for High Dimensional Dynamic system

Feature selection is a crucial preprocessing step in data analysis and machine learning. Since causal relationships imply the underlying mechanism of a system, causality-based feature selection methods have gradually attracted great attentions. For a high dimensional system undergoing dynamic transformation, because of the non-stationarity and sample scarcity, modeling the causal structure among these features is difficult. In this paper, we propose a time-varying Granger causal networks to capture the causal relations underlying high dimensional time-varying vector autoregressive models with high order lagged dependence. A kernel reweighted group lasso method is proposed, which overcomes the limitations of sample scarcity and transforms the problem of Granger causal structural learning into a group variable selection problem. The asymptotic consistency of the proposed algorithm is proved. We apply the time-varying Granger causal networks to simulation experiments and real data in the financial market. The study demonstrates that the method provides an efficient tool to detect changes and analysis characters of causal dependency structure in network evolution.

Machine-Learning-Based Return Predictors and the Spanning Controversy in Macro-Finance

We propose a two-step machine learning algorithm—the Supervised Adaptive Group LASSO (SAGLasso) method—that is suitable for constructing parsimonious return predictors from a large set of macro variables. We apply this method to government bonds and a set of 917 macro variables and construct a new, transparent, and easy-to-interpret macro variable with significant out-of-sample predictive power for excess bond returns. This new macro factor, termed the SAGLasso factor, is a linear combination of merely 30 selected macro variables out of 917. Furthermore, it can be decomposed into three sublevel factors: a novel housing factor, an employment factor, and an inflation factor. Importantly, the predictive power of the SAGLasso factor is robust to bond yields, namely, the SAGLasso factor is not spanned by bond yields. Moreover, we show that the unspanned variation of the SAGLasso factor cannot be attributed to yield measurement error or macro measurement error. The SAGLasso factor therefore provides a potential resolution to the spanning controversy in the macro-finance literature. This paper was accepted by Haoxiang Zhu, finance. Funding: This work was supported by a grant from the Penn State Institute for Real Estate Studies. Supplemental Material: Data and the internet appendices are available at https://doi.org/10.1287/mnsc.2022.4386 .

A sparse fused group lasso regression model for fourier-transform infrared spectroscopic data with application to purity prediction in olive oil blends

The percentage of olive oil present in an oil blend is of interest in the quality control of oils sold to consumers. One way in which this can be measured is using infrared spectroscopy. The analysis of the resulting data is challenging due to the high-dimension of the data and multicollinearity caused by issues such as the similarities between the chemical constituents in vegetable oils. This paper develops a sparse fused group lasso model for simultaneous feature selection and model fitting on Fourier-transform infrared spectroscopic data, and applies it to the task of percentage purity prediction in oil blends. The arising optimization problem is solved via the alternating direction method of multipliers algorithm. The sparse fused group lasso method is seen to improve on the interpretability of the resultant models, while providing comparable predication performance. Most importantly, it provides a flexible model that can capture group structure and smoothness in the coefficient structure.

A clustering group lasso method for quantification of adulteration in black cumin seed oil using Fourier transform infrared spectroscopy

Black cumin seed oil (BCSO) contains a large number of bioactive compounds and thus has many medicinal health benefits and uses. Its high economic profits in the market lead to the frequent occurrence of adulterating this oil with cheaper edible oils such as grape seed oil, walnut oil. It is difficult to detect adulteration as the oil adulterant has similar physical characteristic and even similar chemical composition to the authentic oil. The development of an accurate and rapid analytical method using attenuated total reflection Fourier transform infrared (ATR-FTIR) spectroscopy is of essential importance for determination of authenticity of BCSO and quantification of oil adulterants. In this study, BCSO and grape seed oil (GSO) were mixed in various ratios to mimic the adulteration. A clustering group lasso method was developed by incorporating both the high correlation structure of spectral variables and the underlying group features into the model. Instead of assuming that groups are known a priori as does ordinary group lasso, the clustering group lasso infers groups of spectral features from the data and encourages spectral variables within a group to have a shared association with the response. The model using ATR-FTIR spectroscopy proved to be a powerful tool to quantify BCSO adulteration with high accuracy and can accurately predict the quantity of adulterant at levels as low as 5%. With a substantial reduction in number of spectral features, the clustering group lasso model shows a simple regression coefficient profile with improved interpretability as compared to the ordinary group lasso model and other penalized models. The spectral regions automatically selected for quantification of BCSO adulteration can be helpful for the interpretation of the major chemical constituents of BCSO regarding its anti-cancer and anti-inflammatory effects from a chemometric perspective.

PreCar_Deep：A deep learning framework for prediction of protein carbonylation sites based on Borderline-SMOTE strategy

Carbonylation is an irreversible post-translational modification of proteins and regulates various cellular physiological processes. Due to the limitations of experimental methods, it is necessary to predict carbonylation sites by computational methods. In this paper, a new prediction model of carbonylation, Precar_Deep, is proposed. First, six feature extraction methods are used to obtain the original feature space from the protein sequences. Then, the Group LASSO method is used to remove redundant information and the oversampling Borderline-SMOTE method is employed to balance the data to obtain a new feature space. Finally, the processed data is input into the deep learning framework constructed in this paper to predict the carbonylation sites, and the performance of the model is evaluated by using 10-fold cross-validation and independent test datasets. The AUC values of the four datasets are all more than 90%. The experimental results show that PreCar_Deep is superior to other existing models and is helpful to identify protein carbonylation sites. The source code and all datasets are available at https://github.com/QUST-SHULI/PreCar_Deep/.

A Bayesian group lasso classification for ADNI volumetrics data.

The primary objective of this paper is to develop a statistically valid classification procedure for analyzing brain image volumetrics data obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) in elderly subjects with cognitive impairments. The Bayesian group lasso method thereby proposed for logistic regression efficiently selects an optimal model with the use of a spike and slab type prior. This method selects groups of attributes of a brain subregion encouraged by the group lasso penalty. We conduct simulation studies for high- and low-dimensional scenarios where our method is always able to select the true parameters that are truly predictive among a large number of parameters. The method is then applied on dichotomous response ADNI data which selects predictive atrophied brain regions and classifies Alzheimer's disease patients from healthy controls. Our analysis is able to give an accuracy rate of 80% for classifying Alzheimer's disease. The suggested method selects 29 brain subregions. The medical literature indicates that all these regions are associated with Alzheimer's patients. The Bayesian method of model selection further helps selecting only the subregions that are statistically significant, thus obtaining an optimal model.

Modularity-Guided Functional Brain Network Analysis for Early-Stage Dementia Identification.

Function brain network (FBN) analysis has shown great potential in identifying brain diseases, such as Alzheimer's disease (AD) and its prodromal stage, namely mild cognitive impairment (MCI). It is essential to identify discriminative and interpretable features from function brain networks, so as to improve classification performance and help us understand the pathological mechanism of AD-related brain disorders. Previous studies usually extract node statistics or edge weights from FBNs to represent each subject. However, these methods generally ignore the topological structure (such as modularity) of FBNs. To address this issue, we propose a modular-LASSO feature selection (MLFS) framework that can explicitly model the modularity information to identify discriminative and interpretable features from FBNs for automated AD/MCI classification. Specifically, the proposed MLFS method first searches the modular structure of FBNs through a signed spectral clustering algorithm, and then selects discriminative features via a modularity-induced group LASSO method, followed by a support vector machine (SVM) for classification. To evaluate the effectiveness of the proposed method, extensive experiments are performed on 563 resting-state functional MRI scans from the public ADNI database to identify subjects with AD/MCI from normal controls and predict the future progress of MCI subjects. Experimental results demonstrate that our method is superior to previous methods in both tasks of AD/MCI identification and MCI conversion prediction, and also helps discover discriminative brain regions and functional connectivities associated with AD.

Feature level-based group lasso method for amnestic mild cognitive impairment diagnosis

Background and ObjectivePrevious studies have indicated that brain morphological measures change in patients with amnestic mild cognitive impairment (aMCI). However, most existing classification methods cannot take full advantage of these measures. In this study, we improve traditional multitask learning framework by fully considering the relevance among related tasks and supplementary information from other unrelated tasks at the same time. MethodsWe propose a feature level-based group lasso (FL-GL) method in which a feature represents the average value of each ROI for each measure. First, we design a correlation matrix in which each row represents the relationship among different measures for each ROI. And this matrix is used to guide the feature selection based on a group lasso framework. Then, we train specific support vector machine (SVM) classifiers with the selected features for each measure. Finally, a weighted voting strategy is applied to combine these classifiers for a final prediction of aMCI from normal control (NC). ResultsWe use the leave-one-out cross-validation strategy to verify our method on two datasets, the Xuan Wu Hospital dataset and the ADNI dataset. Compared with the traditional method, the results show that the classification accuracies can be improved by 6.12 and 4.92% with the FL-GL method on the two datasets. ConclusionsThe results of an ablation study indicated that feature level-based group sparsity term was the core of our method. So, considering correlation at the feature level could improve the traditional multitask learning framework and our FL-GL method obtained better classification performance of patients with MCI and NCs.

Robust penalized estimation via Welsch loss with group Lasso

Robust estimation is widely used for analyzing statistical inference. We investigate penalized robust estimation via Welsch loss function with group Lasso method in high-dimensional linear regression models with group structure in this paper. This penalty identifies the significant groups of predictor variables. Robust estimation with group Lasso has crucial meaning in that it accompanies the large p (the number of predictors) small n (sample size) problems. We present the updating algorithms for this group Lasso problem. Compared with other penalty functions, we carried out simulation studies in order to assess the performance of the proposed method and the real dataset was demonstrated numerically for illustration purpose.

Product demand estimation for vending machines using video surveillance data: A group-lasso method

Lost sales information has significant impacts on the estimation of product demand and substitution. However, the difficulties to recognize such information in real applications make it rarely used in research. In this paper, we come up with a novel group-lasso based product demand model using the lost sales information, which is extracted from the video surveillance data provided by the cooperate retailer. A group-lasso method is used to characterize the substitution behaviours among each pair of SKUs. Then, an alternating minimization algorithm whose efficiency and convergence have been proved is designed to solve the model. To evaluate the model, we provide comparative experiments between the proposed method, time series forecasting and a naive method by applying these models to a real data set. The experiment results show that the proposed model obtains 8% higher accuracy at the total sales level and forecasts more accurately at the SKU sales level as well, which demonstrate its superiority. Moreover, we define a pair of welfare functions to measure the social impacts from both the retailer’s and customer’s end.

Variable selection for functional linear models with strong heredity constraint

In this paper, we consider the variable selection problem in functional linear regression with interactions. Our goal is to identify relevant main effects and corresponding interactions associated with the response variable. Heredity is a natural assumption in many statistical models involving two-way or higher-order interactions. Inspired by this, we propose an adaptive group Lasso method for the multiple functional linear model that adaptively selects important single functional predictors and pairwise interactions while obeying the strong heredity constraint. The proposed method is based on the functional principal components analysis with two adaptive group penalties, one for main effects and one for interaction effects. With appropriate selection of the tuning parameters, the rates of convergence of the proposed estimators and the consistency of the variable selection procedure are established. Simulation studies demonstrate the performance of the proposed procedure and a real example is analyzed to illustrate its practical usage.