Group Sparsity Research Articles

Hyperspectral Anomaly Detection Based on Adaptive Low-Rank Transformed Tensor.

Hyperspectral anomaly detection, which is aimed at distinguishing anomaly pixels from the surroundings in spatial features and spectral characteristics, has attracted considerable attention due to its various applications. In this article, we propose a novel hyperspectral anomaly detection algorithm based on adaptive low-rank transform, in which the input hyperspectral image (HSI) is divided into a background tensor, an anomaly tensor, and a noise tensor. To take full advantage of the spatial-spectral information, the background tensor is represented as the product of a transformed tensor and a low-rank matrix. The low-rank constraint is imposed on frontal slices of the transformed tensor to depict the spatial-spectral correlation of the HSI background. Besides, we initialize a matrix with predefined size and then minimize its l2.1 -norm to adaptively derive an appropriate low-rank matrix. The anomaly tensor is constrained with the l2.1.1 -norm to depict the group sparsity of anomalous pixels. We integrate all regularization terms and a fidelity term into a non-convex problem and develop a proximal alternating minimization (PAM) algorithm to solve it. Interestingly, the sequence generated by the PAM algorithm is proven to converge to a critical point. Experimental results conducted on four widely used datasets demonstrate the superiority of the proposed anomaly detector over several state-of-the-art methods.

Sparse group principal component analysis using elastic-net regularisation and its application to virtual metrology in semiconductor manufacturing

Principal component analysis (PCA) is a widely used statistical technique for dimensionality reduction, extracting a low-dimensional subspace in which the variance is maximised (or the reconstruction error is minimised). To improve the interpretability of learned representations, several variants of PCA have recently been developed to estimate the principal components with a small number of input features (variable), such as sparse PCA and group sparse PCA. However, most existing methods suffer from either the requirement of measuring all the input variables or redundancy in the set of selected features. Another challenge for these methods is that they need to specify the sparsity level of the coefficient matrix in advance. To address the above issues, in this paper, we propose an elastic-net regularisation for sparse group PCA (ESGPCA), which incorporates sparsity constraints into the objective function to consider both within-group and between-group sparsities. Such a sparse learning approach allows us to automatically discover the sparse principal loading vectors without any prior assumption of the input features. We solve the non-smooth regularised problem using the alternating direction method of multipliers (ADMM), an efficient distributed optimisation technique. Empirical evaluations on both synthetic and real datasets demonstrate the effectiveness and promising performance of our sparse group PCA than other compared methods.

Continuous Exact Relaxation and Alternating Proximal Gradient Algorithm for Partial Sparse and Partial Group Sparse Optimization Problems

Continuous Exact Relaxation and Alternating Proximal Gradient Algorithm for Partial Sparse and Partial Group Sparse Optimization Problems

Hypergraph-Based Multitask Feature Selection with Temporally Constrained Group Sparsity Learning on fMRI

Localizing the brain regions affected by tasks is crucial to understanding the mechanisms of brain function. However, traditional statistical analysis does not accurately identify the brain regions of interest due to factors such as sample size, task design, and statistical effects. Here, we propose a hypergraph-based multitask feature selection framework, referred to as HMTFS, which we apply to a functional magnetic resonance imaging (fMRI) dataset to extract task-related brain regions. HMTFS is characterized by its ability to construct a hypergraph through correlations between subjects, treating each subject as a node to preserve high-order information of time-varying signals. Additionally, it manages feature selection across different time windows in fMRI data as multiple tasks, facilitating time-constrained group sparse learning with a smoothness constraint. We utilize a large fMRI dataset from the Human Connectome Project (HCP) to validate the performance of HMTFS in feature selection. Experimental results demonstrate that brain regions selected by HMTFS can provide higher accuracy for downstream classification tasks compared to other competing feature selection methods and align with findings from previous neuroscience studies.

Generalized nonconvex nonsmooth four-directional total variation with overlapping group sparsity for image restoration

In this paper, we address the challenge of image deblurring in the presence of Gaussian noise. To achieve high-quality image restoration, we introduce an optimization framework that integrates four-directional total variation regularization, overlapping group sparsity, and a range of nonconvex penalties. This novel model effectively mitigates the staircase artifacts associated with total variation regularization and enhances restoration quality by leveraging domain-specific information about image pixels. Compared to the ℓ1 norm, the nonconvex penalty applied to overlapping groups promotes sparsity in image gradients at the group level. To solve this nonconvex optimization problem, we propose a proximal alternating reweighted minimization algorithm, which has a proximal alternating scheme with a reweighted approximation of its subproblem. Theoretically, we establish that the sequences generated by the proposed algorithm converge to a critical point using the Kurdyka–Łojasiewicz property. Experimental results validate the superiority of our approach over competing methods.

A Generalized Formulation for Group Selection via ADMM

This paper studies a statistical learning model where the model coefficients have a pre-determined non-overlapping group sparsity structure. We consider a combination of a loss function and a regularizer to recover the desired group sparsity patterns, which can embrace many existing works. We analyze directional stationary solutions of the proposed formulation, obtaining a sufficient condition for a directional stationary solution to achieve optimality and establishing a bound of the distance from the solution to a reference point. We develop an efficient algorithm that adopts an alternating direction method of multiplier (ADMM), showing that the iterates converge to a directional stationary solution under certain conditions. In the numerical experiment, we implement the algorithm for generalized linear models with convex and nonconvex group regularizers to evaluate the model performance on various data types, noise levels, and sparsity settings.

Monitoring of group-structured high-dimensional processes via sparse group LASSO

In a general high-dimensional process, a large number of process parameters or quality characteristics is found to be featured through their dependencies and relevance. The features that have similar characteristics or behaviors in the process operation can be categorized into multiple groups. Thus, when a few quality characteristics in the process change, it is highly probable that the process shift would have occurred in a few relevant groups. Recently, several advanced statistical process control techniques are developed to monitor the changes in high-dimensional processes under sparsity. However, monitoring schemes that utilize the grouped pattern of the quality characteristics are sparse. This paper proposes a new method to monitor the high-dimensional process when the grouped structure of the process data is observed. The proposed method identifies the potentially changed groups and individual variables within the groups based on a modified sparse group LASSO (MSGL) model. Then, a monitoring statistic is obtained using MSGL-based likelihood function to test abnormality of the process. Extensive numerical studies are conducted to demonstrate the effectiveness and efficiency of the proposed method. In addition, a real-life application of a liquefied natural gas process is presented to illustrate the proposed method.

Intelligent Diagnosis of Compound Faults of Gearboxes Based on Periodical Group Sparse Model

A gearbox compound fault intelligent diagnosis method based on the period group sparse model is proposed for the problem that the fault features are coupled with each other and the fault components are superimposed on each other and difficult to be separated in the gearbox compound fault signal. Firstly, a binary sequence is constructed to embed the fault pulse period as a priori knowledge into the group sparse model to decouple and separate the composite fault signal while maintaining the amplitude and sparsity of the extracted features. Secondly, the wavelet packet energy features of the decoupled signals are extracted to improve the data quality while enhancing the characterization ability of the dictionary in the classification model. Finally, the wavelet packet energy features are imported into the sparse dictionary classification model, and the fault diagnosis is completed by outputting the fault categories using the self-driven characteristics of the data. The results show that the fault identification accuracy using the proposed method is 97%. In addition, the experimental validation under different states and working conditions with different rotational speeds allows the superiority and effectiveness of the algorithm in this paper to be tested and has the feasibility of a practical application in engineering.

Sparse Group Penalties for bi-level variable selection.

Many data sets exhibit a natural group structure due to contextual similarities or high correlations of variables, such as lipid markers that are interrelated based on biochemical principles. Knowledge of such groupings can be used through bi-level selection methods to identify relevant feature groups and highlight their predictive members. One of the best known approaches of this kind combines the classical Least Absolute Shrinkage and Selection Operator (LASSO) with the Group LASSO, resulting in the Sparse Group LASSO. We propose the Sparse Group Penalty (SGP) framework, which allows for a flexible combination of different SGL-style shrinkage conditions. Analogous to SGL, we investigated the combination of the Smoothly Clipped Absolute Deviation (SCAD), the Minimax Concave Penalty (MCP) and the Exponential Penalty (EP) with their group versions, resulting in the Sparse Group SCAD, the Sparse Group MCP, and the novel Sparse Group EP (SGE). Those shrinkage operators provide refined control of the effect of group formation on the selection process through a tuning parameter. In simulation studies, SGPs were compared with other bi-level selection methods (Group Bridge, composite MCP, and Group Exponential LASSO) for variable and group selection evaluated with the Matthews correlation coefficient. We demonstrated the advantages of the new SGE in identifying parsimonious models, but also identified scenarios that highlight the limitations of the approach. The performance of the techniques was further investigated in a real-world use case for the selection of regulated lipids in a randomized clinical trial.

Group sparse-based Taylor expansion method for liver pharmacokinetic parameters imaging of dynamic fluorescence molecular tomography

Objective. Pharmacokinetic parametric images obtained through dynamic fluorescence molecular tomography (DFMT) has ability of capturing dynamic changes in fluorescence concentration, thereby providing three-dimensional metabolic information for applications in biological research and drug development. However, data processing of DFMT is time-consuming, involves a vast amount of data, and the problem itself is ill-posed, which significantly limits the application of pharmacokinetic parametric images reconstruction. In this study, group sparse-based Taylor expansion method is proposed to address these problems. Approach. Firstly, Taylor expansion framework is introduced to reduce time and computational cost. Secondly, group sparsity based on structural prior is introduced to improve reconstruction accuracy. Thirdly, alternating iterative solution based on accelerated gradient descent algorithm is introduced to solve the problem. Main results. Numerical simulation and in vivo experimental results demonstrate that, in comparison to existing methods, the proposed approach significantly enhances reconstruction speed without a degradation of quality, particularly when confronted with background fluorescence interference from other organs. Significance. Our research greatly reduces time and computational cost, providing strong support for real-time monitoring of liver metabolism.

PIPE: Parallelized inference through ensembling of residual quantization expansions

Deep neural networks (DNNs) are ubiquitous in computer vision and natural language processing, but suffer from high inference cost. This problem can be addressed by quantization, which consists in converting floating point operations into a lower bit-width format. With the growing concerns on privacy rights, we focus our efforts on data-free methods. However, such techniques suffer from their lack of adaptability to the target devices, as a hardware typically only support specific bit widths. Thus, to adapt to a variety of devices, a quantization method shall be flexible enough to find good accuracy v.s. speed trade-offs for every bit width and target device. To achieve this, we propose PIPE, a quantization method that leverages residual error expansion, along with group sparsity and an ensemble approximation for better parallelization. PIPEis backed off by strong theoretical guarantees and achieves superior performance on every benchmarked application (from vision to NLP tasks), architecture (ConvNets, transformers) and bit-width (from int8 to ternary quantization).

Motion-robust free-running volumetric cardiovascular MRI.

To present and assess an outlier mitigation method that makes free-running volumetric cardiovascular MRI (CMR) more robust to motion. The proposed method, called compressive recovery with outlier rejection (CORe), models outliers in the measured data as an additive auxiliary variable. We enforce MR physics-guided group sparsity on the auxiliary variable, and jointly estimate it along with the image using an iterative algorithm. For evaluation, CORe is first compared to traditional compressed sensing (CS), robust regression (RR), and an existing outlier rejection method using two simulation studies. Then, CORe is compared to CS using seven three-dimensional (3D) cine, 12 rest four-dimensional (4D) flow, and eight stress 4D flow imaging datasets. Our simulation studies show that CORe outperforms CS, RR, and the existing outlier rejection method in terms of normalized mean square error and structural similarity index across 55 different realizations. The expert reader evaluation of 3D cine images demonstrates that CORe is more effective in suppressing artifacts while maintaining or improving image sharpness. Finally, 4D flow images show that CORe yields more reliable and consistent flow measurements, especially in the presence of involuntary subject motion or exercise stress. An outlier rejection method is presented and tested using simulated and measured data. This method can help suppress motion artifacts in a wide range of free-running CMR applications.

Robust sparse representation based on fitting error decomposition

Sparse representation (SR) of a signal aims at finding the minimum number of atoms for its representation. In several practical scenarios, the signal is vulnerable to outliers and thus robustness is required for SR based algorithms. However, most existing robust schemes are designed on the assumption that the anomalies are independently distributed, which may not perform well encountering complex interferences, especially the correlated ones. To deal with this drawback, a robust SR model is proposed, where both independent and correlated outliers are considered. Specifically, the fitting error is decomposed as the combination of a low-rank component and a sparse part, corresponding to the correlated gross error and independent outlier, respectively. Then, the group sparsity of the representation coefficient is utilized. Moreover, ℓ0-norm and ℓ2,0-norm are adopted as the sparsity regularization for the sparse outlier and representation coefficients, respectively. The solutions to ℓ0/ℓ2,0-norm minimization are generated by the hard-thresholding strategy, where the decision threshold is adaptively determined using median absolute deviation operator. As for the low-rank regularization, due to the NP-hardness of rank minimization, we employ matrix logarithmic norm as the rank surrogate to lessen the approximation gap. Finally, we apply the proposed model to face recognition task, and the excellent performance demonstrates its effectiveness.

Group sparsity residual constraint model with weighted log-sum penalty for image restoration

In the field of image restoration, non-convex penalties and non-local self-similarity (NSS) prior have been found to enhance image edges and textures. However, NSS-based methods often overlook the inherent variations within patch groups by treating them as a single entity for processing. Despite the existence of numerous convex or non-convex regularizers, few of them have been able to adaptively adjust penalty strength based on NSS information in order to preserve fine details in an image. To address this issue, we propose a novel group sparsity residual constraint model with weighted log-sum penalty (GSRC-Log) for image restoration in this paper. The proposed regularization term is parameterized by an adaptive scalar parameter that relates to NSS, allowing the model to prioritize high NSS group sparse coefficients while penalizing low NSS coefficients. For image denoising, we provide detailed analysis on the optimal condition of our proposed model and prove that it has a simple closed-form global solution. For image restoration tasks, we utilize the ADMM algorithm to solve the GSRC-Log problem and empirically verify its convergence. Extensive experimental results on both image denoising and compressive sensing (CS) recovery demonstrate that our proposed GSRC-Log outperforms many popular or state-of-the-art methods.

Spatial Invariant Tensor Self-Representation Model for Hyperspectral Anomaly Detection.

With the development of hyperspectral imaging technology, the hyperspectral anomaly has attracted considerable attention due to its significant role in many applications. Hyperspectral images (HSIs) with two spatial dimensions and one spectral dimension are intrinsically three-order tensors. However, most of the existing anomaly detectors were designed after converting the 3-D HSI data into a matrix, which destroys the multidimension structure. To solve this problem, in this article, we propose a spatial invariant tensor self-representation (SITSR) hyperspectral anomaly detection algorithm, which is derived based on the tensor-tensor product (t-product) to preserve the multidimension structure and achieve a comprehensive description of the global correlation of HSIs. Specifically, we exploit the t-product to integrate spectral information and spatial information, and the background image of each band is represented as the sum of the t-product of all bands and their corresponding coefficients. Considering the directionality of the t-product, we utilize two tensor self-representation methods with different spatial modes to obtain a more balanced and informative model. To depict the global correlation of the background, we merge the unfolding matrices of two representative coefficients and constrain them to lie in a low-dimensional subspace. Moreover, the group sparsity of anomaly is characterized by l2.1.1 norm regularization to promote the separation of background and anomaly. Extensive experiments conducted on several real HSI datasets demonstrate the superiority of SITSR compared with state-of-the-art anomaly detectors.

Group Variable Selection via Group Sparse Neural Network

Group variable selection is an important issue in high-dimensional data modeling and most of existing methods consider only the linear model. Therefore, a new method based on the deep neural network (DNN), an increasingly popular nonlinear method in both statistics and deep learning communities, is proposed. The method is applicable to general nonlinear models, including the linear model as a special case. Specifically, a group sparse neural network (GSNN) is designed, where the definition of nonlinear group high-level features (NGHFs) is generalized to the network structure. A two-stage group sparse (TGS) algorithm is employed to induce group variables selection by performing group structure selection on the network. GSNN is promising for complex nonlinear systems with interactions and correlated predictors, overcoming the shortcomings of linear or marginal variable selection methods. Theoretical results on convergence and group-level selection consistency are also given. Simulations results and real data analysis demonstrate the superiority of our method.

Towards Multi-Mode Outlier Robust Tensor Ring Decomposition

Conventional Outlier Robust Tensor Decomposition (ORTD) approaches generally represent sparse outlier corruption within a specific mode. However, such an assumption, which may hold for matrices, proves inadequate when applied to high-order tensors. In the tensor domain, the outliers are prone to be corrupted in multiple modes simultaneously. Addressing this limitation, this study proposes a novel ORTD approach by recovering low-rank tensors contaminated by outliers spanning multiple modes. In particular, we conceptualize outliers within high-order tensors as latent tensor group sparsity by decomposing the corrupted tensor into a sum of multiple latent components, where each latent component is exclusive to outliers within a particular direction. Thus, it can effectively mitigate the outlier corruptions prevalent in high-order tensors across multiple modes. To theoretically guarantee recovery performance, we rigorously analyze a non-asymptotic upper bound of the estimation error for the proposed ORTD approach. In the optimization process, we develop an efficient alternate direction method of multipliers (ADMM) algorithm. Empirical validation of the approach's efficacy is undertaken through comprehensive experimentation.

Covariate-Dependent Clustering of Undirected Networks with Brain-Imaging Data

This article focuses on model-based clustering of subjects based on the shared relationships of subject-specific networks and covariates in scenarios when there are differences in the relationship between networks and covariates for different groups of subjects. It is also of interest to identify the network nodes significantly associated with each covariate in each cluster of subjects. To address these methodological questions, we propose a novel nonparametric Bayesian mixture modeling framework with an undirected network response and scalar predictors. The symmetric matrix coefficients corresponding to the scalar predictors of interest in each mixture component involve low-rankness and group sparsity within the low-rank structure. While the low-rank structure in the network coefficients adds parsimony and computational efficiency, the group sparsity within the low-rank structure enables drawing inference on network nodes and cells significantly associated with each scalar predictor. Being a principled Bayesian mixture modeling framework, our approach allows model-based identification of the number of clusters, offers clustering uncertainty in terms of the co-clustering matrix and presents precise characterization of uncertainty in identifying network nodes significantly related to a predictor in each cluster. Empirical results in various simulation scenarios illustrate substantial inferential gains of the proposed framework in comparison with competitors. Analysis of a real brain connectome dataset using the proposed method provides interesting insights into the brain regions of interest (ROIs) significantly related to creative achievement in each cluster of subjects. Supplementary material shows the convergence rate for the posterior predictive density of the proposed model, additional simulation examples with model misspecification, full conditional distributions to run the Markov chain Monte Carlo (MCMC) algorithm and also presents traceplots for various model parameters to demonstrate convergence of the MCMC algorithm.

Bayesian sparsity and class sparsity priors for dictionary learning and coding

Dictionary learning methods continue to gain popularity for the solution of challenging inverse problems. In the dictionary learning approach, the computational forward model is replaced by a large dictionary of possible outcomes, and the problem is to identify the dictionary entries that best match the data, akin to traditional query matching in search engines. Sparse coding techniques are used to guarantee that the dictionary matching identifies only few of the dictionary entries, and dictionary compression methods are used to reduce the complexity of the matching problem. In this article, we propose a work flow to facilitate the dictionary matching process. First, the full dictionary is divided into subdictionaries that are separately compressed. The error introduced by the dictionary compression is handled in the Bayesian framework as a modeling error. Furthermore, we propose a new Bayesian data-driven group sparsity coding method to help identify subdictionaries that are not relevant for the dictionary matching. After discarding irrelevant subdictionaries, the dictionary matching is addressed as a deflated problem using sparse coding. The compression and deflation steps can lead to substantial decreases of the computational complexity. The effectiveness of compensating for the dictionary compression error and using the novel group sparsity promotion to deflate the original dictionary are illustrated by applying the methodology to real world problems, the glitch detection in the LIGO experiment and hyperspectral remote sensing.

Automatic and interpretable prediction of the site of origin in outflow tract ventricular arrhythmias: machine learning integrating electrocardiograms and clinical data.

The treatment of outflow tract ventricular arrhythmias (OTVA) through radiofrequency ablation requires the precise identification of the site of origin (SOO). Pinpointing the SOO enhances the likelihood of a successful procedure, reducing intervention times and recurrence rates. Current clinical methods to identify the SOO are based on qualitative analysis of pre-operative electrocardiograms (ECG), heavily relying on physician's expertise. Although computational models and machine learning (ML) approaches have been proposed to assist OTVA procedures, they either consume substantial time, lack interpretability or do not use clinical information. Here, we propose an alternative strategy for automatically predicting the ventricular origin of OTVA patients using ML. Our objective was to classify ventricular (left/right) origin in the outflow tracts (LVOT and RVOT, respectively), integrating ECG and clinical data from each patient. Extending beyond differentiating ventricle origin, we explored specific SOO characterization. Utilizing four databases, we also trained supervised learning models on the QRS complexes of the ECGs, clinical data, and their combinations. The best model achieved an accuracy of 89%, highlighting the significance of precordial leads V1-V4, especially in the R/S transition and initiation of the QRS complex in V2. Unsupervised analysis revealed that some origins tended to group closer than others, e.g., right coronary cusp (RCC) with a less sparse group than the aortic cusp origins, suggesting identifiable patterns for specific SOOs.