Group Sparsity Research Articles

Sparse Group Lasso: Optimal Sample Complexity, Convergence Rate, and Statistical Inference.

We study sparse group Lasso for high-dimensional double sparse linear regression, where the parameter of interest is simultaneously element-wise and group-wise sparse. This problem is an important instance of the simultaneously structured model - an actively studied topic in statistics and machine learning. In the noiseless case, matching upper and lower bounds on sample complexity are established for the exact recovery of sparse vectors and for stable estimation of approximately sparse vectors, respectively. In the noisy case, upper and matching minimax lower bounds for estimation error are obtained. We also consider the debiased sparse group Lasso and investigate its asymptotic property for the purpose of statistical inference. Finally, numerical studies are provided to support the theoretical results.

Structure Level Pruning of Efficient Convolutional Neural Networks with Sparse Group LASSO

Structure Level Pruning of Efficient Convolutional Neural Networks with Sparse Group LASSO

A Novel Group Recommendation Model With Two-Stage Deep Learning

Group recommendation has recently drawn a lot of attention to the recommender system community. Currently, several deep learning-based approaches are leveraged to learn preferences of groups for items and predict next items in which groups may be interested. Yet, their recommendation performance is still unsatisfactory due to sparse group–item interactions. To address this challenge, this study presents a novel model, called group recommendation model with two-stage deep learning (GRMTDL), which encompasses two sequential stages: 1) group representation learning (GRL) and 2) group preference learning (GPL). In GRL, we first construct an undirected tripartite graph over group–user–item interactions, and then employ it to accurately learn group semantic features through a spatial-based variational graph autoencoder network. While in GPL, we first introduce a dual PL-network that contains two structure-sharing subnetworks: 1) group PL-network employed for GPL and 2) user PL-network utilized for user preference learning. Then, we design a novel layered transfer learning (LTL) method to learn group preferences by alternately optimizing these two subnetworks. In particular, it can effectively absorb knowledge of user preferences into the process of GPL. Furthermore, extensive experiments on four real-world datasets demonstrate that the proposed GRMTDL model outperforms the state-of-the-art baselines for group recommendation.

Comparison of Some Group Variable Selection Methods in High Dimensional Multiple Linear Regression via Simulation Study

In many applications, covariates possess a grouping structure that can be incorporated into the analysis to select important groups as well as important members of those groups. In this paper, we reviewed some group variable selection methods in the penalized regression model. This paper investigates by comparing the performance of seven previously proposed group variable selection methods; the group Lasso estimates, the group Lasso net estimates, the sparse group Lasso estimates, the group scad estimates, the group scad net estimates, the group mcp estimates, and the group gel estimate via a simulation study. The simulation study is used in determining which methods are best in all of the linear regression scenarios.

Joint selection of brain network nodes and edges for MCI identification

Background and ObjectiveFunctional brain graph (FBG), by describing the interactions between different brain regions, provides an effective representation of fMRI data for identifying mild cognitive impairment (MCI), an early stage of Alzheimer's Disease (AD). Prior to the identification task, selecting features from the estimated FBG is a necessary step for reducing computational cost, alleviating the risk of overfitting, and finding potential biomarkers of brain diseases. In practice, either node-based features (e.g., local clustering coefficients) or edge-based features (e.g., adjacency weights) are generally considered in current studies. Despite their popularity, these schemes can only capture one granularity (node or edge) of information in the FBG, which might be insufficient for the classification task and the interpretation of the classification result. MethodsTo address this issue, in this paper, we propose to jointly select nodes and edges from the estimated FBGs. Specifically, we first assign the edges to different node groups. Then, sparse group least absolute shrinkage and selection operator (sgLASSO) is used to select groups (nodes) and edges in the groups towards a better classification performance. Such a technique enables us to simultaneously locate discriminative brain regions, as well as connections between these brain regions, making the classification results more interpretable. ResultsExperimental results show that the proposed method achieves better classification performance than state-of-the-art methods. Moreover, by exploring brain network “features” that contributed most to MCI identification, we discover potential biomarkers for MCI diagnosis. ConclusionA novel method for jointly selecting nodes and edges from the estimated functional brain graphs (FBGs) is proposed.

Raptors avoid the confusion effect by targeting fixed points in dense aerial prey aggregations

Collective behaviours are widely assumed to confuse predators, but empirical support for a confusion effect is often lacking, and its importance must depend on the predator’s targeting mechanism. Here we show that Swainson’s Hawks Buteo swainsoni and other raptors attacking swarming Mexican Free-tailed Bats Tadarida brasiliensis steer by turning towards a fixed point in space within the swarm, rather than by using closed-loop pursuit of any one individual. Any prey with which the predator is on a collision course will appear to remain on a constant bearing, so target selection emerges naturally from the geometry of a collision. Our results show how predators can simplify the demands on their sensory system by decoupling steering from target acquisition when capturing prey from a dense swarm. We anticipate that the same tactic will be used against flocks and schools across a wide range of taxa, in which case a confusion effect is paradoxically more likely to occur in attacks on sparse groups, for which steering and target acquisition cannot be decoupled.

Deep side group sparse coding network for image denoising

Recently, deep learning has made significant progress in image denoising. However, most of existing deep learning based methods are purely data-driven, without considering the knowledge of image denoising. Moreover, the parameters of deep denoising network are not explainable. According to these issues, this paper proposes a deep side group sparse coding network for image denoising, named a side group sparse coding (SGSC)-Net. First, SGSC model for image denoising by exploiting prior information regarding the group sparse coefficients consistency is developed. Specifically, the side information is constructed as the weighted combination of intermediate estimations, and updated iteratively. Then, the optimisation solution of SGSC model is turned into a deep neural network using deep unfolding, that is, SGSC-Net. The computational path of SGSC-Net fully follows the iterations of optimisation solution, and consequently the network parameters are interpretable. Furthermore, the design of SGSC-Net employs the insight of SGSC denoising model. The experimental results on well-known datasets quantitatively and qualitatively demonstrate that SGSC-Net is competitive to existing deep unfolding-based and typical deep neural network-based methods.

Multi-Circular SAR Three-Dimensional Image Formation via Group Sparsity in Adjacent Sub-Apertures

Multi-circular SAR(MCSAR) can obtain holographic three-dimensional (3D) images of interesting observation targets, which is a significant research field at present. For anisotropic scatterers, the multi-circular SAR incoherent 3D imaging strategy combines the principle of tomography SAR inversion to obtain better reconstruction results. However, in incoherent 3D imaging, the traditional L1-norm regularization method only considers sparse representation and reconstruction in the sub-aperture pixel unit along the elevation direction, and the target structural sparsity in the same pixel unit between adjacent sub-apertures is not considered. The hierarchical sparsity constraint method in multi-circular SAR 3D target reconstruction is proposed in this paper, the L1-norm regularization is used in sub-aperture elevation, and the L2,1-regularization-based group sparsity constraint is adopted in elevation of the adjacent sub-aperture. In this paper, the sparse group thresholding iterative method is proposed to reconstruct the 3D observed target image with MCSAR data. Compared with the traditional L1-norm regularization method and IAA-GLRT method for MCSAR target reconstruction in elevation, the proposed method in this paper could obtain clearer 3D observation target images, with fewer desultory points along the elevation direction. Detailed simulation data analysis and 3D imaging processing of real MCSAR data demonstrate the advantages of the proposed method.

Ultrasound images speckle noise removal by nonconvex hybrid overlapping group sparsity model

Ultrasound images speckle noise removal by nonconvex hybrid overlapping group sparsity model

BFS 10: a nascent bipolar H ii region in a filamentary molecular cloud

ABSTRACT We present a study of the compact blister H ii region BFS 10 and its highly filamentary molecular cloud. We utilize 12CO observations from the Five College Radio Astronomy Observatory to determine the distance, size, mass, and velocity structure of the molecular cloud. Infrared observations obtained from the UKIRT Infrared Deep Sky Survey and the Spitzer Infrared Array Camera, as well as radio continuum observations from the Canadian Galactic Plane Survey, are used to extract information about the central H ii region. This includes properties such as the ionizing photon rate and infrared luminosity, as well as identifying a rich embedded star cluster associated with the central O9 V star. Time-scales regarding the expansion rate of the H ii region and lifetime of the ionizing star reveal a high likelihood that BFS 10 will develop into a bipolar H ii region. Although the region is expected to become bipolar, we conclude from the cloud’s velocity structure that there is no evidence to support the idea that star formation at the location of BFS 10 was triggered by two colliding clouds. A search for embedded young stellar objects (YSOs) within the molecular cloud was performed. Two distinct regions of YSOs were identified: one region associated with the rich embedded cluster and another sparse group associated with an intermediate-mass YSO.

SLIC-Based Cloud Removal Approach with Inpainting for Landsat 8 SAR Images

Clouds which exist in optical remote sensing images can degrade their applicability for earth observation. Ground-cover information is degraded by thin clouds or even completely occluded by thick clouds, which limits further analysis and applications. Thus a SLIC based cloud removal approach is formulated for clustering the similar superpixels and forming Column Stack. Group Sparsity Constrained Robust principal component analysis is used to detect cloud and generate a column stack mask. Discriminative Robust Principal Component analysis is conducted to remove clouds. Finally inpainting is performed by finding the similar patches to obtain the gaps filled. The experimental results of reconstructed Landsat 8 real images are compared with the Landsat 7 real images using PSNR and RMSE. The values for PSNR are varying from 25 in Landsat 7 Real images to 90 in Landsat 8 reconstructed real images in Green channel and RMSE has changed from 175 in Landsat 7 real images to 3 in Red channel. This indicates that Landsat 8 reconstructed real images have greater quality and error rate in less.

Distinct decision-making properties underlying the species specificity of group formation of flies.

Many animal species form groups. Group characteristics differ between species, suggesting that the decision-making of individuals for grouping varies across species. However, the actual decision-making properties that lead to interspecific differences in group characteristics remain unclear. Here, we compared the group formation processes of two Drosophilinae fly species, Colocasiomyia alocasiae and Drosophila melanogaster, which form dense and sparse groups, respectively. A high-throughput tracking system revealed that C. alocasiae flies formed groups faster than D. melanogaster flies, and the probability of C. alocasiae remaining in groups was far higher than that of D. melanogaster. C. alocasiae flies joined groups even when the group size was small, whereas D. melanogaster flies joined groups only when the group size was sufficiently large. C. alocasiae flies attenuated their walking speed when the inter-individual distance between flies became small, whereas such behavioural properties were not clearly observed in D. melanogaster. Furthermore, depriving C. alocasiae flies of visual input affected grouping behaviours, resulting in a severe reduction in group formation. These findings show that C. alocasiae decision-making regarding grouping, which greatly depends on vision, is significantly different from D. melanogaster, leading to species-specific group formation properties.

Data‐Driven Discovery of Soil Moisture Flow Governing Equation: A Sparse Regression Framework

AbstractConventionally, soil moisture dynamics are mathematically modeled by the Richardson‐Richards equation, whose derivation is based on the conservation of mass and the Buckingham‐Darcy law. However, it is complicated and even impossible to finish such rigorous derivations based on physical principles due to the complexity and uncertainties in the vadose zone. In this work, we propose a data‐driven sparse regression framework. For the first time, we discover the time‐dependent nonlinear soil moisture flow equation from only volumetric water content observations. The framework leverages linear approximations and group sparsity techniques. Except for a few assumptions, it requires no prior information, including a specific constitutive relationship model, boundary conditions, and initial conditions. Several numerical experiments are tested to demonstrate that the framework successfully discovers the underlying soil moisture flow equation from data and tends to discover the parsimonious equation governed by dominant physical processes. Besides, the identified nonlinear coefficients, which represent soil hydraulic properties, fit well with the actual coefficients, although they deviate slightly at near‐saturated ends. The results demonstrate the satisfactory performance of the proposed framework under various scenarios. Despite being based on a homogeneous soil assumption, this study provides a new perspective for deriving soil moisture flow governing equations.

Factors associated with reversals of COVID-19 vaccination willingness: Results from two longitudinal, national surveys in Japan 2021-2022

BackgroundResearch characterizing changes of heart with respect to vaccine intention is scarce, and very little research considers those who were initially vaccine willing but became hesitant. Here, we sought to assess the factors related to reversals of vaccine willingness.MethodsWe conducted a longitudinal, national survey on vaccination intention among Japanese adults aged 20 years or older, with the first questionnaire performed in February-March 2021 (N = 30,053) and the follow-up in February 2022 (N = 19,195, response rate 63.9%). The study population comprised those who reported vaccine willingness in the first survey, with the outcome variable being development of vaccine hesitancy at follow-up. We performed a regression analysis of vaccination status using sociodemographic, health-related, psychologic/attitudinal, and information-related variables as predictors. We used the sparse group minimax concave penalty (MCP) to select the optimum group of covariates for the logistic regression.FindingsOf 11,118 (57.9%) respondents who previously expressed interest in vaccination, 10,684 (96.1%) and 434 (3.9%) were in the vaccine willing and hesitant groups, respectively. Several covariates were found to significantly predict vaccine hesitancy, including marital status, influenza vaccine history, COVID-19 infection/testing history, engagement in COVID-19 preventive measures, perceived risks/benefits of the COVID-19 vaccine, and attitudes regarding vaccination policies and norms. The use of certain information sources was also associated with vaccine hesitancy.InterpretationSociodemographic, health-related, psychologic/attitudinal, and information-related variables predicted the development of vaccine hesitancy among those with prior willingness. Most of these predictors were also associated with vaccination status.FundingThe present work was supported in part by a grant from the Kanagawa Prefectural Government of Japan and by AIST government subsidies.

Underdetermined direction of arrival estimation of wideband signal based on sparse array

To solve the problem of the mismatch of the wideband underdetermined direction of arrival (DOA) estimation under the condition of the On-grid model, this paper extends the narrowband Off-Grid model to wideband and proposes a new DOA estimation algorithm for Off-Grid sources based on group sparsity. The proposed algorithm first obtains the preliminary estimation result under the current predefined discrete grid through the group sparsity wideband DOA estimation algorithm. Then, the Off-Grid optimisation problem is adopted to calculate the Off-Grid deviation vector. It is also assumed that the off-grid deviation vectors of each frequency subband are exactly the same, thereby reducing the number of parameters to be estimated. Therefore, the proposed algorithm can not only maintain similar or even better estimation accuracy but also greatly reduce the computational complexity. Finally, simulation is conducted and the results verify the effectiveness and performance of the proposed method.

Difference-of-Convex Algorithms for a Class of Sparse Group $\ell_0$ Regularized Optimization Problems

Difference-of-Convex Algorithms for a Class of Sparse Group $\ell_0$ Regularized Optimization Problems

Sparse group variable selection for gene-environment interactions in the longitudinal study.

Penalized variable selection for high-dimensional longitudinal data has received much attention as it can account for the correlation among repeated measurements while providing additional and essential information for improved identification and prediction performance. Despite the success, in longitudinal studies, the potential of penalization methods is far from fully understood for accommodating structured sparsity. In this article, we develop a sparse group penalization method to conduct the bi-level gene-environment (G E) interaction study under the repeatedly measured phenotype. Within the quadratic inference function framework, the proposed method can achieve simultaneous identification of main and interaction effects on both the group and individual levels. Simulation studies have shown that the proposed method outperforms major competitors. In the case study of asthma data from the Childhood Asthma Management Program, we conduct G E study by using high-dimensional single nucleotide polymorphism data as genetic factors and the longitudinal trait, forced expiratory volume in 1 s, as the phenotype. Our method leads to improved prediction and identification of main and interaction effects with important implications.

The ℓ2,p regularized total variation with overlapping group sparsity prior for image restoration with impulse noise

The ℓ2,p regularized total variation with overlapping group sparsity prior for image restoration with impulse noise

Group Logistic Regression Models with lp,q Regularization

In this paper, we proposed a logistic regression model with lp,q regularization that could give a group sparse solution. The model could be applied to variable-selection problems with sparse group structures. In the context of big data, the solutions for practical problems are often group sparse, so it is necessary to study this kind of model. We defined the model from three perspectives: theoretical, algorithmic and numeric. From the theoretical perspective, by introducing the notion of the group restricted eigenvalue condition, we gave the oracle inequality, which was an important property for the variable-selection problems. The global recovery bound was also established for the logistic regression model with lp,q regularization. From the algorithmic perspective, we applied the well-known alternating direction method of multipliers (ADMM) algorithm to solve the model. The subproblems for the ADMM algorithm were solved effectively. From the numerical perspective, we performed experiments for simulated data and real data in the factor stock selection. We employed the ADMM algorithm that we presented in the paper to solve the model. The numerical results were also presented. We found that the model was effective in terms of variable selection and prediction.

RKHSMetaMod: An R Package to Estimate the Hoeffding Decomposition of a Complex Model by Solving RKHS Ridge Group Sparse Optimization Problem

In this paper, we propose an R package, called [RKHSMetaMod](https://CRAN.R-project.org/package=RKHSMetaMod), that implements a procedure for estimating a meta-model of a complex model. The meta-model approximates the Hoeffding decomposition of the complex model and allows us to perform sensitivity analysis on it. It belongs to a reproducing kernel Hilbert space that is constructed as a direct sum of Hilbert spaces. The estimator of the meta-model is the solution of a penalized empirical least-squares minimization with the sum of the Hilbert norm and the empirical $L^2$-norm. This procedure, called RKHS ridge group sparse, allows both to select and estimate the terms in the Hoeffding decomposition, and therefore, to select and estimate the Sobol indices that are non-zero. The [RKHSMetaMod](https://CRAN.R-project.org/package=RKHSMetaMod) package provides an interface from the R statistical computing environment to the C++ libraries Eigen and GSL. In order to speed up the execution time and optimize the storage memory, except for a function that is written in R, all of the functions of this package are written using the efficient C++ libraries through [RcppEigen](https://CRAN.R-project.org/package=RcppEigen) and [RcppGSL](https://CRAN.R-project.org/package=RcppGSL) packages. These functions are then interfaced in the R environment in order to propose a user-friendly package.