Group-wise Sparsity Research Articles

Online adaptive group-wise sparse Penalized Recursive Exponentially Weighted N-way Partial Least Square for epidural intracranial BCI.

Motor Brain-Computer Interfaces (BCIs) create new communication pathways between the brain and external effectors for patients with severe motor impairments. Control of complex effectors such as robotic arms or exoskeletons is generally based on the real-time decoding of high-resolution neural signals. However, high-dimensional and noisy brain signals pose challenges, such as limitations in the generalization ability of the decoding model and increased computational demands. The use of sparse decoders may offer a way to address these challenges. A sparsity-promoting penalization is a common approach to obtaining a sparse solution. BCI features are naturally structured and grouped according to spatial (electrodes), frequency, and temporal dimensions. Applying group-wise sparsity, where the coefficients of a group are set to zero simultaneously, has the potential to decrease computational time and memory usage, as well as simplify data transfer. Additionally, online closed-loop decoder adaptation (CLDA) is known to be an efficient procedure for BCI decoder training, taking into account neuronal feedback. In this study, we propose a new algorithm for online closed-loop training of group-wise sparse multilinear decoders using L p -Penalized Recursive Exponentially Weighted N-way Partial Least Square (PREW-NPLS). Three types of sparsity-promoting penalization were explored using L p with p = 0., 0.5, and 1. The algorithms were tested offline in a pseudo-online manner for features grouped by spatial dimension. A comparison study was conducted using an epidural ECoG dataset recorded from a tetraplegic individual during long-term BCI experiments for controlling a virtual avatar (left/right-hand 3D translation). Novel algorithms showed comparable or better decoding performance than conventional REW-NPLS, which was achieved with sparse models. The proposed algorithms are compatible with real-time CLDA. The proposed algorithm demonstrated good performance while drastically reducing the computational load and the memory consumption. However, the current study is limited to offline computation on data recorded with a single patient, with penalization restricted to the spatial domain only.

Decoding Genetic Markers of Multiple Phenotypic Layers Through Biologically Constrained Genome-To-Phenome Bayesian Sparse Regression.

The applicability of multivariate approaches for the joint analysis of genomics and phenomics information is currently limited by the lack of scalability, and by the difficulty of interpreting the related findings from a biological perspective. To tackle these limitations, we present Bayesian Genome-to-Phenome Sparse Regression (G2PSR), a novel multivariate regression method based on sparse SNP-gene constraints. The statistical framework of G2PSR is based on a Bayesian neural network, were constraints on SNPs-genes associations are integrated by incorporating a priori knowledge linking variants to their respective genes, to then reconstruct the phenotypic data in the output layer. Interpretability is promoted by inducing sparsity on the genes through variational dropout, allowing to estimate the uncertainty associated with each gene, and related SNPs, in the reconstruction task. Ultimately, G2PSR is conceived to prevent multiple testing correction and to assess the combined effect of SNPs, thus increasing the statistical power in detecting genome-to-phenome associations. The effectiveness of G2PSR was demonstrated on synthetic and real data, with respect to state-of-the-art methods based on group-wise sparsity constraints. The application on real data consisted in an imaging-genetics analysis on the Alzheimer's Disease Neuroimaging Initiative data, relating SNPs from more than 3,500 genes to clinical and multi-variate brain volumetric information. The experimental results show that our method can provide accurate selection of relevant genes in dataset with large SNPs-to-samples ratio, thus overcoming the main limitations of current genome-to-phenome association methods.

Construction of 4D infant cortical surface atlases with sharp folding patterns via spherical patch‐based group‐wise sparse representation

4D (spatial + temporal) infant cortical surface atlases covering dense time points are highly needed for understanding dynamic early brain development. In this article, we construct a set of 4D infant cortical surface atlases with longitudinally consistent and sharp cortical attribute patterns at 11 time points in the first six postnatal years, that is, at 1, 3, 6, 9, 12, 18, 24, 36, 48, 60, and 72 months of age, which is targeted for better normalization of the dynamic changing early brain cortical surfaces. To ensure longitudinal consistency and unbiasedness, we adopt a two-stage group-wise surface registration. To preserve sharp cortical attribute patterns on the atlas, instead of simply averaging over the coregistered cortical surfaces, we leverage a spherical patch-based sparse representation using the augmented dictionary to overcome the potential registration errors. Our atlases provide not only geometric attributes of the cortical folding, but also cortical thickness and myelin content. Therefore, to address the consistency across different cortical attributes on the atlas, instead of sparsely representing each attribute independently, we jointly represent all cortical attributes with a group-wise sparsity constraint. In addition, to further facilitate region-based analysis using our atlases, we have also provided two widely used parcellations, that is, FreeSurfer parcellation and multimodal parcellation, on our 4D infant cortical surface atlases. Compared to cortical surface atlases constructed with other methods, our cortical surface atlases preserve sharper cortical folding attribute patterns, thus leading to better accuracy in registration of individual infant cortical surfaces to the atlas.

Penalization methods with group-wise sparsity: econometric applications to eBay Motors online auctions

This paper investigates several recent developments in statistical penalization methods with applications to econometric models and economic data. When the set of covariate variables can be categorized into groups, we propose to use the Group Lasso (Yuan and Lin in J R Stat Soc Ser B 68(1):49–67, 2006) and Sparse Group Lasso (Simon et al. in J Comput Graph Stat 22(2):231–245, 2013) techniques to achieve group-wise sparsity. When estimating a structural model in empirical auctions work, these methods can flexibly control for observable heterogeneity by producing better, simpler first-stage fits for the approaches as proposed by Haile et al. (in: NBER working paper no. 10105, 2003) and Athey and Haile (in: Chapter 60 in handbook of econometrics, Elsevier, Amsterdam, 2007). In applying these methods to eBay Motors auction data, the models with group-wise sparsity are compared to the benchmark models and commonly used penalization methods with only parameter-wise regularization. Empirical results show that the Sparse Group Lasso method yields comparable or even better prediction performance than its counterparts in both linear regression and binary classification. Furthermore, it can drastically reduce the complexity of the model and produce a much more parsimonious model.

CONSTRUCTION OF SPATIOTEMPORAL NEONATAL CORTICAL SURFACE ATLASES USING A LARGE-SCALE DATASET.

The cortical surface atlases constructed from a large representative population of neonates are highly needed in the neonatal neuroimaging studies. However, existing neonatal cortical surface atlases are typically constructed from small datasets, e.g., tens of subjects, which are inherently biased and thus are not representative to the neonatal population. In this paper, we construct neonatal cortical surface atlases based on a large-scale dataset with 764 subjects. To better characterize the dynamic cortical development during the first postnatal weeks, instead of constructing just a single atlas, we construct a set of spatiotemporal atlases at each week from 39 to 44 gestational weeks. The central idea is that, for all cortical surfaces, we first group-wisely register them into the common space to ensure the unbiasedness. Then, rather than simply averaging over the co-registered cortical surfaces, which generally leads to over-smoothed cortical folding patterns, we adopt a spherical patch-based sparse representation using an augmented dictionary to overcome the noises and potential registration errors. Through the group-wise sparsity constraint, we obtain consistent geometric cortical folding attributes on the atlases. Our atlases preserve the sharp cortical folding patterns, thus leading to better registration accuracy when aligning new subjects onto the atlases.

Nested Sparse Approximation: Structured Estimation of V2V Channels Using Geometry-Based Stochastic Channel Model

Future intelligent transportation systems promise increased safety and energy efficiency. Realization of such systems will require vehicle-to-vehicle (V2V) communication technology. High fidelity V2V communication is, in turn, dependent on accurate V2V channel estimation. V2V channels have characteristics differing from classical cellular communication channels. Herein, geometry-based stochastic modeling is employed to develop a characterization of V2V channel channels. The resultant model exhibits significant structure; specifically, the V2V channel is characterized by three distinct regions within the delay-Doppler plane. Each region has a unique combination of specular reflections and diffuse components resulting in a particular element-wise and group-wise sparsity. This joint sparsity structure is exploited to develop a novel channel estimation algorithm. A general machinery is provided to solve the jointly element/group sparse channel (signal) estimation problem using proximity operators of a broad class of regularizers. The alternating direction method of multipliers using the proximity operator is adapted to optimize the mixed objective function. Key properties of the proposed objective functions are proven which ensure that the optimal solution is found by the new algorithm. The effects of pulse shape leakage are explicitly characterized and compensated, resulting in measurably improved performance. Numerical simulation and real V2V channel measurement data are used to evaluate the performance of the proposed method. Results show that the new method can achieve significant gains over previously proposed methods.

Non-Negative Matrix Factorization with Auxiliary Information on Overlapping Groups

Matrix factorization is useful to extract the essential low-rank structure from a given matrix and has been paid increasing attention. A typical example is non-negative matrix factorization (NMF), which is one type of unsupervised learning, having been successfully applied to a variety of data including documents, images and gene expression, where their values are usually non-negative. We propose a new model of NMF which is trained by using auxiliary information of overlapping groups. This setting is very reasonable in many applications, a typical example being gene function estimation where functional gene groups are heavily overlapped with each other. To estimate true groups from given overlapping groups efficiently, our model incorporates latent matrices with the regularization term using a mixed norm. This regularization term allows group-wise sparsity on the optimized low-rank structure. The latent matrices and other parameters are efficiently estimated by a block coordinate gradient descent method. We empirically evaluated the performance of our proposed model and algorithm from a variety of viewpoints, comparing with four methods including MMF for auxiliary graph information, by using both synthetic and real world document and gene expression data sets.

MRI-based intelligence quotient (IQ) estimation with sparse learning.

In this paper, we propose a novel framework for IQ estimation using Magnetic Resonance Imaging (MRI) data. In particular, we devise a new feature selection method based on an extended dirty model for jointly considering both element-wise sparsity and group-wise sparsity. Meanwhile, due to the absence of large dataset with consistent scanning protocols for the IQ estimation, we integrate multiple datasets scanned from different sites with different scanning parameters and protocols. In this way, there is large variability in these different datasets. To address this issue, we design a two-step procedure for 1) first identifying the possible scanning site for each testing subject and 2) then estimating the testing subject’s IQ by using a specific estimator designed for that scanning site. We perform two experiments to test the performance of our method by using the MRI data collected from 164 typically developing children between 6 and 15 years old. In the first experiment, we use a multi-kernel Support Vector Regression (SVR) for estimating IQ values, and obtain an average correlation coefficient of 0.718 and also an average root mean square error of 8.695 between the true IQs and the estimated ones. In the second experiment, we use a single-kernel SVR for IQ estimation, and achieve an average correlation coefficient of 0.684 and an average root mean square error of 9.166. All these results show the effectiveness of using imaging data for IQ prediction, which is rarely done in the field according to our knowledge.

Sparse overlapping group lasso for integrative multi-omics analysis.

Gene networks and graphs are crucial tools for understanding a heterogeneous system of cancer, since cancer is a disease that does not involve individual genes but combinations of genes associated with oncogenic process. A goal of genomic data analysis via gene networks is to identify both gene networks and individual genes within the selected networks. Existing methods, however, perform only network selection, and thus all genes in selected networks are included in models. This leads to overfitting when uncovering driver genes, and the results are not biologically interpretable. To accomplish both "groupwise sparsity" and "within group sparsity" for identifying driver genes based on biological knowledge (i.e., predefined overlapping groups of features), we propose a sparse overlapping group lasso via duplicated predictors in extended space. The proposed method effectively identifies driver genes and their interactions using known biological pathway information. Monte Carlo simulations and The Cancer Genome Atlas (TCGA) project data analysis indicate that the proposed method is effective for fitting a regression model (i.e., feature selection and prediction accuracy) constructed with duplicated predictors in overlapping groups. In the TCGA data analysis, we uncover potential cancer driver genes via expression modules and gene networks constructed by multi-omics data and identify that the uncovered genes have strong evidences as a cancer driver gene. The proposed method is a useful tool for identifying cancer driver genes and for integrative multi-omics analysis.