Sparse Connectivity Patterns Research Articles

Efficient Bayesian CNN Model Compression using Bayes by Backprop and L1-Norm Regularization

The swift advancement of convolutional neural networks (CNNs) in numerous real-world utilizations urges an elevation in computational cost along with the size of the model. In this context, many researchers steered their focus to eradicate these specific issues by compressing the original CNN models by pruning weights and filters, respectively. As filter pruning has an upper hand over the weight pruning method because filter pruning methods don’t impact sparse connectivity patterns. In this work, we suggested a Bayesian Convolutional Neural Network (BayesCNN) with Variational Inference, which prefaces probability distribution over weights. For the pruning task of Bayesian CNN, we utilized a combined version of L1-norm with capped L1-norm to help epitomize the amount of information that can be extracted through filter and control regularization. In this formation, we pruned unimportant filters directly without any test accuracy loss and achieved a slimmer model with comparative accuracy. The whole process of pruning is iterative and to validate the performance of our proposed work, we utilized several different CNN architectures on the standard classification dataset available. We have compared our results with non-Bayesian CNN models particularly, datasets such as CIFAR-10 on VGG-16, and pruned 75.8% parameters with float-point-operations (FLOPs) reduction of 51.3% without loss of accuracy and has achieved advancement in state-of-art.

Hierarchical Extraction of Functional Connectivity Components in Human Brain Using Resting-State fMRI.

The study of functional networks of the human brain has been of significant interest in cognitive neuroscience for over two decades, albeit they are typically extracted at a single scale using various methods, including decompositions like ICA. However, since numerous studies have suggested that the functional organization of the brain is hierarchical, analogous decompositions might better capture functional connectivity patterns. Moreover, hierarchical decompositions can efficiently reduce the very high dimensionality of functional connectivity data. This paper provides a novel method for the extraction of hierarchical connectivity components in the human brain using resting-state fMRI. The method builds upon prior work of Sparse Connectivity Patterns (SCPs) by introducing a hierarchy of sparse, potentially overlapping patterns. The components are estimated by cascaded factorization of correlation matrices generated from fMRI. The goal of the paper is to extract sparse interpretable hierarchically-organized patterns using correlation matrices where a low rank decomposition is formed by a linear combination of a higher rank decomposition. We formulate the decomposition as a non-convex optimization problem and solve it using gradient descent algorithms with adaptive step size. Along with the hierarchy, our method aims to capture the heterogeneity of the set of common patterns across individuals. We first validate our model through simulated experiments. We then demonstrate the effectiveness of the developed method on two different real-world datasets by showing that multi-scale hierarchical SCPs are reproducible between sub-samples and are more reproducible as compared to single scale patterns. We also compare our method with an existing hierarchical community detection approach.

Pruning filters with L1-norm and capped L1-norm for CNN compression

The blistering progress of convolutional neural networks (CNNs) in numerous applications of the real-world usually obstruct by a surge in network volume and computational cost. Recently, researchers concentrate on eliminating these issues by compressing the CNN models, such as pruning filters and weights. In comparison with the technique of pruning weights, the technique of pruning filters doesn’t effect in sparse connectivity patterns. In this article, we have proposed a fresh new technique to estimate the significance of filters. More precisely, we combined L1-norm with capped L1-norm to represent the amount of information extracted by the filter and control regularization. In the process of pruning, the insignificant filters remove directly without any loss in the test accuracy, providing much slimmer and compact models with comparable accuracy and this process is iterated a few times. To validate the effectiveness of our algorithm. We experimentally determine the usefulness of our approach with several advanced CNN models on numerous standard data sets. Particularly, data sets CIFAR-10 is used on VGG-16 and prunes 92.7% parameters with float-point-operations (FLOPs) reduction of 75.8% without loss of accuracy and has achieved advancement in state-of-art.

Convolutional Neural Network With Graphical Lasso to Extract Sparse Topological Features for Brain Disease Classification.

The functional connectivity provides new insights into the mechanisms of the human brain at network-level, which has been proved to be an effective biomarker for brain disease classification. Recently, machine learning methods have played an important role in functional connectivity classification, among which convolutional neural network (CNN) based methods become a new hot topic since they can extract topological features in the brain network. However, the conventional CNN-based methods haven't taken sparse connectivity patterns (SCPs) of the human brain into consideration, which may lead to redundancy of the topological features, and limit their performance and generalization. To solve it, we propose a novel CNN-based model with graphical Lasso (CNNGLasso) to extract sparse topological features for brain disease classification. First, we develop a novel graphical Lasso model for revealing the SCPs at group-level. Then, the SCPs are used to guide the topological feature extraction. Finally, the obtained sparse topological features are used to classify the patients from normal controls. The experiment results on the ABIDE dataset demonstrate that the CNNGLasso outperforms the others on various performances. Besides, the abnormal brain regions derived from the trained model are consistent with the previous investigations, which further proves the application prospect of the CNNGLasso.

Estimation of Dynamic Sparse Connectivity Patterns From Resting State fMRI.

Functional connectivity (FC) estimated from functional magnetic resonance imaging (fMRI) time series, especially during resting state periods, provides a powerful tool to assess human brain functional architecture in health, disease, and developmental states. Recently, the focus of connectivity analysis has shifted toward the subnetworks of the brain, which reveals co-activating patterns over time. Most prior works produced a dense set of high-dimensional vectors, which are hard to interpret. In addition, their estimations to a large extent were based on an implicit assumption of spatial and temporal stationarity throughout the fMRI scanning session. In this paper, we propose an approach called dynamic sparse connectivity patterns (dSCPs), which takes advantage of both matrix factorization and time-varying fMRI time series to improve the estimation power of FC. The feasibility of analyzing dynamic FC with our model is first validated through simulated experiments. Then, we use our framework to measure the difference between young adults and children with real fMRI data set from the Philadelphia Neurodevelopmental Cohort (PNC). The results from the PNC data set showed significant FC differences between young adults and children in four different states. For instance, young adults had reduced connectivity between the default mode network and other subnetworks, as well as hyperconnectivity within the visual system in states 1 and 3, and hypoconnectivity in state 2. Meanwhile, they exhibited temporal correlation patterns that changed over time within functional subnetworks. In addition, the dSCPs model indicated that older people tend to spend more time within a relatively connected FC pattern. Overall, the proposed method provides a valid means to assess dynamic FC, which could facilitate the study of brain networks.

Fused Estimation of Sparse Connectivity Patterns From Rest fMRI-Application to Comparison of Children and Adult Brains.

In this paper, we consider the problem of estimating multiple sparse, co-activated brain regions from functional magnetic resonance imaging (fMRI) observations belonging to different classes. More precisely, we propose a method to analyze similarities and differences in functional connectivity between children and young adults. Often, analysis is conducted on each class separately, and differences across classes are identified with an additional postprocessing step using adequate statistical tools. Here, we propose to rely on a generalized fused Lasso penalty, which allows us to make use of the entire data set in order to estimate connectivity patterns that are either shared across classes, or specific to a given group. By using the entire population during the estimation, we hope to increase the power of our analysis. The proposed model falls in the category of population-wise matrix decomposition, and a simple and efficient alternating direction method of multipliers algorithm is introduced to solve the associated optimization problem. After validating our approach on simulated data, experiments are performed on resting-state fMRI imaging from the Philadelphia neurodevelopmental cohort data set, comprised of normally developing children from ages 8 to 21. Developmental differences were observed in various brain regions, as a total of three class-specific resting-state components were identified. Statistical analysis of the estimated subject-specific features, as well as classification results (based on age groups, up to 81% accuracy, samples) related to these components demonstrate that the proposed method is able to properly extract meaningful shared and class-specific sub-networks.

Identifying Sparse Connectivity Patterns in the brain using resting-state fMRI

The human brain processes information via multiple distributed networks. An accurate model of the brain's functional connectome is critical for understanding both normal brain function as well as the dysfunction present in neuropsychiatric illnesses. Current methodologies that attempt to discover the organization of the functional connectome typically assume spatial or temporal separation of the underlying networks. This assumption deviates from an intuitive understanding of brain function, which is that of multiple, inter-dependent spatially overlapping brain networks that efficiently integrate information pertinent to diverse brain functions. It is now increasingly evident that neural systems use parsimonious formations and functional representations to efficiently process information while minimizing redundancy. Hence we exploit recent advances in the mathematics of sparse modeling to develop a methodological framework aiming to understand complex resting-state fMRI connectivity data. By favoring networks that explain the data via a relatively small number of participating brain regions, we obtain a parsimonious representation of brain function in terms of multiple “Sparse Connectivity Patterns” (SCPs), such that differential presence of these SCPs explains inter-subject variability. In this manner the sparsity-based framework can effectively capture the heterogeneity of functional activity patterns across individuals while potentially highlighting multiple sub-populations within the data that display similar patterns. Our results from simulated as well as real resting state fMRI data show that SCPs are accurate and reproducible between sub-samples as well as across datasets. These findings substantiate existing knowledge of intrinsic functional connectivity and provide novel insights into the functional organization of the human brain.

Discriminative sparse connectivity patterns for classification of fMRI Data.

Functional connectivity using resting-state fMRI has emerged as an important research tool for understanding normal brain function as well as changes occurring during brain development and in various brain disorders. Most prior work has examined changes in pairwise functional connectivity values using a multi-variate classification approach, such as Support Vector Machines (SVM). While it is powerful, SVMs produce a dense set of high-dimensional weight vectors as output, which are difficult to interpret, and require additional post-processing to relate to known functional networks. In this paper, we propose a joint framework that combines network identification and classification, resulting in a set of networks, or Sparse Connectivity Patterns (SCPs) which are functionally interpretable as well as highly discriminative of the two groups. Applied to a study of normal development classifying children vs. adults, the proposed method provided accuracy of 76%(AUC= 0.85), comparable to SVM (79%,AUC=0.87), but with dramatically fewer number of features (50 features vs. 34716 for the SVM). More importantly, this leads to a tremendous improvement in neuro-scientific interpretability, which is specially advantageous in such a study where the group differences are wide-spread throughout the brain. Highest-ranked discriminative SCPs reflect increases in long-range connectivity in adults between the frontal areas and posterior cingulate regions. In contrast, connectivity between the bilateral parahippocampal gyri was decreased in adults compared to children.