Stochastic Block Model Research Articles

Bad Local Minima Exist in the Stochastic Block Model

AbstractWe study the disassortative stochastic block model with three communities, a well-studied model of graph partitioning and Bayesian inference for which detailed predictions based on the cavity method exist (Decelle et al. in Phys Rev E 84:066106, 2011). We provide strong evidence that for a part of the phase where efficient algorithms exist that approximately reconstruct the communities, inference based on maximum a posteriori (MAP) fails. In other words, we show that there exist modes of the posterior distribution that have a vanishing agreement with the ground truth. The proof is based on the analysis of a graph colouring algorithm from Achlioptas and Moore (J Comput Syst Sci 67:441–471, 2003).

Almost exact recovery in noisy semi-supervised learning

Abstract Graph-based semi-supervised learning methods combine the graph structure and labeled data to classify unlabeled data. In this work, we study the effect of a noisy oracle on classification. In particular, we derive the maximum a posteriori (MAP) estimator for clustering a degree corrected stochastic block model when a noisy oracle reveals a fraction of the labels. We then propose an algorithm derived from a continuous relaxation of the MAP, and we establish its consistency. Numerical experiments show that our approach achieves promising performance on synthetic and real data sets, even in the case of very noisy labeled data.

Network community detection via neural embeddings

Recent advances in machine learning research have produced powerful neural graph embedding methods, which learn useful, low-dimensional vector representations of network data. These neural methods for graph embedding excel in graph machine learning tasks and are now widely adopted. However, how and why these methods work—particularly how network structure gets encoded in the embedding—remain largely unexplained. Here, we show that node2vec—shallow, linear neural network—encodes communities into separable clusters better than random partitioning down to the information-theoretic detectability limit for the stochastic block models. We show that this is due to the equivalence between the embedding learned by node2vec and the spectral embedding via the eigenvectors of the symmetric normalized Laplacian matrix. Numerical simulations demonstrate that node2vec is capable of learning communities on sparse graphs generated by the stochastic blockmodel, as well as on sparse degree-heterogeneous networks. Our results highlight the features of graph neural networks that enable them to separate communities in the embedding space.

Lost in the Shuffle: Testing Power in the Presence of Errorful Network Vertex Labels

Two-sample network hypothesis testing is an important inference task with applications across diverse fields such as medicine, neuroscience, and sociology. Many of these testing methodologies operate under the implicit assumption that the vertex correspondence across networks is a priori known. This assumption is often untrue, and the power of the subsequent test can degrade when there are misaligned/label-shuffled vertices across networks. This power loss due to shuffling is theoretically explored in the context of random dot product and stochastic block model networks for a pair of hypothesis tests based on Frobenius norm differences between estimated edge probability matrices or between adjacency matrices. The loss in testing power is further reinforced by numerous simulations and experiments, both in the stochastic block model and in the random dot product graph model, where the power loss across multiple recently proposed tests in the literature is considered. Lastly, the impact that shuffling can have in real-data testing is demonstrated in a pair of examples from neuroscience and from social network analysis.

Partial recovery and weak consistency in the non-uniform hypergraph stochastic block model

Abstract We consider the community detection problem in sparse random hypergraphs under the non-uniform hypergraph stochastic block model (HSBM), a general model of random networks with community structure and higher-order interactions. When the random hypergraph has bounded expected degrees, we provide a spectral algorithm that outputs a partition with at least a $\gamma$ fraction of the vertices classified correctly, where $\gamma \in (0.5,1)$ depends on the signal-to-noise ratio (SNR) of the model. When the SNR grows slowly as the number of vertices goes to infinity, our algorithm achieves weak consistency, which improves the previous results in Ghoshdastidar and Dukkipati ((2017) Ann. Stat.45(1) 289–315.) for non-uniform HSBMs. Our spectral algorithm consists of three major steps: (1) Hyperedge selection: select hyperedges of certain sizes to provide the maximal signal-to-noise ratio for the induced sub-hypergraph; (2) Spectral partition: construct a regularised adjacency matrix and obtain an approximate partition based on singular vectors; (3) Correction and merging: incorporate the hyperedge information from adjacency tensors to upgrade the error rate guarantee. The theoretical analysis of our algorithm relies on the concentration and regularisation of the adjacency matrix for sparse non-uniform random hypergraphs, which can be of independent interest.

When does the mean network capture the topology of a sample of networks?

The notion of Fréchet mean (also known as “barycenter”) network is the workhorse of most machine learning algorithms that require the estimation of a “location” parameter to analyse network-valued data. In this context, it is critical that the network barycenter inherits the topological structure of the networks in the training dataset. The metric–which measures the proximity between networks–controls the structural properties of the barycenter. This work is significant because it provides for the first time analytical estimates of the sample Fréchet mean for the stochastic blockmodel, which is at the cutting edge of rigorous probabilistic analysis of random networks. We show that the mean network computed with the Hamming distance is unable to capture the topology of the networks in the training sample, whereas the mean network computed using the effective resistance distance recovers the correct partitions and associated edge density. From a practical standpoint, our work informs the choice of metrics in the context where the sample Fréchet mean network is used to characterize the topology of networks for network-valued machine learning.

Discovering prominent differences in structural and functional connectomes using a multinomial stochastic block model

ABSTRACT Understanding the differences between functional and structural human brain connectivity has been a focus of an extensive amount of neuroscience research. We employ a novel approach using the multinomial stochastic block model (MSBM) to explicitly extract components that characterize prominent differences across graphs. We analyze structural and functional connectomes derived from high-resolution diffusion-weighted MRI and fMRI scans of 250 Human Connectome Project subjects, analyzed at group connectivity level across 50 subjects. The inferred brain partitions revealed consistent, spatially homogeneous clustering patterns across inferred resolutions demonstrating the MSBM’s reliability in identifying brain areas with prominent structure-function differences. Prominent differences in low-resolution brain maps (K = {3, 4} clusters) were attributed to weak functional connectivity in the bilateral anterior temporal lobes, while higher resolution results (K ≥ 25) revealed stronger interhemispheric functional than structural connectivity. Our findings emphasize significant differences in high-resolution functional and structural connectomes, revealing challenges in extracting meaningful connectivity measurements from both modalities, including tracking fibers through the corpus callosum and attenuated functional connectivity in anterior temporal lobe fMRI data, which we attribute to increased noise levels. The MSBM emerges as a valuable tool for understanding differences across graphs, with potential future applications and avenues beyond the current focus on characterizing modality-specific distinctions in connectomics data.

A Principled Framework to Assess the Information-Theoretic Fitness of Brain Functional Sub-Circuits

In systems and network neuroscience, many common practices in brain connectomic analysis are often not properly scrutinized. One such practice is mapping a predetermined set of sub-circuits, like functional networks (FNs), onto subjects’ functional connectomes (FCs) without adequately assessing the information-theoretic appropriateness of the partition. Another practice that goes unchallenged is thresholding weighted FCs to remove spurious connections without justifying the chosen threshold. This paper leverages recent theoretical advances in Stochastic Block Models (SBMs) to formally define and quantify the information-theoretic fitness (e.g., prominence) of a predetermined set of FNs when mapped to individual FCs under different fMRI task conditions. Our framework allows for evaluating any combination of FC granularity, FN partition, and thresholding strategy, thereby optimizing these choices to preserve the important topological features of the human brain connectomes. By applying to the Human Connectome Project with Schaefer parcellations at multiple levels of granularity, the framework showed that the common thresholding value of 0.25 was indeed information-theoretically valid for group-average FCs, despite its previous lack of justification. Our results pave the way for the proper use of FNs and thresholding methods, and provide insights for future research in individualized parcellations.

Stochastic block hypergraph model.

The stochastic block model is widely used to generate graphs with a community structure, but no simple alternative currently exists for hypergraphs, in which more than two nodes can be connected together through a hyperedge. We discuss here such a hypergraph generalization, based on the clustering connection probability P_{ij} between nodes of communities i and j, and that uses an explicit and modulable hyperedge formation process. We focus on the standard case where P_{ij}=pδ_{ij}+q(1-δ_{ij}) when 0≤q≤p (δ_{ij} is the Kronecker symbol). We propose a simple model that satisfies three criteria: it should be as simple as possible, when p=q the model should be equivalent to the standard hypergraph random model, and it should use an explicit and modulable hyperedge formation process so that the model is intuitive and can easily express different real-world formation processes. We first show that for such a model the degree distribution and hyperedge size distribution can be approximated by binomial distributions with effective parameters that depend on the number of communities and q/p. Also, the composition of hyperedges goes for q=0 from 'pure' hyperedges (comprising nodes belonging to the same community) to 'mixed' hyperedges that comprise nodes from different communities for q=p. We test various formation processes and our results suggest that when they depend on the composition of the hyperedge, they tend to favor the dominant community and lead to hyperedges with a smaller diversity. In contrast, for formation processes that are independent from the hyperedge structure, we obtain hyperedges comprising a larger diversity of communities. The advantages of the model proposed here are its simplicity and flexibility that make it a good candidate for testing community-related problems, such as their detection, impact on various dynamics, and visualization.

Stochastic Block Smooth Graphon Model

In this article, we propose combining the stochastic blockmodel and the smooth graphon model, two of the most prominent modeling approaches in statistical network analysis. Stochastic blockmodels are generally used for clustering networks into groups of actors with homogeneous connectivity behavior. Smooth graphon models, on the other hand, assume that all nodes can be arranged on a one-dimensional scale such that closeness implies a similar behavior in connectivity. Both frameworks belong to the class of node-specific latent variable models, entailing a natural relationship. While these two modeling concepts have developed independently, this article proposes their generalization toward stochastic block smooth graphon models. This combined approach enables to exploit the advantages of both worlds. Employing concepts of the EM-type algorithm allows to develop an appropriate estimation routine, where MCMC techniques are used to accomplish the E-step. Simulations and real-world applications support the practicability of our novel method and demonstrate its advantages. The article is accompanied by supplementary material covering details about computation and implementation.

Model‐based clustering in simple hypergraphs through a stochastic blockmodel

AbstractWe propose a model to address the overlooked problem of node clustering in simple hypergraphs. Simple hypergraphs are suitable when a node may not appear multiple times in the same hyperedge, such as in co‐authorship datasets. Our model generalizes the stochastic blockmodel for graphs and assumes the existence of latent node groups and hyperedges are conditionally independent given these groups. We first establish the generic identifiability of the model parameters. We then develop a variational approximation Expectation‐Maximization algorithm for parameter inference and node clustering, and derive a statistical criterion for model selection. To illustrate the performance of our R package HyperSBM, we compare it with other node clustering methods using synthetic data generated from the model, as well as from a line clustering experiment and a co‐authorship dataset.

Hierarchical communities in the larval Drosophila connectome: Links to cellular annotations and network topology

One of the longstanding aims of network neuroscience is to link a connectome's topological properties-i.e., features defined from connectivity alone-with an organism's neurobiology. One approach for doing so is to compare connectome properties with annotational maps. This type of analysis is popular at the meso-/macroscale, but is less common at the nano-scale, owing to a paucity of neuron-level connectome data. However, recent methodological advances have made possible the reconstruction of whole-brain connectomes at single-neuron resolution for a select set of organisms. These include the fruit fly, Drosophila melanogaster, and its developing larvae. In addition to fine-scale descriptions of connectivity, these datasets are accompanied by rich annotations. Here, we use a variant of the stochastic blockmodel to detect multilevel communities in the larval Drosophila connectome. We find that communities partition neurons based on function and cell type and that most interact assortatively, reflecting the principle of functional segregation. However, a small number of communities interact nonassortatively, forming form a "rich-club" of interneurons that receive sensory/ascending inputs and deliver outputs along descending pathways. Next, we investigate the role of community structure in shaping communication patterns. We find that polysynaptic signaling follows specific trajectories across modular hierarchies, with interneurons playing a key role in mediating communication routes between modules and hierarchical scales. Our work suggests a relationship between system-level architecture and the biological function and classification of individual neurons. We envision our study as an important step toward bridging the gap between complex systems and neurobiological lines of investigation in brain sciences.

Community detection in multiplex continous weighted nodes networks using an extension of the stochastic block model

Community detection in multiplex continous weighted nodes networks using an extension of the stochastic block model

A multi-symptomatic model of heroin use disorder in rats reveals distinct behavioral profiles and neuronal correlates of heroin vulnerability versus resiliency.

The behavioral and diagnostic heterogeneity within human opioid use disorder (OUD) diagnosis is not readily captured in current animal models, limiting translational relevance of the mechanistic research that is conducted in experimental animals. We hypothesize that a non-linear clustering of OUD-like behavioral traits will capture population heterogeneity and yield subpopulations of OUD vulnerable rats with distinct behavioral and neurocircuit profiles. Over 900 male and female heterogeneous stock rats, a line capturing genetic and behavioral heterogeneity present in humans, were assessed for several measures of heroin use and rewarded and non-rewarded seeking behaviors. Using a non-linear stochastic block model clustering analysis, rats were assigned to OUD vulnerable, intermediate and resilient clusters. Additional behavioral tests and circuit analyses using c-fos protein activation were conducted on the vulnerable and resilient subpopulations. OUD vulnerable rats exhibited greater heroin taking and seeking behaviors relative to those in the intermediate and resilient clusters. Akin to human OUD diagnosis, further vulnerable rat sub-clustering revealed subpopulations with different combinations of behavioral traits, including sex differences. Lastly, heroin cue-induced neuronal patterns of circuit activation differed between resilient and vulnerable phenotypes. Behavioral sex differences were recapitulated in patterns of circuitry activation, including males preferentially engaging extended amygdala stress circuitry, and females cortico-striatal drug cue-seeking circuitry. Using a non-linear clustering approach in rats, we captured behavioral diagnostic heterogeneity reflective of human OUD diagnosis. OUD vulnerability and resiliency were associated with distinct neuronal activation patterns, posing this approach as a translational tool in assessing neurobiological mechanisms underpinning OUD.

Model selection for network data based on spectral information

In this work, we explore the extent to which the spectrum of the graph Laplacian can characterize the probability distribution of random graphs for the purpose of model evaluation and model selection for network data applications. Network data, often represented as a graph, consist of a set of pairwise observations between elements of a population of interests. The statistical network analysis literature has developed many different classes of network data models, with notable model classes including stochastic block models, latent node position models, and exponential families of random graph models. We develop a novel methodology which exploits the information contained in the spectrum of the graph Laplacian to predict the data-generating model from a set of candidate models. Through simulation studies, we explore the extent to which network data models can be differentiated by the spectrum of the graph Laplacian. We demonstrate the potential of our method through two applications to well-studied network data sets and validate our findings against existing analyses in the statistical network analysis literature.

Node-Level Community Detection within Edge Exchangeable Models for Interaction Processes

Scientists are increasingly interested in discovering community structure from modern relational data arising on large-scale social networks. While many methods have been proposed for learning community structure, few account for the fact that these modern networks arise from processes of interactions in the population. We introduce block edge exchangeable models (BEEM) for the study of interaction networks with latent node-level community structure. The block vertex components model (B-VCM) is derived as a canonical example. Several theoretical and practical advantages over traditional vertex-centric approaches are highlighted. In particular, BEEMs allow for sparse degree structure and power-law degree distributions within communities. Our theoretical analysis bounds the misspecification rate of block assignments while supporting simulations show the properties of the network can be recovered. A computationally tractable Gibbs algorithm is derived. We demonstrate the proposed model using post-comment interaction data from Talklife, a large-scale online peer-to-peer support network, and contrast the learned communities from those using standard algorithms including degree-corrected stochastic block models, popularity-adjusted block models, and weighted stochastic block models. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A novel control chart scheme for online social network monitoring using multivariate nonparametric profile techniques

Online social networks have become indispensable in modern life, facilitating knowledge sharing, social communication, and business marketing. To gain a deeper understanding of individuals' behavior within social networks, researchers have undertaken essential analytical tasks such as change point detection. Recently, nonparametric change point detection methods have attracted researchers' attention due to their generality and flexibility. However, existing methods exhibit limitations, including overlooking network structure, reliance on case-based network attributes, and neglecting the dynamic nature of data, which may have correlations in evolving social networks. In this study, we propose a novel multivariate mixed-effects nonparametric profile control (MENPC) algorithm to address these limitations. The advantage of MENPC relies on its unique point of view in approaching network data, where it incorporates the dynamic nature of data into the monitoring process without assuming internal independence of networks over time. Additionally, it takes into account the network structure by considering both nodal and network-level attributes. Furthermore, by introducing an updating trick formula, the proposed algorithm simplifies computations, effectively balancing memory and speed for online monitoring. We evaluate the effectiveness of MENPC through comprehensive numerical experiments using the degree correlated stochastic block model to simulate interactions in evolving online social networks. The results demonstrate MENPC's superior performance in terms of expected detection delay, showcasing its accuracy and efficiency in comparison to competing approaches including Wilson, and eigenvalue methods. Applying MENPC to the Enron email network dataset further confirms its significant progress in social network monitoring, expanding its potential for various applications.

Reassessing the modularity of gene co-expression networks using the Stochastic Block Model.

Finding communities in gene co-expression networks is a common first step toward extracting biological insight from these complex datasets. Most community detection algorithms expect genes to be organized into assortative modules, that is, groups of genes that are more associated with each other than with genes in other groups. While it is reasonable to expect that these modules exist, using methods that assume they exist a priori is risky, as it guarantees that alternative organizations of gene interactions will be ignored. Here, we ask: can we find meaningful communities without imposing a modular organization on gene co-expression networks, and how modular are these communities? For this, we use a recently developed community detection method, the weighted degree corrected stochastic block model (SBM), that does not assume that assortative modules exist. Instead, the SBM attempts to efficiently use all information contained in the co-expression network to separate the genes into hierarchically organized blocks of genes. Using RNAseq gene expression data measured in two tissues derived from an outbred population of Drosophila melanogaster, we show that (a) the SBM is able to find ten times as many groups as competing methods, that (b) several of those gene groups are not modular, and that (c) the functional enrichment for non-modular groups is as strong as for modular communities. These results show that the transcriptome is structured in more complex ways than traditionally thought and that we should revisit the long-standing assumption that modularity is the main driver of the structuring of gene co-expression networks.

Detecting planted partition in sparse multilayer networks

Abstract Multilayer networks are used to represent the interdependence between the relational data of individuals interacting with each other via different types of relationships. To study the information-theoretic phase transitions in detecting the presence of planted partition among the nodes of a multilayer network with additional nodewise covariate information and diverging average degree, Ma and Nandy (2023, IEEE Trans. Inf. Theory, 69, 3203–3239) introduced Multi-Layer Contextual Stochastic Block Model. In this paper, we consider the problem of detecting planted partitions in the Multi-Layer Contextual Stochastic Block Model, when the average node degrees for each network are greater than $1$. We establish the sharp phase transition threshold for detecting the planted bi-partition. Above the phase-transition threshold testing the presence of a bi-partition is possible, whereas below the threshold no procedure to identify the planted bi-partition can perform better than random guessing. We further establish that the derived detection threshold coincides with the threshold for weak recovery of the partition and provides a quasi-polynomial time algorithm to estimate it.

Dynamicity of brain network organization & their community architecture as characterizing features for classification of common mental disorders from whole-brain connectome

The urgency of addressing common mental disorders (bipolar disorder, attention-deficit hyperactivity disorder (ADHD), and schizophrenia) arises from their significant societal impact. Developing strategies to support psychiatrists is crucial. Previous studies focused on the relationship between these disorders and changes in the resting-state functional connectome’s modularity, often using static functional connectivity (sFC) estimation. However, understanding the dynamic reconfiguration of resting-state brain networks with rich temporal structure is essential for comprehending neural activity and addressing mental health disorders. This study proposes an unsupervised approach combining spatial and temporal characterization of brain networks to classify common mental disorders using fMRI timeseries data from two cohorts (N = 408 participants). We employ the weighted stochastic block model to uncover mesoscale community architecture differences, providing insights into network organization. Our approach overcomes sFC limitations and biases in community detection algorithms by modelling the functional connectome’s temporal dynamics as a landscape, quantifying temporal stability at whole-brain and network levels. Findings reveal individuals with schizophrenia exhibit less assortative community structure and participate in multiple motif classes, indicating less specialized network organization. Patients with schizophrenia and ADHD demonstrate significantly reduced temporal stability compared to healthy controls. This study offers insights into functional connectivity (FC) patterns’ spatiotemporal organization and their alterations in common mental disorders, highlighting the potential of temporal stability as a biomarker.