Stochastic Block Model Research Articles

Blockmodeling for analysis of social structures: theoretical and methodological foundations

The article discusses the features of blockmodeling as a class of methods for clustering network data in the analysis of social structures. Blockmodeling is considered as an approach to the analysis of social structure, which combines network components into groups (clusters) based on their equivalent structural positions. The basic concepts of blockmodeling are described – matrix, matrix image, cluster, clustering, position, block, blockmodel; an illustrating example is given. The concept of equivalence is presented, and two types of equivalence, structural and regular, are described. The main approaches of blockmodeling – indirect and direct – and related methods and algorithms are presented. For each approach, examples of the practical application in social sciences are provided. Other methods of blockmodeling (stochastic blockmodeling) and similar methods of subgroups detection in networks are mentioned. It is shown that the methodology of blockmodeling has heuristic potential for analyzing social structures and is promising for identifying cohesive groups and determining the role and structural positions of individuals within them. In conclusion, the open questions and limitations of this research methodology are discussed.

Analysis Method of Agricultural Total Factor Productivity Based on Stochastic Block Model (SBM) and Machine Learning

When analyzing agriculture’s total factor productivity, traditional measurement approaches do not take into account the inefficiency value. The production functions are assumed to be analyzed on basis of the random boundaries, which makes the analysis results inaccurate and unreliable. As a result, this paper proposes an analytical approach for agricultural total factor productivity based on the stochastic block model (SBM), which combines the benefits of statistics and machine learning. It uses the SBM direction distance function and the Luenberger productivity index to measure the agricultural efficiency, total factor productivity, and their components. The research study considers the data from 31 provinces from 2006 to 2018 years. First, one output indicator and six input indicators are established. The time-varying variations of the national agricultural inefficiency value and its source decomposition under variable scale returns are then determined using the SBM-based algorithm of agricultural total factor productivity and the obtained sample data. The changes of the agricultural total factor productivity and its components are comprehensively analyzed. Following an examination of the elements impacting agricultural efficiency and productivity, the socioeconomic development of the agricultural total factor productivity is examined in order to achieve efficient growth.

Randomized Spectral Clustering in Large-Scale Stochastic Block Models

Spectral clustering has been one of the widely used methods for community detection in networks. However, large-scale networks bring computational challenges to the eigenvalue decomposition therein. In this paper, we study the spectral clustering using randomized sketching algorithms from a statistical perspective, where we typically assume the network data are generated from a stochastic block model that is not necessarily of full rank. To do this, we first use the recently developed sketching algorithms to obtain two randomized spectral clustering algorithms, namely, the random projection-based and the random sampling-based spectral clustering. Then we study the theoretical bounds of the resulting algorithms in terms of the approximation error for the population adjacency matrix, the misclassification error, and the estimation error for the link probability matrix. It turns out that, under mild conditions, the randomized spectral clustering algorithms lead to the same theoretical bounds as those of the original spectral clustering algorithm. We also extend the results to degree-corrected stochastic block models. Numerical experiments support our theoretical findings and show the efficiency of randomized methods. A new R package called Rclust is developed and made available to the public. Supplementary materials for this article are available online.

Uncovering Cortical Units of Processing From Multi-Layered Connectomes.

Modern diffusion and functional magnetic resonance imaging (dMRI/fMRI) provide non-invasive high-resolution images from which multi-layered networks of whole-brain structural and functional connectivity can be derived. Unfortunately, the lack of observed correspondence between the connectivity profiles of the two modalities challenges the understanding of the relationship between the functional and structural connectome. Rather than focusing on correspondence at the level of connections we presently investigate correspondence in terms of modular organization according to shared canonical processing units. We use a stochastic block-model (SBM) as a data-driven approach for clustering high-resolution multi-layer whole-brain connectivity networks and use prediction to quantify the extent to which a given clustering accounts for the connectome within a modality. The employed SBM assumes a single underlying parcellation exists across modalities whilst permitting each modality to possess an independent connectivity structure between parcels thereby imposing concurrent functional and structural units but different structural and functional connectivity profiles. We contrast the joint processing units to their modality specific counterparts and find that even though data-driven structural and functional parcellations exhibit substantial differences, attributed to modality specific biases, the joint model is able to achieve a consensus representation that well accounts for both the functional and structural connectome providing improved representations of functional connectivity compared to using functional data alone. This implies that a representation persists in the consensus model that is shared by the individual modalities. We find additional support for this viewpoint when the anatomical correspondence between modalities is removed from the joint modeling. The resultant drop in predictive performance is in general substantial, confirming that the anatomical correspondence of processing units is indeed present between the two modalities. Our findings illustrate how multi-modal integration admits consensus representations well-characterizing each individual modality despite their biases and points to the importance of multi-layered connectomes as providing supplementary information regarding the brain's canonical processing units.

Large deviations of semisupervised learning in the stochastic block model.

In semisupervised community detection, the membership of a set of revealed nodes is known in addition to the graph structure and can be leveraged to achieve better inference accuracies. While previous works investigated the case where the revealed nodes are selected at random, this paper focuses on correlated subsets leading to atypically high accuracies. In the framework of the dense stochastic block model, we employ statistical physics methods to derive a large deviation analysis of the number of these rare subsets, as characterized by their free energy. We find theoretical evidence of a nonmonotonic relationship between reconstruction accuracy and the free energy associated to the posterior measure of the inference problem. We further discuss possible implications for active learning applications in community detection.

Role detection in bicycle-sharing networks using multilayer stochastic block models

AbstractIn urban systems, there is an interdependency between neighborhood roles and transportation patterns between neighborhoods. In this paper, we classify docking stations in bicycle-sharing networks to gain insight into the human mobility patterns of three major cities in the United States. We propose novel time-dependent stochastic block models, with degree-heterogeneous blocks and either mixed or discrete block membership, which classify nodes based on their time-dependent activity patterns. We apply these models to (1) detect the roles of bicycle-sharing stations and (2) describe the traffic within and between blocks of stations over the course of a day. Our models successfully uncover work blocks, home blocks, and other blocks; they also reveal activity patterns that are specific to each city. Our work gives insights for the design and maintenance of bicycle-sharing systems, and it contributes new methodology for community detection in temporal and multilayer networks with heterogeneous degrees.

Fundamental Privacy Limits in Bipartite Networks Under Active Attacks

This work considers active deanonymization of bipartite networks. The scenario arises naturally in evaluating privacy in various applications such as social networks, mobility networks, and medical databases. For instance, in active deanonymization of social networks, an anonymous victim is targeted by an attacker (e.g. the victim visits the attacker’s website), and the attacker queries her group memberships (e.g. by querying the browser history) to deanonymize her. In this work, the fundamental limits of privacy, in terms of the minimum number of queries necessary for deanonymization, is investigated. A stochastic model is considered, where 1) the bipartite network of group memberships is generated randomly; 2) the attacker has partial prior knowledge of the group memberships; and 3) it receives noisy responses to its real-time queries. The bipartite network is generated based on linear and sublinear preferential attachment, and the stochastic block model. The victim’s identity is chosen randomly based on a distribution modeling the users’ risk of being the victim (e.g. probability of visiting the website). An attack algorithm is proposed which builds upon techniques from communication with feedback, and its performance, in terms of expected number of queries, is analyzed. Simulation results are provided to verify the theoretical derivations.

Improved Clustering Algorithms for the Bipartite Stochastic Block Model

We establish sufficient conditions of exact and almost full recovery of the node partition in Bipartite Stochastic Block Model (BSBM) using polynomial time algorithms. First, we improve upon the known conditions of almost full recovery by spectral clustering algorithms in BSBM. Next, we propose a new computationally simple and fast procedure achieving exact recovery under milder conditions than the state of the art. Namely, if the vertex sets <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$V_{1}$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$V_{2}$ </tex-math></inline-formula> in BSBM have sizes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n_{1}$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n_{2}$ </tex-math></inline-formula> , we show that the condition <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$ p = \Omega \left ({\max \left ({\sqrt {\frac {\log {n_{1}}}{n_{1}n_{2}}},\frac {\log {n_{1}}}{n_{2}}}\right )}\right )$ </tex-math></inline-formula> on the edge intensity <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> is sufficient for exact recovery within <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$V_{1}$ </tex-math></inline-formula> . This condition exhibits an elbow at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n_{2} \asymp n_{1}\log {n_{1}}$ </tex-math></inline-formula> between the low-dimensional and high-dimensional regimes. The suggested procedure is a variant of Lloyd’s iterations initialized with a well-chosen spectral estimator leading to what we expect to be the optimal condition for exact recovery in BSBM. The optimality conjecture is supported by showing that, for a supervised oracle procedure, such a condition is necessary to achieve exact recovery. The key elements of the proof techniques are different from classical community detection tools on random graphs. Numerical studies confirm our theory, and show that the suggested algorithm is both very fast and achieves almost the same performance as the supervised oracle. Finally, using the connection between planted satisfiability problems and the BSBM, we improve upon the sufficient number of clauses to completely recover the planted assignment.

Reciprocity, community detection, and link prediction in dynamic networks

Many complex systems change their structure over time, in these cases dynamic networks can provide a richer representation of such phenomena. As a consequence, many inference methods have been generalized to the dynamic case with the aim to model dynamic interactions. Particular interest has been devoted to extend the stochastic block model and its variant, to capture community structure as the network changes in time. While these models assume that edge formation depends only on the community memberships, recent work for static networks show the importance to include additional parameters capturing structural properties, as reciprocity for instance. Remarkably, these models are capable of generating more realistic network representations than those that only consider community membership. To this aim, we present a probabilistic generative model with hidden variables that integrates reciprocity and communities as structural information of networks that evolve in time. The model assumes a fundamental order in observing reciprocal data, that is an edge is observed, conditional on its reciprocated edge in the past. We deploy a Markovian approach to construct the network’s transition matrix between time steps and parameters’ inference is performed with an expectation-maximization algorithm that leads to high computational efficiency because it exploits the sparsity of the dataset. We test the performance of the model on synthetic dynamical networks, as well as on real networks of citations and email datasets. We show that our model captures the reciprocity of real networks better than standard models with only community structure, while performing well at link prediction tasks.

Poisson degree corrected dynamic stochastic block model

Stochastic Block Model (SBM) provides a statistical tool for modeling and clustering network data. In this paper, we propose an extension of this model for discrete-time dynamic networks that takes into account the variability in node degrees, allowing us to model a broader class of networks. We develop a probabilistic model that generates temporal graphs with a dynamic cluster structure and time-dependent degree corrections for each node. Thanks to these degree corrections, the nodes can have variable in- and out-degrees, allowing us to model complex cluster structures as well as interactions that decrease or increase over time. We compare the proposed model to a model without degree correction and highlight its advantages in the case of inhomogenous degree distributions in the clusters and in the recovery of unstable cluster dynamics. We propose an inference procedure based on Variational Expectation-Maximization (VEM) that also provides the means to estimate the time-dependent degree corrections. Extensive experiments on simulated and real datasets confirm the benefits of our approach and show the effectiveness of the proposed algorithm.

Multiomics Topic Modeling for Breast Cancer Classification.

Simple SummaryTopic models are algorithms introduced for discovering hidden topics or latent variables in large, unstructured text corpora. Leveraging on analogies between texts and gene expression profiles, these algorithms can be used to find structures in expression data. This work presents an application of topic modeling techniques for the identification of breast cancer subtypes. In particular, we extended a specific class of topic models to allow a multiomics approach. As an illustrative example, considering both messenger RNA and microRNA expression levels, we were able to clearly distinguish healthy from tumor samples as well as the different breast cancer subtypes. The integration of different layers of information is crucial for the observed classification accuracy. Our approach naturally provides the genes and the microRNAs associated to the specific topics that are used for sample organization. We show that indeed these topics often contain genes involved in breast cancer development and are associated to different survival probabilities.The integration of transcriptional data with other layers of information, such as the post-transcriptional regulation mediated by microRNAs, can be crucial to identify the driver genes and the subtypes of complex and heterogeneous diseases such as cancer. This paper presents an approach based on topic modeling to accomplish this integration task. More specifically, we show how an algorithm based on a hierarchical version of stochastic block modeling can be naturally extended to integrate any combination of ’omics data. We test this approach on breast cancer samples from the TCGA database, integrating data on messenger RNA, microRNAs, and copy number variations. We show that the inclusion of the microRNA layer significantly improves the accuracy of subtype classification. Moreover, some of the hidden structures or “topics” that the algorithm extracts actually correspond to genes and microRNAs involved in breast cancer development and are associated to the survival probability.

Stochastic blockmodeling of linked networks

Blockmodeling linked networks aims to simultaneously cluster two or more sets of units into clusters based on a network where ties are possible both between units from the same set as well as between units of different sets. While this has already been developed for generalized and k-means blockmodeling, our approach is based on the well-known stochastic blockmodeling technique, utilizing a mixture model. Estimation is performed using the CEM algorithm, which iteratively estimates the parameters by maximizing a suitable likelihood function and reclusters the units according to the parameters. The steps are repeated until the likelihood function ceases to improve.A key drawback of the basic algorithm is that it treats all units equally, consequently yielding higher influence to larger parts of the data. The greater size, however, does not necessarily imply higher importance. To mitigate this asymmetry, we propose a solution where underrepresented parts of the data are given more influence through an appropriate weighting. This idea leads to the so-called weighted likelihood approach, where the ordinary likelihood function is replaced by a weighted likelihood.The efficiency of the different approaches is tested via simulations. It is shown through simulations that the weighted likelihood approach performs better for larger networks and a clearer blockmodel structure, especially when the one-mode blockmodels within the smaller sets are clearer.

A hidden challenge of link prediction: which pairs to check?

The traditional setup of link prediction in networks assumes that a test set of node pairs, which is usually balanced, is available over which to predict the presence of links. However, in practice, there is no test set: the ground truth is not known, so the number of possible pairs to predict over is quadratic in the number of nodes in the graph. Moreover, because graphs are sparse, most of these possible pairs will not be links. Thus, link prediction methods, which often rely on proximity-preserving embeddings or heuristic notions of node similarity, face a vast search space, with many pairs that are in close proximity, but that should not be linked. To mitigate this issue, we introduce LinkWaldo, a framework for choosing from this quadratic, massively skewed search space of node pairs, a concise set of candidate pairs that, in addition to being in close proximity, also structurally resemble the observed edges. This allows it to ignore some high-proximity but low-resemblance pairs, and also identify high-resemblance, lower-proximity pairs. Our framework is built on a model that theoretically combines stochastic block models (SBMs) with node proximity models. The block structure of the SBM maps out where in the search space new links are expected to fall, and the proximity identifies the most plausible links within these blocks, using locality sensitive hashing to avoid expensive exhaustive search. LinkWaldo can use any node representation learning or heuristic definition of proximity and can generate candidate pairs for any link prediction method, allowing the representation power of current and future methods to be realized for link prediction in practice. We evaluate LinkWaldo on 13 networks across multiple domains and show that on average it returns candidate sets containing 7–33% more missing and future links than both embedding-based and heuristic baselines’ sets. Our code is available at https://github.com/GemsLab/LinkWaldo.

Graph matching beyond perfectly-overlapping Erd\u0151s\u2013R\xe9nyi random graphs

Graph matching is a fruitful area in terms of both algorithms and theories. Given two graphs G_1 = (V_1, E_1) and G_2 = (V_2, E_2), where V_1 and V_2 are the same or largely overlapped upon an unknown permutation pi ^*, graph matching is to seek the correct mapping pi ^*. In this paper, we exploit the degree information, which was previously used only in noiseless graphs and perfectly-overlapping Erdős–Rényi random graphs matching. We are concerned with graph matching of partially-overlapping graphs and stochastic block models, which are more useful in tackling real-life problems. We propose the edge exploited degree profile graph matching method and two refined variations. We conduct a thorough analysis of our proposed methods’ performances in a range of challenging scenarios, including coauthorship data set and a zebrafish neuron activity data set. Our methods are proved to be numerically superior than the state-of-the-art methods. The algorithms are implemented in the R (A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, 2020) package GMPro (GMPro: graph matching with degree profiles, 2020).

Hybrid maximum likelihood inference for stochastic block models

Stochastic block models have known a flowering interest in the social network literature. They provide a tool for discovering communities and identifying clusters of individuals characterized by similar social behaviors. In this framework, full maximum likelihood estimates are not achievable due to the intractability of the likelihood function. For this reason, several approximate solutions are available in the literature. In this respect, a new and more efficient approximate method for estimating model parameters is introduced. This has a hybrid nature, in the sense that it exploits different features of existing methods. The proposal is illustrated by an intensive Monte Carlo simulation study and an application to a real-world network.

A multilayered block network model to forecast large dynamic transportation graphs: An application to US air transport

Dynamic transportation networks have been analyzed for years by means of static graph-based indicators in order to study the temporal evolution of relevant network components, and to reveal complex dependencies that would not be easily detected by a direct inspection of the data. This paper presents a state-of-the-art probabilistic latent network model to forecast multilayer dynamic graphs that are increasingly common in transportation and proposes a community-based extension to reduce the computational burden. Flexible time series analysis is obtained by modeling the probability of edges between vertices through latent Gaussian processes. The models and Bayesian inference are illustrated on a sample of 10-year data from four major airlines within the US air transportation system. Results show how the estimated latent parameters from the models are related to the airlines’ connectivity dynamics, and their ability to project the multilayer graph into the future for out-of-sample full network forecasts, while stochastic blockmodeling allows for the identification of relevant communities. Reliable network predictions would allow policy-makers to better understand the dynamics of the transport system, and help in their planning on e.g. route development, or the deployment of new regulations.

Community detection for multilayer weighted networks

Multilayer networks are used to encode multiple types of relations arising in complex systems and have received significant attention in recent years. Community detection in multilayer networks is an important issue in various fields; hence, stochastic block models have emerged as a popular probabilistic framework over the past decades. However, stochastic block models are suited to binary networks rather than weighted networks. A generalised stochastic block model is proposed herein to address multilayer sparse or dense weighted networks. A variational expectation-maximisation algorithm is derived to estimate the parameters of interest. In addition, an upper bound is derived for the probability of misclassification, which is governed by the Rényi divergence of order 12. Furthermore, our model is compared with four competing methods on synthetic networks. Finally, our approach is examined on financial markets and bicycle sharing systems.

Consistent estimation of the number of communities in stochastic block models using cross‐validation

The stochastic block model (SBM) and its variants constitute an important family of probabilistic tools for studying network data. There is a rich literature on methods for estimating block labels and model parameters of stochastic block models. Most of these studies require the number of communities K as an input, making the estimation of K an important problem. Cross‐validation is a natural option for this problem since it is a widely used generic method for evaluating model fitting. However, cross‐validation is known to be inconsistent and prone to overfitting unless impractical split ratios are used. Cross‐validation with confidence (CVC) is proposed with better theoretical guarantees in conventional settings. We study the properties of CVC for stochastic block models. Our theoretical studies show that CVC, unlike the standard cross‐validation, can consistently pick the optimal K under suitable conditions. We implement this method and check its performance against other established methods on both synthetic and real datasets.

A Bayesian Allocation Model Based Approach to Mixed Membership Stochastic Blockmodels

ABSTRACT Although detecting communities in networks has attracted considerable recent attention, estimating the number of communities is still an open problem. In this paper, we propose a model, which replicates the generative process of the mixed-membership stochastic block model (MMSB) within the generic allocation framework of Bayesian allocation model (BAM) and BAM-MMSB. In contrast to traditional blockmodels, BAM-MMSB considers the observations as Poisson counts generated by a base Poisson process and marks according to the generative process of MMSB. Moreover, the optimal number of communities for BAM-MMSB is estimated by computing the variational approximations of the marginal likelihood for each model order. Experiments on synthetic and real data sets show that the proposed approach promises a generalized model selection solution that can choose not only the model size but also the most appropriate decomposition.

Sea-land interdependence and delimitation of port hinterland-foreland structures in the international transportation system

Acknowledging that sea-land intermodal integration and transshipment are core transformations of the contemporary international logistic system, we study the spatial and functional structure of a port system across land and sea in this context. Previous port-driven regional development models have focused only on landside port-hinterland spatial structures in the local vicinity of ports, not seaside inter-port connections together. By revisiting the concepts of port triptych and hinterland-foreland continuum, we argue that the spatial structure of a port system should consider both flows to and from ports together when tracing spatial structures of hinterlands and forelands. Incorporating the network-based analytical model of the nonparametric weighted stochastic blockmodel, we study the global-scale structures of hinterlands and forelands under integrated landside-seaside freight flow dynamics. We investigate the network block structures of cargo shipping routes between Europe and the United States based on micro-level shipping flow data. We find a rich and meaningful collection of different network blocks of hinterlands, feeder and hub ports, and forelands that mirror the functional division of logistic processes across space, the interdependent relationships between hinterlands and forelands along a logistical continuum.