Mixed Membership Stochastic Blockmodel Research Articles

Monitoring networks with overlapping communities based on latent mixed-membership stochastic block model

Monitoring networks with overlapping communities based on latent mixed-membership stochastic block model

Variational Bayesian inference for bipartite mixed-membership stochastic block model with applications to collaborative filtering

A network-based method applied to collaborative filtering in recommender systems is introduced in this paper. Specifically, a novel mixed-membership stochastic block model with a conjugate prior from the exponential family is proposed for bipartite networks. The analytical expression of the model is derived, and a variational Bayesian algorithm that is computationally feasible for approximating the untractable posterior distributions is presented. Extensive simulations show that the proposed model provides more accurate inference than competing methods with the presence of outliers. The proposed model is also applied to a MovieLens dataset for a real data application.

Power enhancement and phase transitions for global testing of the mixed membership stochastic block model

The mixed-membership stochastic block model (MMSBM) is a common model for social networks. Given an n-node symmetric network generated from a K-community MMSBM, we would like to test K=1 versus K>1. We first study the degree-based χ2 test and the orthodox Signed Quadrilateral (oSQ) test. These two statistics estimate an order-2 polynomial and an order-4 polynomial of a “signal” matrix, respectively. We derive the asymptotic null distribution and power for both tests. However, for each test, there exists a parameter regime where its power is unsatisfactory. It motivates us to propose a power enhancement (PE) test to combine the strengths of both tests. We show that the PE test has a tractable null distribution and improves the power of both tests. To assess the optimality of PE, we consider a randomized setting, where the n membership vectors are independently drawn from a distribution on the standard simplex. We show that the success of global testing is governed by a quantity βn(K,P,h), which depends on the community structure matrix P and the mean vector h of memberships. For each given (K,P,h), a test is called optimal if it distinguishes two hypotheses when βn(K,P,h)→∞. A test is called optimally adaptive if it is optimal for all (K,P,h). We show that the PE test is optimally adaptive, while many existing tests are only optimal for some particular (K,P,h), hence, not optimally adaptive.

Regularized spectral clustering under the mixed membership stochastic block model

Mixed membership community detection is a challenging problem in network analysis. Previous spectral clustering algorithms for this problem are developed based on the adjacency matrix instead of regularized Laplacian matrix. To close this gap, under the popular mixed membership stochastic blockmodels (MMSB), this article proposes two efficient spectral clustering algorithms based on an application of the regularized Laplacian matrix, the Simplex Regularized Spectral Clustering (SRSC) algorithm, and the Cone Regularized Spectral Clustering (CRSC) algorithm. SRSC and CRSC are developed based on the simplex structure and the cone structure in the variants of the eigendecomposition of the regularized Laplacian matrix. We show that these two approaches SRSC and CRSC are asymptotically consistent under mild conditions by providing error bounds for the estimated membership vector of each node under MMSB. These two proposed approaches are successfully applied to synthetic and empirical networks with encouraging results compared with some benchmark methods.

PPISB: A Novel Network-Based Algorithm of Predicting Protein-Protein Interactions With Mixed Membership Stochastic Blockmodel.

Protein-protein interactions (PPIs) play an essential role for most of biological processes in cells. Many computational algorithms have thus been proposed to predict PPIs. However, most of them heavily rest on the biological information of proteins while ignoring the latent structural features of proteins presented in a PPI network. In this paper, we propose an efficient network-based prediction algorithm, namely PPISB, based on a mixed membership stochastic blockmodel. By simulating the generative process of a PPI network, PPISB is able to capture the latent community structures. The inference procedure adopted by PPISB further optimizes the membership distributions of proteins over different complexes. After that, a distance measure is designed to compute the similarity between two proteins in terms of their likelihoods of being in the same complex, thus verifying whether they interact with each other or not. To evaluate the performance of PPISB, a series of extensive experiments have been conducted with five PPI networks collected from different species and the results demonstrate that PPISB has a promising performance when applied to predict PPIs in terms of several evaluation metrics. Hence, we reason that PPISB is preferred over state-of-the-art network-based prediction algorithms especially for predicting potential PPIs.

A Useful Criterion on Studying Consistent Estimation in Community Detection.

In network analysis, developing a unified theoretical framework that can compare methods under different models is an interesting problem. This paper proposes a partial solution to this problem. We summarize the idea of using a separation condition for a standard network and sharp threshold of the Erdös–Rényi random graph to study consistent estimation, and compare theoretical error rates and requirements on the network sparsity of spectral methods under models that can degenerate to a stochastic block model as a four-step criterion SCSTC. Using SCSTC, we find some inconsistent phenomena on separation condition and sharp threshold in community detection. In particular, we find that the original theoretical results of the SPACL algorithm introduced to estimate network memberships under the mixed membership stochastic blockmodel are sub-optimal. To find the formation mechanism of inconsistencies, we re-establish the theoretical convergence rate of this algorithm by applying recent techniques on row-wise eigenvector deviation. The results are further extended to the degree-corrected mixed membership model. By comparison, our results enjoy smaller error rates, lesser dependence on the number of communities, weaker requirements on network sparsity, and so forth. The separation condition and sharp threshold obtained from our theoretical results match the classical results, so the usefulness of this criterion on studying consistent estimation is guaranteed. Numerical results for computer-generated networks support our finding that spectral methods considered in this paper achieve the threshold of separation condition.

A Statistical Interpretation of Spectral Embedding: The Generalised Random Dot Product Graph

AbstractSpectral embedding is a procedure which can be used to obtain vector representations of the nodes of a graph. This paper proposes a generalisation of the latent position network model known as the random dot product graph, to allow interpretation of those vector representations as latent position estimates. The generalisation is needed to model heterophilic connectivity (e.g. ‘opposites attract’) and to cope with negative eigenvalues more generally. We show that, whether the adjacency or normalised Laplacian matrix is used, spectral embedding produces uniformly consistent latent position estimates with asymptotically Gaussian error (up to identifiability). The standard and mixed membership stochastic block models are special cases in which the latent positions take only K distinct vector values, representing communities, or live in the (K − 1)-simplex with those vertices respectively. Under the stochastic block model, our theory suggests spectral clustering using a Gaussian mixture model (rather than K-means) and, under mixed membership, fitting the minimum volume enclosing simplex, existing recommendations previously only supported under non-negative-definite assumptions. Empirical improvements in link prediction (over the random dot product graph), and the potential to uncover richer latent structure (than posited under the standard or mixed membership stochastic block models) are demonstrated in a cyber-security example.

A Bayesian Allocation Model Based Approach to Mixed Membership Stochastic Blockmodels

ABSTRACT Although detecting communities in networks has attracted considerable recent attention, estimating the number of communities is still an open problem. In this paper, we propose a model, which replicates the generative process of the mixed-membership stochastic block model (MMSB) within the generic allocation framework of Bayesian allocation model (BAM) and BAM-MMSB. In contrast to traditional blockmodels, BAM-MMSB considers the observations as Poisson counts generated by a base Poisson process and marks according to the generative process of MMSB. Moreover, the optimal number of communities for BAM-MMSB is estimated by computing the variational approximations of the marginal likelihood for each model order. Experiments on synthetic and real data sets show that the proposed approach promises a generalized model selection solution that can choose not only the model size but also the most appropriate decomposition.

Dynamic Stochastic Blockmodel Regression for Network Data: Application to International Militarized Conflicts

The decision to engage in military conflict is shaped by many factors, including state- and dyad-level characteristics as well as the state’s membership in geopolitical coalitions. Supporters of the democratic peace theory, for example, hypothesize that the community of democratic states is less likely to wage war with each other. Such theories explain the ways in which nodal and dyadic characteristics affect the evolution of conflict patterns over time via their effects on group memberships. To test these arguments, we develop a dynamic model of network data by combining a hidden Markov model with a mixed-membership stochastic blockmodel that identifies latent groups underlying the network structure. Unlike existing models, we incorporate covariates that predict dynamic node memberships in latent groups as well as the direct formation of edges between dyads. While prior substantive research often assumes the decision to engage in international militarized conflict is independent across states and static over time, we demonstrate that conflict is driven by states’ evolving membership in geopolitical blocs. Our analysis of militarized disputes from 1816 to 2010 identifies two distinct blocs of democratic states, only one of which exhibits unusually low rates of conflict. Changes in monadic covariates like democracy shift states between coalitions, making some states more pacific but others more belligerent. Supplementary materials for this article are available online.

Assessing unintended human‐mediated dispersal using visitation networks

Abstract Human visitors are associated with the unintended dispersal of weeds, seeds and pathogens across ecological communities. With the increasing popularity of nature‐based tourism, access to protected areas has increased, in turn increasing the risks of unintended dispersal of exotic species to these areas. Here, we assess the potential contribution of both international and domestic visitors travelling within New Zealand to the spread of exotic species. To get an overview of the visitors’ travelling patterns across the country, we constructed visitation networks at two spatial scales—a regional scale (which is a coarse scale) and a local territorial scale (which is a finer scale). We then used a Mixed Membership Stochastic Block Model to identify characteristic groups of visitors and places based on the similarities of the visitors’ travelling patterns across the country. Overall, we found that there are 10 characteristic groups of visitors travelling to 12 characteristic groups of places at the regional scale and 6 characteristic groups of visitors travelling to 6 characteristic groups of places at the territorial scale. The resulting characteristic travelling patterns of the visitors across New Zealand further allowed us to estimate the different visitor groups’ likelihood to travel to protected areas. Overall, we found that some visitor groups are much more likely than others to travel to protected areas of high protection status, at both spatial scales. Synthesis and applications. Our results highlight the importance of accounting for human behaviour—that is, understanding how visitors travel to places—when assessing human‐mediated dispersal. More specifically, we illustrate how to assess the relative contribution of a potential vector dispersing exotic species based on their travelling patterns—especially in cases where the target exotic species are not yet identified or when there is limited information regarding the dispersal routes of exotic species and their potential vectors. As a result, our work offers a holistic perspective on human‐mediated dispersal of exotic species. Moreover, it provides a potential baseline against which both field biologists and practitioners can identify areas that would benefit from further investigation to better understand invasion processes in their focal systems.

Mixed Membership Stochastic Blockmodels for Heterogeneous Networks

Heterogeneous networks are useful for modeling complex systems that consist of different types of objects. However, there are limited statistical models to deal with heterogeneous networks. In this paper, we propose a statistical model for community detection in heterogeneous networks. We formulate a heterogeneous version of the mixed membership stochastic blockmodel to accommodate heterogeneity in the data and the content dependent property of the pairwise relationship. We also apply a variational algorithm for posterior inference. The proposed procedure is shown to be consistent for community detection under mixed membership stochastic blockmodels for heterogeneous networks. We demonstrate the advantage of the proposed method in modeling overlapping communities and multiple memberships through simulation studies and applications to a real data set.

K-Medoids Clustering Based Next Location Prediction in Wireless Local Area Network

User mobility prediction in wireless network is being investigated from various angles to improve performance of the network. Access to user’s movement information such as time, direction, speed etc provides an opportunity for wireless networks to effectively manage resources to satisfy user needs. A next location prediction technique is required for transferring the existing connections of user to the next Access Point (AP) beforehand to ensure better Quality of Service (QoS) of the network. There are several techniques for next location prediction of mobile users in Wireless Local Area Network (WLAN) which include Indoor Next Location Prediction with Wi-Fi model, Extended Mobility Markov Chain Model, Hidden Markov Model and Mixed Membership Stochastic Blockmodel. In the Indoor Next Location Prediction with Wi-Fi model the area of prediction is fixed and small which makes this approach inefficient when the number of locations traversed by the mobile user is large. The paper addresses the issue of predicting the next location of mobile users in a Wireless Local Area Network (WLAN) when the area of prediction is vast. In this paper an intelligent clustering technique i.e., the K-Medoids clustering algorithm has been implemented on the indoor next location prediction, which is based on a Markov-chain model, for predicting the next location of a user when the number of locations traversed by the user is vast. The accuracy of prediction of mobile user’s next location by the proposed K-Medoids clustering based next location prediction technique ranges from 67% to 97%.

Estimating Mixed Memberships With Sharp Eigenvector Deviations

We consider the problem of estimating community memberships of nodes in a network, where every node is associated with a vector determining its degree of membership in each community. Existing provably consistent algorithms often require strong assumptions about the population, are computationally expensive, and only provide an overall error bound for the whole community membership matrix. This article provides uniform rates of convergence for the inferred community membership vector of each node in a network generated from the mixed membership stochastic blockmodel (MMSB); to our knowledge, this is the first work to establish per-node rates for overlapping community detection in networks. We achieve this by establishing sharp row-wise eigenvector deviation bounds for MMSB. Based on the simplex structure inherent in the eigen-decomposition of the population matrix, we build on established corner-finding algorithms from the optimization community to infer the community membership vectors. Our results hold over a broad parameter regime where the average degree only grows poly-logarithmically with the number of nodes. Using experiments with simulated and real datasets, we show that our method achieves better error with lower variability over competing methods, and processes real world networks of up to 100,000 nodes within tens of seconds. Supplementary materials for this article are available online.

Fragmentation Coagulation Based Mixed Membership Stochastic Blockmodel

The Mixed-Membership Stochastic Blockmodel (MMSB) is proposed as one of the state-of-the-art Bayesian relational methods suitable for learning the complex hidden structure underlying the network data. However, the current formulation of MMSB suffers from the following two issues: (1), the prior information (e.g. entities' community structural information) can not be well embedded in the modelling; (2), community evolution can not be well described in the literature. Therefore, we propose a non-parametric fragmentation coagulation based Mixed Membership Stochastic Blockmodel (fcMMSB). Our model performs entity-based clustering to capture the community information for entities and linkage-based clustering to derive the group information for links simultaneously. Besides, the proposed model infers the network structure and models community evolution, manifested by appearances and disappearances of communities, using the discrete fragmentation coagulation process (DFCP). By integrating the community structure with the group compatibility matrix we derive a generalized version of MMSB. An efficient Gibbs sampling scheme with Polya Gamma (PG) approach is implemented for posterior inference. We validate our model on synthetic and real world data.

Scalable Graph Convolutional Networks With Fast Localized Spectral Filter for Directed Graphs

Graph convolutional neural netwoks (GCNNs) have been emerged to handle graph-structured data in recent years. Most existing GCNNs are either spatial approaches working on neighborhood of each node, or spectral approaches based on graph Laplacian. Compared with spatial-based GCNNs, spectral-based GCNNs are capable of highly exploiting graph structure information, but always regard graphs undiredcted. Actually, there are many scenarios where the graph structures are directed, such as social networks, citation networks, etc. Treating graphs undirected may lose important information, which is helpful for graph learning tasks. This motivate us to construct a spectral-based GCNN for directed graphs. In this paper, we propose a scalable graph convolutional neural network with fast localized convolution operators derived from directed graph Laplacian, which is called fast directed graph convolutional network (FDGCN). FDGCN can directly work on directed graphs and can scale to large graphs as the convolution operation is linear with the number of edges. Furthermore, we find that FDGCN can unify the graph convolutional network (GCN), which is a classic spectral-based GCNN. The mechanism of FDGCN is thoroughly analyzed from spatial aggregation point of view. Since previous work has confirmed that considering uncertainty of graph could promote GCN a lot, the proposed FDGCN is further enhanced through extra training epochs on random graphs generated by mixed membership stochastic block model (MMSBM). Experiments are conducted for semi-supervised node classification tasks to evaluate the performance of FDGCN. Results show that our model can outperform or match state-of-the-art models in most cases.

Bayesian Graph Convolutional Neural Networks for Semi-Supervised Classification

Recently, techniques for applying convolutional neural networks to graph-structured data have emerged. Graph convolutional neural networks (GCNNs) have been used to address node and graph classification and matrix completion. Although the performance has been impressive, the current implementations have limited capability to incorporate uncertainty in the graph structure. Almost all GCNNs process a graph as though it is a ground-truth depiction of the relationship between nodes, but often the graphs employed in applications are themselves derived from noisy data or modelling assumptions. Spurious edges may be included; other edges may be missing between nodes that have very strong relationships. In this paper we adopt a Bayesian approach, viewing the observed graph as a realization from a parametric family of random graphs. We then target inference of the joint posterior of the random graph parameters and the node (or graph) labels. We present the Bayesian GCNN framework and develop an iterative learning procedure for the case of assortative mixed-membership stochastic block models. We present the results of experiments that demonstrate that the Bayesian formulation can provide better performance when there are very few labels available during the training process.

Detecting Overlapping Community Structure With Node Influence

Discovering the underlying overlapping community divisions can guide us in better exploring and predicting the structure and properties of the network. However, a large number of existing methods assume that nodes belong only to a single community. In this paper, we designed a posterior probabilistic prediction model under the Mixed-Membership Stochastic Blockmodel framework to accurately detect the overlapping community structure that exists in the network. In order to capture the degree of nodes that exhibit heterogeneous characteristics in the network, the model takes into account the influence of the nodes. In addition, we developed a non-conjugated stochastic variational inference to deduce the link probability prediction model with node influence. The key strategy is to use the mean-domain variational family with variable distribution to approximate the posterior community strengthen and node influence distribution in the prediction model. We compared the performance of this model with the previous algorithm models on computer-generated and real-world networks and found that it gives better results, especially when the heterogeneity of the network is very serious. In general, the combination of node influence and link probabilistic predictive model provides a new idea for us to use a statistical model to explore large-scale overlapping networks.

Modeling Social Networks as Mediators: A Mixed Membership Stochastic Blockmodel for Mediation

There are some educational interventions aimed at changing the ways in which individuals interact, and social networks are particularly useful for quantifying these changes. For many of these interventions, the ultimate goal is to change some outcome of interest such as teacher quality or student achievement, and social networks act as a natural mediator; the intervention changes the social networks of the teachers in schools, and teachers with certain types of social networks tend to use better teaching practices, for example. Due to lack of methodology, however, social networks have not been modeled as mediators. We present a new framework for modeling social networks as mediators in which a social network model is embedded into a mediation model and both models are estimated simultaneously. As a proof of concept, we introduce a new network model for mediation, applicable for interventions that affect subgroup structure. We provide a small simulation study to demonstrate the feasibility of this model and explore some potential operating characteristics. Finally, we apply our model to examine the effects of instructional coaches on teacher advice-seeking networks and subsequent changes in beliefs about mathematics.

Overlapping thematic structures extraction with mixed-membership stochastic blockmodel

It is increasing important to identify automatically thematic structures from massive scientific literature. The interdisciplinarity enables thematic structures without natural boundaries. In this work, the identification of thematic structures is regarded as an overlapping community detection problem from the large-scale citation-link network. A mixed-membership stochastic blockmodel, armed with stochastic variational inference algorithm, is utilized to detect the overlapping thematic structures. In the meanwhile, in order to enhance readability, each theme is labeled with soft mutual information based method by several topical terms. Extensive experimental results on the astro dataset indicate that mixed-membership stochastic blockmodel primarily uses the local information and allows for the pervasive overlaps, but it favors similar sized themes, which disqualifies this approach from being used to extract the thematic structures from scientific literature. In addition, the thematic structures from the bibliographic coupling network is similar to those from the co-citation network.

Estimating the effects of network covariates on subgroup insularity with a hierarchical mixed membership stochastic blockmodel

Social networks describe the relationships and interactions among a group of individuals. In many peer relationships, individuals tend to associate more often with some members than others, forming subgroups or clusters. Subgroup structure varies across networks; subgroups may be insular, appearing distinct and isolated from one another, or subgroups may be so integrated that subgroup structure is not visually apparent, and there are numerous ways of quantifying these types of structures. We propose a new model that relates the amount of subgroup integration to network attributes, building on the mixed membership stochastic blockmodel (Airoldi et al., 2008) and subsequent work by Sweet and Zheng (2017) and Sweet et al. (2014). We explore some of the operating characteristics of this model with simulated data and apply this model to determine the relationship between teachers’ instructional practices and their classrooms’ peer network subgroup structure.