Completeness of Community Structure in Networks

Jia-Rong Xie,Pan Zhang,Bing-Hong Wang,Hai-Feng Zhang

doi:10.1038/s41598-017-05585-6

Abstract

By defining a new measure to community structure, exclusive modularity, and based on cavity method of statistical physics, we develop a mathematically principled method to determine the completeness of community structure, which represents whether a partition that is either annotated by experts or given by a community-detection algorithm, carries complete information about community structure in the network. Our results demonstrate that the expert partition is surprisingly incomplete in some networks such as the famous political blogs network, indicating that the relation between meta-data and community structure in real-world networks needs to be re-examined. As a byproduct we find that the exclusive modularity, which introduces a null model based on the degree-corrected stochastic block model, is of independent interest. We discuss its applications as principled ways of detecting hidden structures, finding hierarchical structures without removing edges, and obtaining low-dimensional embedding of networks.

Highlights

Community structure, a partition of nodes into groups in such a way that the number of edges within groups is comparatively larger than the number of edges between groups, has attracted great attention over the past decade[1,2,3]
In the first we define a new measure to the community structure, namely Exclusive Modularity (EM), by using a null model based on the degree-corrected stochastic block model where a given partition is planted; in the second we map the network to a (Potts) spin glass system at finite temperature, use exclusive modularity as negative Hamiltonian and apply statistical mechanics to determine whether system has a retrieval state which represents a statistically significant community structure
On the synthetic networks generated by the stochastic block model and on the karate club network, by excluding the planted partition or expert partition, we find that the remaining network has no retrieval state, which indicates that the partitions contain complete information of community structure

Summary

Introduction

A partition of nodes into groups in such a way that the number of edges within groups is comparatively larger than the number of edges between groups, has attracted great attention over the past decade[1,2,3]. These include modularity-based methods[7,8,9,10], spectral clustering[11,12,13] and statistical inference[14, 15] For evaluating their performance, those methods are usually validated and compared on two kinds of networks, synthetic benchmark networks with plant-in structure[16,17,18], and real-world networks for which there is an expert partition[17, 19, 20]. The expert partition is considered to describe the most important information regarding the network connectivity and is given by domain experts or based on additional information of the network It may be incomplete in describing communities in the network, due to artificial operations to the labels, or some unknown information to the community structures. As political blogs network, our algorithm reports that the expert partition is incomplete, showing that there are other information hidden by the expert partition

Methods

Results

Conclusion