Limits of modularity maximization in community detection

Community structure detection from networks with weighted modularity

The human brain can be regarded as a complex network with interacting connections between brain regions. Complex brain network analyses have been widely applied to functional magnetic resonance imaging (fMRI) data and have revealed the existence of community structures in brain networks. The identification of communities may provide insight into understanding the topological functions of brain networks. Among various community detection methods, the modularity maximization (MM) method has the advantages of model conciseness, fast convergence and strong adaptability to large-scale networks and has been extended from single-layer networks to multilayer networks to investigate the community structure changes of brain networks. However, the problems of MM, suffering from instability and failing to detect hierarchical community structure in networks, largely limit the application of MM in the community detection of brain networks. In this study, we proposed the weighted modularity maximization (WMM) method by using the weight matrix to weight the adjacency matrix and improve the performance of MM. Moreover, we further proposed the two-step WMM method to detect the hierarchical community structures of networks by utilizing node attributes. The results of the synthetic networks without node attributes demonstrated that WMM showed better partition accuracy than both MM and robust MM and better stability than MM. The two-step WMM method showed better accuracy of community partitioning than WMM for synthetic networks with node attributes. Moreover, the results of resting state fMRI (rs-fMRI) data showed that two-step WMM had the advantage of detecting the hierarchical communities over WMM and was more insensitive to the density of the rs-fMRI networks than WMM.

The technology used to detect community structures in graphs, or graph clustering technology, is important in a wide range of disciplines, such as sociology, biology, and computer science. Previously, many successful community detection methods have relied on the optimization of a quantity referred to as modularity, which is a quality index for the partition of a graph into communities. However, such methods suffer from a key drawback, namely, the inability to identify relatively small communities. To overcome this drawback, we propose a novel community detection method that can detect small communities. This is based on the property that a random walker will not readily leave a community even if it is small. The work presented in this paper demonstrates that our method detects both small and large communities in the practical application of clustering tourist attraction images obtained from Flickr.

Information mining from networks by identifying communities is an important problem across a number of research fields including social science, biology, physics, and medicine. Most existing community detection algorithms are graph theoretic and lack the ability to detect accurate community boundaries if the ratio of intra-community to inter-community links is low. Also, algorithms based on modularity maximization may fail to resolve communities smaller than a specific size if the community size varies significantly. We propose a fundamentally different community detection algorithm based on the fact that each network community spans a different subspace in the geodesic space. Therefore, each node can only be efficiently represented as a linear combination of nodes spanning the same subspace (Fig. 1). To make the process of community detection more robust, we use sparse linear coding with l1 norm constraint. In order to find a community label for each node, sparse spectral clustering algorithm is used. The proposed community detection technique is compared with more than ten state of the art methods on two benchmark networks (with known clusters) using normalized mutual information criterion. Our proposed algorithm outperformed existing methods with a significant margin on both benchmark networks.

Many complex networks show signs of modular structure, uncovered by community detection. Although many methods succeed in revealing various partitions, it remains difficult to detect at what scale some partition is significant. This problem shows foremost in multi-resolution methods. We here introduce an efficient method for scanning for resolutions in one such method. Additionally, we introduce the notion of “significance” of a partition, based on subgraph probabilities. Significance is independent of the exact method used, so could also be applied in other methods, and can be interpreted as the gain in encoding a graph by making use of a partition. Using significance, we can determine “good” resolution parameters, which we demonstrate on benchmark networks. Moreover, optimizing significance itself also shows excellent performance. We demonstrate our method on voting data from the European Parliament. Our analysis suggests the European Parliament has become increasingly ideologically divided and that nationality plays no role.

Understanding the community structure in graphs arising from complex network is an important and difficult problem, both from theoretical and practical points of views. Although a lot of community detection algorithms have been proposed in the last decade, there is still no satisfactory way to determine if a given network possesses or not a community structure, that is, its nodes can be partitioned in well separated clusters. In this paper, we propose a new criterion based on the study of the formation of consensual communities obtained by running several times a non deterministic algorithm. By testing on synthetic benchmarks (with known structure) and on several real world networks, we show that the graphs can be categorized in several classes according to the dynamic of the consensual communities formation process. This result is promising to derive new approaches to characterize the modular structure of graphs.

The identification of community structure in graphs continues to attract great interest in several fields. Network neuroscience is particularly concerned with this problem considering the key roles communities play in brain processes and functionality. Most methods used for community detection in brain graphs are based on the maximization of a parameter-dependent modularity function that often obscures the physical meaning and hierarchical organization of the partitions of network nodes. In this work, we present a new method able to detect communities at different scales in a natural, unrestricted way. First, to obtain an estimation of the information flow in the network we release random walkers to freely move over it. The activity of the walkers is separated into oscillatory modes by using empirical mode decomposition. After grouping nodes by their co-occurrence at each time scale, k-modes clustering returns the desired partitions. Our algorithm was first tested on benchmark graphs with favorable performance. Next, it was applied to real and simulated anatomical and/or functional connectomes in the macaque and human brains. We found a clear hierarchical repertoire of community structures in both the anatomical and the functional networks. The observed partitions range from the evident division in two hemispheres –in which all processes are managed globally– to specialized communities seemingly shaped by physical proximity and shared function. Additionally, the spatial scales of a network's community structure (characterized by a measure we term within-communities path length) appear inversely proportional to the oscillatory modes’ average frequencies. The proportionality constant may constitute a network-specific propagation velocity for the information flow. Our results stimulate the research of hierarchical community organization in terms of temporal scales of information flow in the brain network.

Cultures of the Central Highlands, New Guinea

Information mining from networks by identifying communities is an important problem across a number of research fields including social science, biology, physics, and medicine. Most existing community detection algorithms are graph theoretic and lack the ability to detect accurate community boundaries if the ratio of intra-community to inter-community links is low. Also, algorithms based on modularity maximization may fail to resolve communities smaller than a specific size if the community size varies significantly. In this paper, we present a fundamentally different community detection algorithm based on the fact that each network community spans a different subspace in the geodesic space. Therefore, each node can only be efficiently represented as a linear combination of nodes spanning the same subspace. To make the process of community detection more robust, we use sparse linear coding with $\ell _1$ norm constraint. In order to find a community label for each node, sparse spectral clustering algorithm is used. The proposed community detection technique is compared with more than 10 state of the art methods on two benchmark networks (with known clusters) using normalized mutual information criterion. Our proposed algorithm outperformed existing algorithms with a significant margin on both benchmark networks. The proposed algorithm has also shown excellent performance on three real-world networks.

Detecting community structures in graphs is a well studied problem in graph data analytics. Unprecedented growth in graph structured data due to the development of the world wide web and social networks in the past decade emphasizes the need for fast graph data analytics techniques. In this paper we present a simple yet efficient approach to detect communities in large scale graphs by modifying the sequential Louvain algorithm for community detection. The proposed distributed memory parallel algorithm targets the costly first iteration of the initial method by parallelizing it. Experimental results on a MPI setup with 128 parallel processes shows that up to ≈5× performance improvement is achieved as compared to the sequential version while not compromising the correctness of the final result.

Revealing ideal community structure efficiently is very important for scientists from many fields. However, it is difficult to infer an ideal community division structure by only analyzing the topology information due to the increment and complication of the social network. Recent research on community detection uncovers that its performance could be improved by incorporating the node attribute information. Along this direction, this paper improves the Blondel–Guillaume–Lambiotte (BGL) method, which is a fast algorithm based on modularity maximization, by integrating the community attribute entropy. To fulfill this goal, our algorithm minimizes the community attribute entropy by removing the boundary nodes which are generated in the modularity maximization at each iteration. By this way, the communities detected by our algorithm make a balance between modularity maximization and community attribute entropy minimization. In addition, another merit of our algorithm is that it is free of parameters. Comprehensive experiments have been conducted on both artificial and real networks to compare the proposed community detection algorithm with several state-of-the-art ones. As the experimental results indicate, our algorithm demonstrates superior performance.

Graph vertices are often organized into groups that seem to live fairly independently of the rest of the graph, with which they share but a few edges, whereas the relationships between group members are stronger, as shown by the large number of mutual connections. Such groups of vertices, or communities, can be considered as independent compartments of a graph. Detecting communities is of great importance in sociology, biology and computer science, disciplines where systems are often represented as graphs. The task is very hard, though, both conceptually, due to the ambiguity in the definition of community and in the discrimination of different partitions and practically, because algorithms must find “good” partitions among an exponentially large number of them. Other complications are represented by the possible occurrence of hierarchies, i.e. communities which are nested inside larger communities, and by the existence of overlaps between communities, due to the presence of nodes belonging to more groups. All these aspects are dealt with in some detail and many methods are described, from traditional approaches used in computer science and sociology to recent techniques developed mostly within statistical physics.

Community Structure in Graphs

Graph vertices are often organized into groups that seem to live fairly independently of the rest of the graph, with which they share but a few edges, whereas the relationships between group members are stronger, as shown by the large number of mutual connections. Such groups of vertices, or communities, can be considered as independent compartments of a graph. Detecting communities is of great importance in sociology, biology and computer science, disciplines where systems are often represented as graphs. The task is very hard, though, both conceptually, due to the ambiguity in the definition of community and in the discrimination of different partitions and practically, because algorithms must find ``good'' partitions among an exponentially large number of them. Other complications are represented by the possible occurrence of hierarchies, i.e. communities which are nested inside larger communities, and by the existence of overlaps between communities, due to the presence of nodes belonging to more groups. All these aspects are dealt with in some detail and many methods are described, from traditional approaches used in computer science and sociology to recent techniques developed mostly within statistical physics.

Graph neural networks (GNNs) are powerful models to handle graph data and can achieve state-of-the-art in many critical tasks including node classification and link prediction. However, existing graph neural networks still face both challenges of over-smoothing and over-squashing based on previous literature. To this end, we propose a new Curvature-based topology-aware Dropout sampling technique named CurvDrop, in which we integrate the Discrete Ricci Curvature into graph neural networks to enable more expressive graph models. Also, this work can improve graph neural networks by quantifying connections in graphs and using structural information such as community structures in graphs. As a result, our method can tackle the both challenges of over-smoothing and over-squashing with theoretical justification. Also, numerous experiments on public datasets show the effectiveness and robustness of our proposed method. The code and data are released in https://github.com/liu-yang-maker/Curvature-based-Dropout.