Community Detection Methods Research Articles

Mapping flows on weighted and directed networks with incomplete observations

Abstract Detecting significant community structure in networks with incomplete observations is challenging because the evidence for specific solutions fades away with missing data. For example, recent research shows that flow-based community detection methods can highlight spurious communities in sparse undirected and unweighted networks with missing links. Current Bayesian approaches developed to overcome this problem do not work for incomplete observations in weighted and directed networks that describe network flows. To overcome this gap, we extend the idea behind the Bayesian estimate of the map equation for unweighted and undirected networks to enable more robust community detection in weighted and directed networks. We derive an empirical Bayes estimate of the transitions rates that can incorporate metadata information and show how an efficient implementation in the community-detection method Infomap provides more reliable communities even with a significant fraction of data missing.

Discovering transition patterns among OpenStreetMap feature classes based on the Louvain method

AbstractNumerous studies have shown that OpenStreetMap (OSM) data can achieve high positional quality. However, the thematic attributes of OSM objects can be modified several times, which has a large impact on semantic heterogeneity. Identifying transition patterns within OSM feature classes is an important preliminary step for the tag recommendation algorithm, which can reduce the number of modifications and enhance the efficiency of OSM data updates. In this article, we propose an approach for discovering transition patterns among OSM feature classes. We first produced the transition matrix of feature classes and then developed a graph. Next, the Louvain method for community detection was utilized to cluster the feature classes. OSM data from Indiana, USA, and the Azores, Portugal, were used for our experiments. Some transition patterns were discovered: (1) many feature classes with the most transitions are the same in both datasets and most transitions occur in road‐related feature classes; (2) people tend to tag general classes if they are unsure of the specific classes of tagged objects; and (3) most class transitions occurred as a result of volunteers improving the specificity and precision of feature classes. Moreover, consistently confusing concept pairs were identified.

MSN‐CDF: New community detection framework to improve routing in mobile social networks

SummaryIn mobile social networks (MSN), portable devices of mobile users communicate with each other. The contact and intercontact times are the two most important parameters that specify the behavior of MSN. Many social features such as similarity, centrality, closeness, and distance are derived from contact and intercontact time patterns. Those parameters and features can be used in the community detection process in community‐based routing as a vector weight of network graph edges. But there is no solution for proper exploitation of them as a multicriteria optimization of community detection algorithms so that the communities become more adaptive with the MSN quiddity and the computational complexity of the algorithms to be stay constant. In this paper, we propose the MSN‐adaptive community detection framework, MSN‐CDF, which meets these demands. We improve the quality function of community detection methods, for example, the Intensity in clique percolation methods and the Modularity in modularity‐based methods, through our new model‐based joint optimization of their quality functions. To this end, to express the MSN contact and social features, we develop a criterion called MCSF. It holds the contact time and intercontact time distributions parameters and implies their social features for MSN purposes. This criterion has as little traffic as possible and enjoys from the probabilistic joint distribution model. The simulations show that MSN‐CDF reduces the network overhead significantly compared with the DiBUBB, Epidemic, and Dynamic Routing. Moreover, it outperforms DiBUBB and Dynamic Routing in terms of delivery rate and delivery latency.

A Spin-Glass Model for Semi-Supervised Community Detection

Current modularity-based community detection methods show decreased performance as relational networks become increasingly noisy. These methods also yield a large number of diverse community structures as solutions, which is problematic for applications that impose constraints on the acceptable solutions or in cases where the user is focused on specific communities of interest. To address both of these problems, we develop a semi-supervised spin-glass model that enables current community detection methods to incorporate background knowledge in the forms of individual labels and pairwise constraints. Unlike current methods, our approach shows robust performance in the presence of noise in the relational network, and the ability to guide the discovery process toward specific community structures. We evaluate our algorithm on several benchmark networks and a new political sentiment network representing cooperative events between nations that was mined from news articles over six years.

Modularity in Biological Networks.

Network modeling, from the ecological to the molecular scale has become an essential tool for studying the structure, dynamics and complex behavior of living systems. Graph representations of the relationships between biological components open up a wide variety of methods for discovering the mechanistic and functional properties of biological systems. Many biological networks are organized into a modular structure, so methods to discover such modules are essential if we are to understand the biological system as a whole. However, most of the methods used in biology to this end, have a limited applicability, as they are very specific to the system they were developed for. Conversely, from the statistical physics and network science perspective, graph modularity has been theoretically studied and several methods of a very general nature have been developed. It is our perspective that in particular for the modularity detection problem, biology and theoretical physics/network science are less connected than they should. The central goal of this review is to provide the necessary background and present the most applicable and pertinent methods for community detection in a way that motivates their further usage in biological research.

A new single-chromosome evolutionary algorithm for community detection in complex networks by combining content and structural information

Community detection is an important step in perceiving network structure and performance for complex network analysis. The rapid growth of network data in recent years has piqued the interest of many researchers in community detection. The majority of community detection methods only consider the network structure. Nonetheless, real-world network nodes may have some characteristics that can be useful for community detection. This study proposed a novel single-chromosome evolutionary algorithm with a distinctive architecture modification operator for community detection in complex networks using a combination of structural and content information. To this end, a novel virtual network was created by taking into account the structure and content of nodes, and communities were discovered for this network by optimizing the objective function (and using the combinatorial adjacency matrix instead of the structural adjacency matrix) in a series of steps. The nodes in this network were the same as the nodes in the main network; however, the links were developed based on similarities between nodes and their structural neighborhood. The proposed algorithm also included a method for sorting new nodes in order to determine the analysis order of nodes along with the local improvement of solution, as well as a new criterion, CS, for measuring the content similarity of nodes. The proposed algorithm was evaluated in real-networks and compared to various state-of-the-art and widely used methods. The Friedman rank algorithm was then used to rank the proposed algorithm and the existing methods using six real networks. According to the NMI criterion used in the Friedman rank test, the rank of the proposed algorithms increased by 96.8762%, 70.2693%, 26.0005%, 23.5294%, 46.5109%, and 23.5294% compared respectively with ASCD-ARC, BTLSC, Adapt-SA, PSB-PG, RSECD, and NEMBP, which have all been proposed in recent years.

A fast variable neighborhood search approach for multi-objective community detection

Community detection in social networks is becoming one of the key tasks in social network analysis, since it helps analyzing groups of users with similar interests. This task is also useful in different areas, such as biology (interactions of genes and proteins), psychology (diagnostic criteria), or criminology (fraud detection). This paper presents a metaheuristic approach based on Variable Neighborhood Search (VNS) which leverages the combination of quality and diversity of a constructive procedure inspired in Greedy Randomized Adaptative Search Procedure (GRASP) for detecting communities in social networks. In this work, the community detection problem is modeled as a bi-objective optimization problem, where the two objective functions to be optimized are the Negative Ratio Association (NRA) and Ratio Cut (RC), two objectives that have already been proven to be in conflict. To evaluate the quality of the obtained solutions, we use the Normalized Mutual Information (NMI) metric for the instances under evaluation whose optimal solution is known, and modularity for those in which the optimal solution is unknown. Furthermore, we use metrics widely used in multi-objective optimization community to evaluate solutions, such as coverage, ε-indicator, hypervolume, and inverted generational distance. The obtained results outperform the state-of-the-art method for community detection over a set of real-life instances in both, quality and computing time.

A Novel Emerging Topic Identification and Evolution Discovery Method on Time-Evolving and Heterogeneous Online Social Networks

With the fast development of web 2.0, information generation and propagation among online users become deeply interweaved. How to effectively and immediately discover the new emerging topic and further how to uncover its evolution law are still wide open and urgently needed by both research and practical fields. This paper proposed a novel early emerging topic detection and its evolution law identification framework based on dynamic community detection method on time-evolving and scalable heterogeneous social networks. The framework is composed of three major steps. Firstly, a time-evolving and scalable complex network denoted as KeyGraph is built up by deeply analyzing the text features of all kinds of data crawled from heterogeneous online social network platforms; secondly, a novel dynamic community detection method is proposed by which the new emerging topic is detected on the modeled time-evolving and scalable KeyGraph network; thirdly, a unified directional topic propagation network modeled by a great number of short texts including microblogs and news titles is set up, and the topic evolution law of the previously detected early emerging topic is identified by fully utilizing local network variations and modularity optimization of the “time-evolving” and directional topic propagation network. Our method is proved to yield preferable results on both a huge amount of computer-generated test data and a great amount of real online network data crawled from mainstream heterogeneous social networks.

Extracting complements and substitutes from sales data: a network perspective

The complementarity and substitutability between products are essential concepts in retail and marketing. Qualitatively, two products are said to be substitutable if a customer can replace one product by the other, while they are complementary if they tend to be bought together. In this article, we take a network perspective to help automatically identify complements and substitutes from sales transaction data. Starting from a bipartite product-purchase network representation, with both transaction nodes and product nodes, we develop appropriate null models to infer significant relations, either complements or substitutes, between products, and design measures based on random walks to quantify their importance. The resulting unipartite networks between products are then analysed with community detection methods, in order to find groups of similar products for the different types of relationships. The results are validated by combining observations from a real-world basket dataset with the existing product hierarchy, as well as a large-scale flavour compound and recipe dataset.

Community detection in directed acyclic graphs of adversary interactions

Certain real networks are represented as directed acyclic graphs (DAGs) and the links represent adversary interactions between two entities, such as food webs where links exist only from prey to predators and temporal war networks where links point from attackers to defenders. In such DAGs, similar nodes may form communities, such as top carnivores in food webs and war alliances in war networks, where nodes cannot be directly connected but have similar orders and neighbors. However, most previous community detection methods are developed based on an assumption that a link between nodes indicates similarity, not applicable to such cases. In this work, we define the community in DAGs of adversary interactions based on the nodes’ orders and similarities, and propose a Katz–Simrank method to detect communities. We first convert the DAG into an equivalent weighted undirected network based on nodes’ orders and similarities, then the problem of community detection in such DAG can be converted into an equivalent problem of detecting cliques in this weighted undirected network. We apply this method to both synthetic and real DAGs, such as food webs and war networks, and find that Katz–Simrank method could effectively identify communities and demonstrates superior performance over other baseline methods in DAG community detection.

Exploring Biographical Ties among Party and State Leaders in China: A Social Network Analysis

ABSTRACT This article offers a new perspective on Chinese elite politics in the Xi Jinping era by overcoming the drawbacks of the factional approach to the study of the Chinese elite. It analyses professional connections among 71 members of the category of party and state leaders of China based on information drawn from their official biographies and compiled by the authors into a single dataset. Four types of centrality – degree, betweenness, closeness, and eigenvector centralities – as well as the Louvain method for community detection, are applied to weighted and unweighted edge lists deriving from the dataset. The results reveal a positive correlation between the eigenvector centrality (unweighted) and the level of the formal political influence of a member of the Chinese elite. A distinct hierarchy of power is observed, with Politburo members and those coming from important areas, such as Shanghai, Fujian, and Zhejiang, ranking at the top. Louvain clustering (weighted) distributes members of the elite across institutional affiliation, while for the unweighted edge list it identifies both territorial and institutional, as well as mixed, clusters. The model reflects the formal political influence of the members of the elite and visualises the existence of informal groups.

A Core Drug Discovery Framework from Large-Scale Literature for Cold Pathogenic Disease Treatment in Traditional Chinese Medicine.

Cold pathogenic disease is a widespread disease in traditional Chinese medicine, which includes influenza and respiratory infection associated with high incidence and mortality. Discovering effective core drugs in Chinese medicine prescriptions for treating the disease and reducing patients' symptoms has attracted great interest. In this paper, we explore the core drugs for curing various syndromes of cold pathogenic disease from large-scale literature. We propose a core drug discovery framework incorporating word embedding and community detection algorithms, which contains three parts: disease corpus construction, drug network generation, and core drug discovery. First, disease corpus is established by collecting and preprocessing large-scale literature about the Chinese medicine treatment of cold pathogenic disease from China National Knowledge Infrastructure. Second, we adopt the Chinese word embedding model SSP2VEC for mining the drug implication implied in the literature; then, a drug network is established by the semantic similarity among drugs. Third, the community detection method COPRA based on label propagation is adopted to reveal drug communities and identify core drugs in the drug network. We compute the community size, closeness centrality, and degree distributions of the drug network to analyse the patterns of core drugs. We acquire 4681 literature from China national knowledge infrastructure. Twelve significant drug communities are discovered, in which the top-10 drugs in every drug community are recognized as core drugs with high accuracy, and four classical prescriptions for treating different syndromes of cold pathogenic disease are discovered. The proposed framework can identify effective core drugs for curing cold pathogenic disease, and the research can help doctors to verify the compatibility laws of Chinese medicine prescriptions.

Where Online Friends Meet: Social Communities in Location-Based Networks

Recent research suggests that, as in offline scenarios, spatial proximity increases the likelihood that two individuals establish an online social connection, and geographic closeness could therefore influence the formation of online communities. In this work we present a study of communities in two online social networks with location-sharing features and analyze their social and spatial properties. We study the places users visit to understand whether communities revolve around places or whether they exist independently. Our results suggest that community structure in social networks may arise from both social and spatial factors, so that exploiting information about the places where people go could benefit the definition of new community detection methods and community evolution models.

A signal processing perspective to community detection in dynamic networks

Many real world systems ranging from biological to social systems can be modeled as networks. As network data becomes more ubiquitous, it is important to reduce the large-scale networks into smaller modules or communities. Most of the original research in community detection has focused on static networks while many real world networks such as social and biological networks exhibit varying topology over time. For this reason, over the past twenty years various community detection methods for dynamic networks have been developed in physics, statistics, computer science and signal processing. In this paper, we present an overview of the current state-of-the-art in community detection for dynamic networks and illustrate some of the shortcomings of existing methods. We also outline future challenges and opportunities in this research area.

Community detection with dependent connectivity

In network analysis, within-community members are more likely to be connected than between-community members, which is reflected in that the edges within a community are intercorrelated. However, existing probabilistic models for community detection such as the stochastic block model (SBM) are not designed to capture the dependence among edges. In this paper, we propose a new community detection approach to incorporate intra-community dependence of connectivities through the Bahadur representation. The proposed method does not require specifying the likelihood function, which could be intractable for correlated binary connectivities. In addition, the proposed method allows for heterogeneity among edges between different communities. In theory, we show that incorporating correlation information can achieve a faster convergence rate compared to the independent SBM, and the proposed algorithm has a lower estimation bias and accelerated convergence compared to the variational EM. Our simulation studies show that the proposed algorithm outperforms the existing multinetwork community detection methods assuming conditional independence among edges. We also demonstrate the application of the proposed method to agricultural product trading networks from different countries and to brain fMRI imaging networks.

Null Models and Community Detection in Multi-Layer Networks

Multi-layer networks of multiplex type represent relational data on a set of entities (nodes) with multiple types of relations (edges) among them where each type of relation is represented as a network layer. A large group of popular community detection methods in networks are based on optimizing a quality function known as the modularity score, which is a measure of the extent of presence of module or community structure in networks compared to a suitable null model. Here we introduce several multi-layer network modularity and model likelihood quality function measures using different null models of the multi-layer network, motivated by empirical observations in networks from a diverse field of applications. In particular, we define multi-layer variants of the Chung-Lu expected degree model as null models that differ in their modeling of the multi-layer degrees. We propose simple estimators for the models and prove their consistency properties. A hypothesis testing procedure is also proposed for selecting an appropriate null model for data. These null models are used to define modularity measures as well as model likelihood based quality functions. The proposed measures are then optimized to detect the optimal community assignment of nodes (Code available at: https://u.osu.edu/subhadeep/codes/ ). We compare the effectiveness of the measures in community detection in simulated networks and then apply them to four real multi-layer networks.

Overcoming the Link Prediction Limitation in Sparse Networks using Community Detection

Link prediction seeks to detect missing links and the ones that may be established in the future given the network structure or node features. Numerous methods have been presented for improving the basic unsupervised neighbourhood-based methods of link prediction. A major issue confronted by all these methods, is that many of the available networks are sparse. This results in high volume of computation, longer processing times, more memory requirements, and more poor results. This research has presented a new, distinct method for link prediction based on community detection in large-scale sparse networks. Here, the communities over the network are first identified, and the link prediction operations are then performed within each obtained community using neighbourhood-based methods. Next, a new method for link prediction has been carried out between the clusters with a specified manner for maximal utilization of the network capacity. Utilized community detection algorithms are Best partition, Link community, Info map and Girvan-Newman, and the datasets used in experiments are Email, HEP, REL, Wikivote, Word and PPI. For evaluation of the proposed method, three measures have been used: precision, computation time and AUC. The results obtained over different datasets demonstrate that extra calculations have been prevented, and precision has been increased. In this method, runtime has also been reduced considerably. Moreover, in many cases Best partition community detection method has good results compared to other community detection algorithms.

A novel fuzzy community detection method based on improved community adjacency list

With the development of economy and society, network analysis is widely used in more and more fields. Signed network has a good effect in the process of representation and display. As an important part of network analysis, fuzzy community detection plays an increasingly important role in analyzing and visualizing the real world. Fuzzy community detection helps to detect nodes that belong to some communities but are still closely related to other communities. These nodes are helpful for mining information from the network more realistically. However, there is little research in this field. This paper proposes a fuzzy community detection algorithm based on pointer and adjacency list. The model adopts a new ICALF network data structure, which can achieve the effect of storing community partition structure and membership value between community and node at the same time, with low time complexity and storage space. Experiments on real networks verify the correctness of the method, and prove that the method is suitable for large-scale network applications.

Finding core–periphery structures in large networks

Finding core–periphery structures in networks is very useful in many disciplines such as biology and sociology. However, most of the previous works focus on the single core–periphery structure in the network. A few recent algorithms considering multiple core–periphery are usually not suitable for large networks. Inspired by the modularity maximization method for community detection, we propose a simple but effective approach to detect core–periphery structures in this work. Moreover, we propose a metric called core–periphery score to evaluate the performance of core–periphery structure detection algorithms. In the experiment, we find that the score is consistent with the normalized mutual information when ground-truth structures are given. Our approach also outperforms other core–periphery detection algorithms for randomly generated networks and real-world networks.

ACSIMCD: A 2-phase framework for detecting meaningful communities in dynamic social networks

Detecting and analyzing community structure is a challenging topic in dynamic social network analysis. Although the number of methods in this area is on the rise, there are only a few algorithms that can discover meaningful communities based on different aspects of social networks. Indeed, social networks contain various information sources that can be used to analyze them. The most important part of this information is related to users’ topics of interest (content information) and users’ interactions (structure information). One promising solution to discover meaningful communities is to combine these two concepts. Based on this, we introduce ACSIMCD, a 2-phase framework for discovering and updating community structure without recomputing them from scratch at each snapshot. This article mainly includes two parts. In the first part, a static community detection algorithm which is called Content and Structure Information based Method for Community Detection (CSIMCD for short) is proposed to discover the initial community structure. The CSIMCD uses a hybrid approach founded on statistical and semantic measures to extract the users’ topics of interest. Accordingly, the original network is divided into several clusters (topical clusters) so that each one represents a distinct topic, then by performing a link analysis on each topical cluster, the communities are detected. In the second part, we propose ACSIMCD (Adaptive CSIMCD), an adaptive method for detecting and updating community structure in dynamic social networks. More precisely, the ACSIMCD explores the topics of interest of each changed node to identify the topical cluster it belongs to. After that, we update the community structure in this topical cluster, and we keep others as they are. We compare the ACSIMCD model with algorithms from different approaches including content-based methods on real-world networks. The experimental results showed that ACSIMCD produces a community structure of high quality from the perspective of links and interests compared with the classical methods, and that it is able to process network changes effectively in a reasonable time scale.