Communities In Real-world Networks Research Articles

Discrete-Time Quantum Walks Community Detection in Multi-Domain Networks

Abstract The problem of detecting communities in real-world networks has been extensively studied in the past, but most of the existing approaches work on single-domain networks, i.e. they consider only one type of relationship between nodes. Single-domain networks may contain noisy edges and they may lack some important information. Thus, some authors have proposed to consider the multiple relationships that connect the nodes of a network, thus obtaining multi-domain networks. However, most community detection approaches are limited to multi-layer networks, i.e. networks generated from the superposition of several single-domain networks (called layers) that are regarded as independent of each other. In addition to being computationally expensive, multi-layer approaches might yield inaccurate results because they ignore potential dependencies between layers. This paper proposes a multi-domain discrete-time quantum walks (MDQW) model for multi-domain networks. First, the walking space of network nodes in multi-domain network is constructed. Second, the quantum permutation circuit of the coin state is designed based on the coded particle state. Then, using different coin states, the shift operator performs several quantum walks on the particles. Finally, the corresponding update rule is selected to move the node according to the measurement result of the quantum state. With continuous update iteration, the shift operator automatically optimizes the discovered community structure. We experimentally compared our MDQW method with four state-of-the-art competitors on five real datasets. We used the normalized mutual information (NMI) to compare clustering quality, and we report an increase in NMI of up to 3.51 of our MDQW method in comparison with the second-best performing competitor. The MDQW method is much faster than its competitors, allowing us to conclude that MDQW is a useful tool in the analysis of large real-life multi-domain networks. Finally, we illustrate the usefulness of our approach on two real-world case studies.

Understanding digitally enabled complex networks: a plural granulation based hybrid community detection approach

PurposeCommunities representing groups of agents with similar interests or functions are one of the essential features of complex networks. Finding communities in real-world networks is critical for analyzing complex systems in various areas ranging from collaborative information to political systems. Given the different characteristics of networks and the capability of community detection in handling a plethora of societal problems, community detection methods represent an emerging area of research. Contributing to this field, the authors propose a new community detection algorithm based on the hybridization of node and link granulation.Design/methodology/approachThe proposed algorithm utilizes a rough set-theoretic concept called closure on networks. Initial sets are constructed by using neighborhood topology around the nodes as well as links and represented as two different categories of granules. Subsequently, the authors iteratively obtain the constrained closure of these sets. The authors use node mutuality and link mutuality as merging criteria for node and link granules, respectively, during the iterations. Finally, the constrained closure subsets of nodes and links are combined and refined using the Jaccard similarity coefficient and a local density function to obtain communities in a binary network.FindingsExtensive experiments conducted on twelve real-world networks followed by a comparison with state-of-the-art methods demonstrate the viability and effectiveness of the proposed algorithm.Research limitations/implicationsThe study also contributes to the ongoing effort related to the application of soft computing techniques to model complex systems. The extant literature has integrated a rough set-theoretic approach with a fuzzy granular model (Kundu and Pal, 2015) and spectral clustering (Huang and Xiao, 2012) for node-centric community detection in complex networks. In contributing to this stream of work, the proposed algorithm leverages the unexplored synergy between rough set theory, node granulation and link granulation in the context of complex networks. Combined with experiments of network datasets from various domains, the results indicate that the proposed algorithm can effectively reveal co-occurring disjoint, overlapping and nested communities without necessarily assigning each node to a community.Practical implicationsThis study carries important practical implications for complex adaptive systems in business and management sciences, in which entities are increasingly getting organized into communities (Jacucci et al., 2006). The proposed community detection method can be used for network-based fraud detection by enabling experts to understand the formation and development of fraudulent setups with an active exchange of information and resources between the firms (Van Vlasselaer et al., 2017). Products and services are getting connected and mapped in every walk of life due to the emergence of a variety of interconnected devices, social networks and software applications.Social implicationsThe proposed algorithm could be extended for community detection on customer trajectory patterns and design recommendation systems for online products and services (Ghose et al., 2019; Liu and Wang, 2017). In line with prior research, the proposed algorithm can aid companies in investigating the characteristics of implicit communities of bloggers or social media users for their services and products so as to identify peer influencers and conduct targeted marketing (Chau and Xu, 2012; De Matos et al., 2014; Zhang et al., 2016). The proposed algorithm can be used to understand the behavior of each group and the appropriate communication strategy for that group. For instance, a group using a specific language or following a specific account might benefit more from a particular piece of content than another group. The proposed algorithm can thus help in exploring the factors defining communities and confronting many real-life challenges.Originality/valueThis work is based on a theoretical argument that communities in networks are not only based on compatibility among nodes but also on the compatibility among links. Building up on the aforementioned argument, the authors propose a community detection method that considers the relationship among both the entities in a network (nodes and links) as opposed to traditional methods, which are predominantly based on relationships among nodes only.

Overcoming Bias in Community Detection Evaluation

Community detection is a key task to further understand the function and the structure of complex networks. Therefore, a strategy used to assess this task must be able to avoid biased and incorrect results that might invalidate further analyses or applications that rely on such communities. Two widely used strategies to assess this task are generally known as structural and functional. The structural strategy basically consists in detecting and assessing such communities by using multiple methods and structural metrics. On the other hand, the functional strategy might be used when ground truth data are available to assess the detected communities. However, the evaluation of communities based on such strategies is usually done in experimental configurations that are largely susceptible to biases, a situation that is inherent to algorithms, metrics and network data used in this task. Furthermore, such strategies are not systematically combined in a way that allows for the identification and mitigation of bias in the algorithms, metrics or network data to converge into more consistent results. In this context, the main contribution of this article is an approach that supports a robust quality evaluation when detecting communities in real-world networks. In our approach, we measure the quality of a community by applying the structural and functional strategies, and the combination of both, to obtain different pieces of evidence. Then, we consider the divergences and the consensus among the pieces of evidence to identify and overcome possible sources of bias in community detection algorithms, evaluation metrics, and network data. Experiments conducted with several real and synthetic networks provided results that show the effectiveness of our approach to obtain more consistent conclusions about the quality of the detected communities.

Detecting Hierarchical and Overlapping Network Communities Based on Opinion Dynamics

It is common for communities in real-world networks to possess hierarchical and overlapping structures, which make community detection even more challenging. In this paper, by investigating consensus process of the classical DeGroot model in opinion dynamics, we propose a novel method based on the cumulative opinion distance (COD) to discover hierarchical and overlapping communities. It is shown that this method is different from those classical algorithms relying on static fitness metrics that depict the inhomogeneous connectivity across the network. The proposed method is validated from two aspects. First, by estimating the eigenvectors of adjacency matrices, we investigate the detectability limit of our algorithms on random networks, which together with the results concerning the convergence speed of consensus guarantees the performance of our method theoretically. Second, experiments on both \tcb{large scale} real-world networks and artificial benchmarks show that our method is very effective and competitive on hierarchical modular graphs. In particular, it outperforms the state-of-the-art algorithms on overlapping community detection.

Node-community membership diversifies community structures: An overlapping community detection algorithm based on local expansion and boundary re-checking

Local expansion methods excel in efficiency for mining overlapping communities in real-world networks. However, two problems prevent such methods from identifying diversely structured communities. First, local expansion methods generate independent communities only. Second, local expansion methods depend heavily on quality functions. This work provides a solution for local expansion methods to identify diversely structured communities. The proposed overlapping community detection algorithm performs local expansion and boundary re-checking sub-processes in order. The local expansion process first gets a cover of the network, and then the boundary re-checking process optimizes the cover of the network resulting from the local expansion process. To solve the first problem, the proposed algorithm establishes associations between boundaries of adjacent communities via the boundary re-checking process. To solve the second problem, the proposed algorithm expands and optimizes communities based on node-community membership optimization. We compared the proposed algorithm to seven state-of-the-art algorithms by examining their performance on five groups of artificial networks and sixteen real-world networks. Experimental results showed that the proposed algorithm outperforms compared algorithms in identifying diversely structured communities.

Detecting community structure by belonging intensity analysis of intermediate nodes

Community structure is a common characteristic of complex networks and community detection is an important methodology to reveal the structure of real-world networks. In recent years, many algorithms have been proposed to detect the high-quality communities in real-world networks. However, these algorithms have shortcomings of performing calculation on the whole network or defining objective function and the number of commonties in advance, which affects the performance and complexity of community detection algorithms. In this paper, a novel algorithm has been proposed to detect communities in networks by belonging intensity analysis of intermediate nodes, named BIAS, which is inspired from the interactive behavior in human communication networks. More specifically, intermediate nodes are middlemen between different groups in social networks. BIAS algorithm defines belonging intensity using local interactions and metrics between nodes, and the belonging intensity of intermediate node in different communities is analyzed to distinguish which community the intermediate node belongs to. The experiments of our algorithm with other state-of-the-art algorithms on synthetic networks and real-world networks have shown that BIAS algorithm has better accuracy and can significantly improve the quality of community detection without prior information.

A node-priority based large-scale overlapping community detection using evolutionary multi-objective optimization

Community structure is one of the most important features in complex networks. However, with increasing of network scale, some existing methods cannot effectively detect the community structure of complex network, and the available methods mostly aimed at non-overlapping networks. In this paper, we focus on overlapping community detection in large-scale networks, because most of the communities in real-world networks are overlapped. In order to improve the accuracy of large-scale overlapping community detection, we suggest a community detection method based on node priority. The proposed algorithm has two advantages: (1) We define a priority function $${\text{f}}_{\text{NN}}$$ to assess the closeness between adjacent nodes. It explores the potential community structure in advance and reduces the scale of networks. (2) We employ NSGA-II and select all Pareto fronts to mine large-scale overlapping communities. The proposed algorithm is tested by the artificial and real datasets. The results show that the proposed algorithm can effectively improve the accuracy of community detection and has better optimization effect.

Towards Efficient Detection of Overlapping Communities in Massive Networks

Community detection is essential to analyzing and exploring natural networks such as social networks, biological networks, and citation networks. However, few methods could be used as off-the-shelf tools to detect communities in real world networks for two reasons. On the one hand, most existing methods for community detection cannot handle massive networks that contain millions or even hundreds of millions of nodes. On the other hand, communities in real world networks are generally highly overlapped, requiring that community detection method could capture the mixed community membership. In this paper, we aim to offer an off-the-shelf method to detect overlapping communities in massive real world networks. For this purpose, we take the widely-used Poisson model for overlapping community detection as starting point and design two speedup strategies to achieve high efficiency. Extensive tests on synthetic and large scale real networks demonstrate that the proposed strategies speedup the community detection method based on Poisson model by 1 to 2 orders of magnitudes, while achieving comparable accuracy at community detection.

Identification of Overlapping Communities via Constrained Egonet Tensor Decomposition

Detection of overlapping communities in real-world networks is a generally challenging task. Upon recognizing that a network is in fact the union of its egonets, a novel network representation using multi-way data structures is advocated in this contribution. The introduced sparse tensor-based representation exhibits richer structure compared to its matrix counterpart, and thus enables a more robust approach to community detection. To leverage this structure, a constrained tensor approximation framework is introduced using PARAFAC decomposition. The arising constrained trilinear optimization is handled via alternating minimization, where intermediate subproblems are solved using the alternating direction method of multipliers (ADMM) to ensure convergence. The factors obtained provide soft community memberships, which can further be exploited for crisp, and possibly-overlapping community assignments. The framework is further broadened to include time-varying graphs, where the edgeset as well as the underlying communities evolve through time. Performance of the proposed approach is assessed via tests on benchmark synthetic graphs as well as real-world networks. As corroborated by numerical tests, the proposed tensor-based representation captures multi-hop nodal connections, that is, connectivity patterns within single-hop neighbors, whose exploitation yields a more robust community identification in the presence of mixing as well as overlapping communities.

The ground truth about metadata and community detection in networks.

Across many scientific domains, there is a common need to automatically extract a simplified view or coarse-graining of how a complex system's components interact. This general task is called community detection in networks and is analogous to searching for clusters in independent vector data. It is common to evaluate the performance of community detection algorithms by their ability to find so-called ground truth communities. This works well in synthetic networks with planted communities because these networks' links are formed explicitly based on those known communities. However, there are no planted communities in real-world networks. Instead, it is standard practice to treat some observed discrete-valued node attributes, or metadata, as ground truth. We show that metadata are not the same as ground truth and that treating them as such induces severe theoretical and practical problems. We prove that no algorithm can uniquely solve community detection, and we prove a general No Free Lunch theorem for community detection, which implies that there can be no algorithm that is optimal for all possible community detection tasks. However, community detection remains a powerful tool and node metadata still have value, so a careful exploration of their relationship with network structure can yield insights of genuine worth. We illustrate this point by introducing two statistical techniques that can quantify the relationship between metadata and community structure for a broad class of models. We demonstrate these techniques using both synthetic and real-world networks, and for multiple types of metadata and community structures.

Mining Overlapping Communities in Real-world Networks Based on Extended Modularity Gain

Detecting communities plays a vital role in studying group level patterns of a social network and it can be helpful in developing several recommendation systems such as movie recommendation, book recommendation, friend recommendation and so on. Most of the community detection algorithms can detect disjoint communities only, but in the real time scenario, a node can be a member of more than one community at the same time, that leads to overlapping communities. A novel approach is proposed to detect such overlapping communities by extending the definition of newman’s modularity for overlapping communities. The proposed algorithm is tested on LFR benchmark networks with overlapping communities and on real-world networks. The performance of the algorithm is evaluated using popular metrics such as ONMI, Omega Index, F-score and Overlap modularity and the results are compared with its competent algorithms. It is observed that extended modularity gain can detect highly modular structures in complex networks with overlapping communities.

Overlapping Community Detection Using Neighborhood-Inflated Seed Expansion

Community detection is an important task in network analysis. A community (also referred to as a cluster) is a set of cohesive vertices that have more connections inside the set than outside. In many social and information networks, these communities naturally overlap. For instance, in a social network, each vertex in a graph corresponds to an individual who usually participates in multiple communities. In this paper, we propose an efficient overlapping community detection algorithm using a seed expansion approach. The key idea of our algorithm is to find good seeds, and then greedily expand these seeds based on a community metric. Within this seed expansion method, we investigate the problem of how to determine good seed nodes in a graph. In particular, we develop new seeding strategies for a personalized PageRank clustering scheme that optimizes the conductance community score. An important step in our method is the neighborhood inflation step where seeds are modified to represent their entire vertex neighborhood. Experimental results show that our seed expansion algorithm outperforms other state-of-the-art overlapping community detection methods in terms of producing cohesive clusters and identifying ground-truth communities. We also show that our new seeding strategies are better than existing strategies, and are thus effective in finding good overlapping communities in real-world networks.

Discovering natural communities in networks

Understanding and detecting natural communities in networks have been a fundamental challenge in networks, and in science generally. Recently, we proposed a hypothesis that homophyly/kinship is the principle of natural communities based on real network experiments, proposed a model of networks to explore the principle of natural selection in nature evolving, and proposed the measure of structure entropy of networks. Here we proposed a community finding algorithm by our measure of structure entropy of networks. We found that our community finding algorithm exactly identifies almost all natural communities of networks generated by natural selection, if any, and that the algorithm exactly identifies or precisely approximates almost all the communities planted in the networks of the existing models. We verified that our algorithm identifies or very well approximates the ground-truth communities of some real world networks, if the ground-truth communities are semantically well-defined, that our algorithm naturally finds the balanced communities, and that the communities found by our algorithm may have larger modularity than that by the algorithms based on modularity, for some networks. Our algorithm provides for the first time an approach to detecting and analyzing natural or true communities in real world networks. Our results demonstrate that structure entropy minimization is the principle of detecting the natural or true communities in large-scale networks.

Detecting overlapping communities in massive networks

Community detection is an essential work for network analysis. However, few methods could be used as off-the-shelf tools to detect communities in real-world networks for two main reasons: Real networks often contain millions of nodes or even hundreds of millions of nodes while most methods cannot handle networks at this scale. One node often belongs to multiple communities, posing another big challenge. In this paper, we circumvent the tricky problem of detecting overlapping communities using a two-stage framework, balancing efficiency and accuracy. Given a network, we first focus on efficiently finding its coarse-grained communities. Starting from them, we next obtain overlapping communities by optimizing a principled objective function. In this divide-and-conquer way, the framework achieves a much better performance than detecting overlapping communities from scratch. Extensive tests on synthetic and real networks demonstrate that it outperforms state-of-the-art methods in terms of both efficiency and accuracy.

Homophyly/kinship hypothesis: Natural communities, and predicting in networks

It has been a longstanding challenge to understand natural communities in real world networks. We proposed a community finding algorithm based on fitness of networks, two algorithms for prediction, accurate prediction and confirmation of keywords for papers in the citation network Arxiv HEP-TH (high energy physics theory), and the measures of internal centrality, external de-centrality, internal and external slopes to characterize the structures of communities. We implemented our algorithms on 2 citation and 5 cooperation graphs. Our experiments explored and validated a homophyly/kinship principle of real world networks. The homophyly/kinship principle includes: (1) homophyly is the natural selection in real world networks, similar to Darwin’s kinship selection in nature, (2) real world networks consist of natural communities generated by the natural selection of homophyly, (3) most individuals in a natural community share a short list of common attributes, (4) natural communities have an internal centrality (or internal heterogeneity) that a natural community has a few nodes dominating most of the individuals in the community, (5) natural communities have an external de-centrality (or external homogeneity) that external links of a natural community homogeneously distributed in different communities, and (6) natural communities of a given network have typical structures determined by the internal slopes, and have typical patterns of outgoing links determined by external slopes, etc. Our homophyly/kinship principle perfectly matches Darwin’s observation that animals from ants to people form social groups in which most individuals work for the common good, and that kinship could encourage altruistic behavior. Our homophyly/kinship principle is the network version of Darwinian theory, and builds a bridge between Darwinian evolution and network science.

Ubiquitousness of link-density and link-pattern communities in real-world networks

Community structure appears to be an intrinsic property of many complex real-world networks. However, recent work shows that real-world networks reveal even more sophisticated modules than classical cohesive (link-density) communities. In particular, networks can also be naturally partitioned according to similar patterns of connectedness among the nodes, revealing link-pattern communities. We here propose a propagation based algorithm that can extract both link-density and link-pattern communities, without any prior knowledge of the true structure. The algorithm was first validated on different classes of synthetic benchmark networks with community structure, and also on random networks. We have further applied the algorithm to different social, information, technological and biological networks, where it indeed reveals meaningful (composites of) link-density and link-pattern communities. The results thus seem to imply that, similarly as link-density counterparts, link-pattern communities appear ubiquitous in nature and design.

Community Detection Through Optimal Density Contrast of Adjacency Matrix

Detecting communities in real world networks is an important problem for data analysis in science and engineering. By clustering nodes intelligently, a recursive algorithm is designed to detect community. Since the relabeling of nodes does not alter the topology of the network, the problem of community detection corresponds to the finding of a good labeling of nodes so that the adjacency matrix form blocks. By putting a fictitious interaction between nodes, the relabeling problem becomes one of energy minimization, where the total energy of the network is defined by putting interaction between the labels of nodes so that clustering nodes that are in the same community will decrease the total energy. A greedy method is used for the computation of minimum energy. The method shows efficient detection of community in artificial as well as real world network. The result is illustrated in a tree showing hierarchical structure of communities on the basis of sub-matrix density. Applications of the method to weighted and directed networks are discussed.

Expanding network communities from representative examples

We present an approach to leverage a small subset of a coherent community within a social network into a much larger, more representative sample. Our problem becomes identifying a small conductance subgraph containing many (but not necessarily all) members of the given seed set. Starting with an initial seed set representing a sample of a community, we seek to discover as much of the full community as possible. We present a general method for network community expansion, demonstrating that our methods work well in expanding communities in real world networks starting from small given seed groups (20 to 400 members). Our approach is marked by incremental expansion from the seeds with retrospective analysis to determine the ultimate boundaries of our community. We demonstrate how to increase the robustness of the general approach through bootstrapping multiple random partitions of the input set into seed and evaluation groups. We go beyond statistical comparisons against gold standards to careful subjective evaluations of our expanded communities. This process explains the causes of most disagreement between our expanded communities and our gold-standards—arguing that our expansion methods provide more reliable communities than can be extracted from reference sources/gazetteers such as Wikipedia.