Distributed Community Detection Research Articles

Simple-based Dynamic Decentralized Community Detection Algorithm in socially aware networks

This paper introduces a novel, Simple-based Dynamic Decentralized Community Detection Algorithm (S-DCDA) for Socially Aware Networks. This algorithm aims to address the resource-intensive nature, instabilities and inaccuracies of traditional distributed community detection algorithms. The dynamics of decentralization is evident in the threefold nature of the algorithm: (i) each node of the community is the core of the entire network or community for a certain period of time dependent on their need, (ii) nodes are not centralized around themselves, requiring the consent of the other node to join a community, and (iii) Communities start from a single node to form an initial scale community, the number of nodes and the relationship among them are constantly changing. The algorithm requires low processor performance and memory capacity size of each node, to a certain extent, effectively improve the accuracy and stability of community detection and maintenance. Experimental results demonstrate that in comparison to classical and classical-based improved community detection algorithms, S-DCDA yields superior detection results.

Federated Clique Percolation for Privacy-preserving Overlapping Community Detection

Community structure is a typical characteristic of complex networks. Finding communities in complex networks has many important applications, such as the advertisement and recommendation based on social networks and the discovery of new protein molecules in biological networks, which make it a hot topic in the field of complex network analysis. With the increasing concerns about the leakage of personal privacy, discovering communities spread across the local networks owned by multiple participants accurately while preserving each participant’s privacy has become an emerging challenge in distributed community detection. In this article, we propose a general federated graph learning model for privacy-preserving distributed graph learning and develop two federated clique percolation algorithms (CPAs) based on it to discover overlapping communities distributed across multiple participants’ local networks without disclosing any participant’s network privacy. Homomorphic encryption and hash operation are used in combination to protect the privacy of the vertices and edges of each local network. Furthermore, vertex attributes are involved in the calculation of clique similarity and clique percolation when dealing with attributed networks. The experimental results on real-world and artificial datasets demonstrate that the proposed algorithms achieve identical results to those of their stand-alone counterparts and more than 200% higher accuracy than the simple distributed CPAs without federating learning.

A distributed community detection algorithm for large scale networks under stochastic block models

Community detection for large scale networks is of great importance in modern data analysis. In this work, we develop a distributed spectral clustering algorithm to handle this task. Specifically, we distribute a certain number of pilot network nodes on the master server and the others on worker servers. A spectral clustering algorithm is first conducted on the master to select pseudo centers. Next, the indexes of the pseudo centers are broadcasted to workers to complete the distributed community detection task using an SVD (singular value decomposition) type algorithm. The proposed distributed algorithm has three advantages. First, the communication cost is low, since only the indexes of pseudo centers are communicated. Second, no further iterative algorithm is needed on workers while a “one-shot” computation suffices. Third, both the computational complexity and the storage requirements are much lower compared to using the whole adjacency matrix. We develop a Python package DCD (The Python package is provided in https://github.com/Ikerlz/dcd.) to implement the distributed algorithm on a Spark system and establish theoretical properties with respect to the estimation accuracy and mis-clustering rates under the stochastic block model. Experiments on a variety of synthetic and empirical datasets are carried out to further illustrate the advantages of the methodology.

PLDLS: A novel parallel label diffusion and label Selection-based community detection algorithm based on Spark in social networks

Parallel and distributed community detection in large-scale complex networks, such as social networks, is a challenging task. Parallel and distributed algorithm with high accuracy and low computational complexity is one of the essential issues in the community detection field. In this paper, we propose a novel fast, and accurate Spark-based parallel label diffusion and label selection-based (PLDLS) community detection algorithm with two-step of label diffusion of core nodes along with a new label selection (propagation) method. We have used multi-factor criteria for computing node's importance and adopted a new method for selecting core nodes. In the first phase, utilizing the fact that nodes forming triangles, tend to be in the same community, parallel label diffusion of core nodes is performed to diffuse labels up to two levels. In the second phase, through an iterative and parallel process, the most appropriate labels are assigned to the remaining nodes. PLDLS proposes an improved robust version of LPA by putting aside randomness parameter tuning. Furthermore, we utilize a fast and parallel merge phase to get even more dense and accurate communities. Conducted experiments on real-world and artificial networks, indicates the better accuracy and low execution time of PLDLS in comparison with other examined methods.

Energy efficient target set selection and buffer management for D2D mobile data offloading

Data offloading offers a significant solution to the problem of explosive rise in mobile data traffic. A naive approach would be to utilize the infrastructure (cellular tower, WiFi, femtocell) or other mobile devices to offload data. However, increasing the number of a cellular towers, WiFi, or femtocells is costlier deal for data delivery. Recently, device-to-device (D2D) paradigm of data communication has emerged out as one of the most promising solutions to deal with cost effective cellular traffic offloading. D2D communication provides a direct communication link between closely located mobile users. Another significant feature of D2D is its content centric nature, which makes it useful in data offloading. In this paper, we have addressed the issue of data offloading in mobile devices and proposed a hybrid model of D2D communication with ad-hoc nature. The paper also considers the issues like memory constraints of the devices, pruning of replicated messages and energy efficiency to increase the lifetime of the battery. Considering all the constraints and trade off, we have modeled our problem into optimal target selection problem and distributed community detection problem, both of which are NP-hard. We propose a clustering algorithm to optimize the cooperative mobile nodes. The proposed algorithm uses the betweenness centrality and k-means for optimizing target set section. Our proposed algorithm requires less time in terms of computational complexity with limited space. We compare it with the community-based approach in terms of load transferred for varying target set sizes for validation. The simulation results of the suggested algorithm may reduce the energy requirements up to 16.7% and is able to accommodate 80% more traffic as compared to the community-based algorithm.

A Distributed Bagging Ensemble Methodology for Community Prediction in Social Networks

Presently, due to the extended availability of gigantic information networks and the beneficial application of graph analysis in various scientific fields, the necessity for efficient and highly scalable community detection algorithms has never been more essential. Despite the significant amount of published research, the existing methods—such as the Girvan–Newman, random-walk edge betweenness, vertex centrality, InfoMap, spectral clustering, etc.—have virtually been proven incapable of handling real-life social graphs due to the intrinsic computational restrictions that lead to mediocre performance and poor scalability. The purpose of this article is to introduce a novel, distributed community detection methodology which in accordance with the community prediction concept, leverages the reduced complexity and the decreased variance of the bagging ensemble methods, to unveil the subjacent community hierarchy. The proposed approach has been thoroughly tested, meticulously compared against different classic community detection algorithms, and practically proven exceptionally scalable, eminently efficient, and promisingly accurate in unfolding the underlying community structure.

A Distributed Hybrid Community Detection Methodology for Social Networks

Nowadays, the amount of digitally available information has tremendously grown, with real-world data graphs outreaching the millions or even billions of vertices. Hence, community detection, where groups of vertices are formed according to a well-defined similarity measure, has never been more essential affecting a vast range of scientific fields such as bio-informatics, sociology, discrete mathematics, nonlinear dynamics, digital marketing, and computer science. Even if an impressive amount of research has yet been published to tackle this NP-hard class problem, the existing methods and algorithms have virtually been proven inefficient and severely unscalable. In this regard, the purpose of this manuscript is to combine the network topology properties expressed by the loose similarity and the local edge betweenness, which is a currently proposed Girvan–Newman’s edge betweenness measure alternative, along with the intrinsic user content information, in order to introduce a novel and highly distributed hybrid community detection methodology. The proposed approach has been thoroughly tested on various real social graphs, roundly compared to other classic divisive community detection algorithms that serve as baselines and practically proven exceptionally scalable, highly efficient, and adequately accurate in terms of revealing the subjacent network hierarchy.

InfoFlow: A Distributed Algorithm to Detect Communities According to the Map Equation

Formidably sized networks are becoming more and more common, including in social sciences, biology, neuroscience, and the technology space. Many network sizes are expected to challenge the storage capability of a single physical computer. Here, we take two approaches to handle big networks: first, we look at how big data technology and distributed computing is an exciting approach to big data storage and processing. Second, most networks can be partitioned or labeled into communities, clusters, or modules, thus capturing the crux of the network while reducing detailed information, through the class of algorithms known as community detection. In this paper, we combine these two approaches, developing a distributed community detection algorithm to handle big networks. In particular, the map equation provides a way to identify network communities according to the information flow between nodes, where InfoMap is a greedy algorithm that uses the map equation. We develop discrete mathematics to adapt InfoMap into a distributed computing framework and then further develop the mathematics for a greedy algorithm, InfoFlow, which has logarithmic time complexity, compared to the linear complexity in InfoMap. Benchmark results of graphs up to millions of nodes and hundreds of millions of edges confirm the time complexity improvement, while maintaining community accuracy. Thus, we develop a map equation based community detection algorithm suitable for big network data processing.

Distributed Community Detection via Metastability of the 2-Choices Dynamics

We investigate the behavior of a simple majority dynamics on networks of agents whose interaction topology exhibits a community structure. By leveraging recent advancements in the analysis of dynamics, we prove that, when the states of the nodes are randomly initialized, the system rapidly and stably converges to a configuration in which the communities maintain internal consensus on different states. This is the first analytical result on the behavior of dynamics for nonconsensus problems on non-complete topologies, based on the first symmetry-breaking analysis in such setting.Our result has several implications in different contexts in which dynamics are adopted for computational and biological modeling purposes. In the context of Label Propagation Algorithms, a class of widely used heuristics for community detection, it represents the first theoretical result on the behavior of a distributed label propagation algorithm with quasi-linear message complexity. In the context of evolutionary biology, dynamics such as the Moran process have been used to model the spread of mutations in genetic populations (Lieberman, Hauert, and Nowak 2005); our result shows that, when the probability of adoption of a given mutation by a node of the evolutionary graph depends super-linearly on the frequency of the mutation in the neighborhood of the node and the underlying evolutionary graph exhibits a community structure, there is a non-negligible probability for species differentiation to occur.

Distributed Community Detection based on Apache Spark using Multi Label Propagation for Digital Social Networks

Organization, Government and Individual (OGI) have popularized the use of Digital Social Networks (DSN) that reduces the processing time of social-aware tasks. To accomplish a community-based communication, each social-aware task should identify its community group. The identified group uses a task to avail all the DSN benefits to their customers / citizens. As a result, the community-based detection algorithm has played a significant role in literature. However, the existing algorithms have had several challenging issues, such as performance and scalability. Thus, a distributed community detection algorithm is presented using Apache Spark’s Resilient Distributed Data Set (RDD) framework based on the Scala programming language. The Apache Spark framework provides an ideal solution that offers ease of coding, performance, interactive mode and disk Input-Output bottlenecks in Hadoop /Map Reduce. Besides, it presents a platform of distributed community detection that reduces the computational computation by applying transformations, aggregations and joins. The experimental results show that the proposed framework achieves high accuracy for both real-world and synthetic networks.

Enhancing the reliability on data delivery and energy efficiency by combining swarm intelligence and community detection in large-scale WSNs

In Wireless Sensor Networks (WSNs) to address the duality between the cost-effective energy efficiency and the reliable data delivery is a relevant issue. This paper presents a novel bio-inspired routing protocol, named CB-RACO, that combines the Ant Colony Optimization (ACO) meta-heuristic with the computationally cheap and distributed community detection technique Label Propagation (LP). CB-RACO creates communities in the WSNs and meets the balance of energy consumption by routing data inside-communities through swarm intelligence. As a consequence, CB-RACO demands low memory and overhead in construction and maintenance of routing paths. Additionally, CB-RACO achieves high data delivery reliability through a data retransmission strategy based on acknowledgments between communities. We simulated CB-RACO in large-scale scenarios according to the goodput, delivery delay and energy consumption metrics. The results have shown that the proposed approach may provide significant improvement in comparison to ant-based strategies that do not rely on community structures.

An evidential influence-based label propagation algorithm for distributed community detection in social networks

Community detection in social networks is a computationally challenging task that has attracted many researchers in the last decade. Most of approaches in the literature focus only on modeling structural properties, ignoring the social aspect in the relations between users. Additionally, they detect the communities, one after another, in a serial manner. However, the size of actual real-world social networks grows exponentially which makes such approaches inefficient. For this, several models tend to parallelize the community detection task. Unfortunately, social networks data often exhibits a high degree of dependency which renders the parallelization task more difficult. To overcome this difficulty, amongst the proposed distributed community detection methods, the label propagation algorithm (LPA) emerges as an effective detection method due to its time efficiency. Despite this advantage in computational time, the performance of LPA is affected by randomness in the algorithm. Indeed, LPA suffers from poor stability and occurrence of monster community. This paper introduces a new LPA algorithm for distributed community detection based on evidence theory which has shown a high efficiency in handling information. In our model, we will use the belief functions in the update of labels as well as in their propagation in order to improve the quality of the solutions computed by the standard LPA. The mass assignments and the plausibility, in our model, are computed based on the social influence for detecting the domain label of each node. Experimentation of our model on real-world and artificial LFR networks shows its efficiency compared to the state of the art algorithms.

Congestion‐aware message forwarding in delay tolerant networks: a community perspective

SummaryIn delay tolerant networks, most of the existing message forwarding algorithms prefer to deliver messages to the nodes with a higher popularity or centrality in the hope of maximizing the delivery ratio or minimizing the end‐to‐end delay. This forwarding scheme is prone to cause unfair load distribution and further lead to network congestion, overlooked in the previous work. In this paper, we discuss the network congestion from a community perspective and take it into account the design of message forwarding algorithms. We first put forward a novel distributed community detection approach, which could track the evolution of communities. Based on the identified communities, we develop a congestion avoidance mechanism to divert the load away from the congested areas to the alternative custodians and further present a congestion‐aware message forwarding algorithm where messages can avoid being transmitted to the congested nodes. We finally evaluate the effectiveness of distributed community detection and congestion‐aware message forwarding through the extensive real‐trace driven simulations. Copyright © 2015 John Wiley & Sons, Ltd.

Distributed community detection in dynamic graphs

Inspired by the increasing interest in self-organizing social opportunistic networks, we investigate the problem of distributed detection of unknown communities in dynamic random graphs. As a formal framework, we consider the dynamic version of the well-studied Planted Bisection Model dyn-G(n,p,q) where the node set [n] of the network is partitioned into two unknown communities and, at every time step, each possible edge (u,v) is active with probability p if both nodes belong to the same community, while it is active with probability q (with q≪p) otherwise. We also consider a time-Markovian generalization of this model.We propose a distributed protocol based on the popular Label-Propagation approach and prove that, when the ratio p/q is larger than nb (for an arbitrarily small constant b>0), the protocol finds the right “planted” partition in O(log⁡n) time even when the snapshots of the dynamic graph are sparse and disconnected (i.e., when p=Θ(1/n)).

Distributed Community Detection Optimization Algorithm for Complex Networks

To investigate coupled oscillator model and community detection algorithm, an improvement method about the phase synchronization oscillator model and an optimized method (community detection via model modification) was proposed. By using the Kuramoto oscillator model as a basis, after joining the negative coupling strength, the nodes can be divided into several different synchronized clusters. In the synchronization process, the internal nodes in the same matrix are connected closely. In this method Kuramoto coupled oscillator model are expanded. The network can realize synchronization or be partitioned into several clusters depending on its structure. If all the nodes of the network are densely connected as a whole entity, synchronization will appear. If the network consists of several groups within which the connections are dense and between which the connections are sparser, the network will be partitioned into several clusters by their phases. The networks are divided into several communities because of this clustering phenomenon. The experiments show that the method is very promising. The simulation results in a variety of community structure that the proposed algorithm is an accurate, efficient and practical method for detecting community structure in networks

Improving Smart Conference Participation Through Socially Aware Recommendation

This research addresses recommending presentation sessions at smart conferences to participants. We propose a venue recommendation algorithm, Socially-Aware Recommendation of Venues and Environments (SARVE). SARVE computes correlation and social characteristic information of conference participants. In order to model a recommendation process using distributed community detection, SARVE further integrates the current context of both the smart conference community and participants. SARVE recommends presentation sessions that may be of high interest to each participant. We evaluate SARVE using a real world dataset. In our experiments, we compare SARVE to two related state-of-the-art methods, namely: Context-Aware Mobile Recommendation Services (CAMRS) and Conference Navigator (Recommender) Model. Our experimental results show that in terms of the utilized evaluation metrics: precision, recall, and f-measure, SARVE achieves more reliable and favorable social (relations and context) recommendation results.