Hierarchical Community Structures Research Articles

Understand your shady neighborhood: An approach for detecting and investigating hacker communities

Cyber threat intelligence (CTI) researchers strive to uncover collaborations and emerging techniques within hacker networks. This study proposes an empirical approach to detect communities within hacker forums for CTI purposes. Eighteen algorithms are systematically evaluated, including state-of-the-art and benchmark methods for identifying overlapping and disjoint groups. Using discussions from five prominent English hacker forums, a comparative analysis examines the influence of the algorithms’ theoretical foundations on community detection. Since ground truths are unattainable for such networks, the study utilizes a multi-metric strategy, incorporating modularity, coverage, performance, and a newly introduced quality measure, Triplet Hub Potential, which quantifies the presence of influential hubs. The findings reveal that while modularity optimization algorithms such as Leiden and Louvain deliver consistent results, neighbor-based expanding techniques tend to provide superior performance. In particular, the Expansion algorithm stood out by uncovering granular hierarchical community structures. The ability to investigate these intimacies is helpful for CTI researchers. Ultimately, we suggest an approach to investigate hacker forums using community detection methods and encourage the future development of algorithms tailored to expose nuances within hacker networks.

Improved brain community structure detection by two-step weighted modularity maximization

The human brain can be regarded as a complex network with interacting connections between brain regions. Complex brain network analyses have been widely applied to functional magnetic resonance imaging (fMRI) data and have revealed the existence of community structures in brain networks. The identification of communities may provide insight into understanding the topological functions of brain networks. Among various community detection methods, the modularity maximization (MM) method has the advantages of model conciseness, fast convergence and strong adaptability to large-scale networks and has been extended from single-layer networks to multilayer networks to investigate the community structure changes of brain networks. However, the problems of MM, suffering from instability and failing to detect hierarchical community structure in networks, largely limit the application of MM in the community detection of brain networks. In this study, we proposed the weighted modularity maximization (WMM) method by using the weight matrix to weight the adjacency matrix and improve the performance of MM. Moreover, we further proposed the two-step WMM method to detect the hierarchical community structures of networks by utilizing node attributes. The results of the synthetic networks without node attributes demonstrated that WMM showed better partition accuracy than both MM and robust MM and better stability than MM. The two-step WMM method showed better accuracy of community partitioning than WMM for synthetic networks with node attributes. Moreover, the results of resting state fMRI (rs-fMRI) data showed that two-step WMM had the advantage of detecting the hierarchical communities over WMM and was more insensitive to the density of the rs-fMRI networks than WMM.

Hierarchical community structure in networks.

Modular and hierarchical community structures are pervasive in real-world complex systems. A great deal of effort has gone into trying to detect and study these structures. Important theoretical advances in the detection of modular have included identifying fundamental limits of detectability by formally defining community structure using probabilistic generative models. Detecting hierarchical community structure introduces additional challenges alongside those inherited from community detection. Here we present a theoretical study on hierarchical community structure in networks, which has thus far not received the same rigorous attention. We address the following questions. (1) How should we define a hierarchy of communities? (2) How do we determine if there is sufficient evidence of a hierarchical structure in a network? (3) How can we detect hierarchical structure efficiently? We approach these questions by introducing a definition of hierarchy based on the concept of stochastic externally equitable partitions and their relation to probabilistic models, such as the popular stochastic block model. We enumerate the challenges involved in detecting hierarchies and, by studying the spectral properties of hierarchical structure, present an efficient and principled method for detecting them.

A non-binary hierarchical tree overlapping community detection based on multi-dimensional similarity

Overlapping communities exist in real networks, where the communities represent hierarchical community structures, such as schools and government departments. A non-binary tree allows a vertex to belong to multiple communities to obtain a more realistic overlapping community structure. It is challenging to select appropriate leaf vertices and construct a hierarchical tree that considers a large amount of structural information. In this paper, we propose a non-binary hierarchical tree overlapping community detection based on multi-dimensional similarity. The multi-dimensional similarity fully considers the local structure characteristics between vertices to calculate the similarity between vertices. First, we construct a similarity matrix based on the first and second-order neighbor vertices and select a leaf vertex. Second, we expand the leaf vertex based on the principle of maximum community density and construct a non-binary tree. Finally, we choose the layer with the largest overlapping modularity as the result of community division. Experiments on real-world networks demonstrate that our proposed algorithm is superior to other representative algorithms in terms of the quality of overlapping community detection.

How to make ‘new heritage’ in a fragile institutional context: the case of Tzari Mali Grad in Bulgaria as ‘community of patrons’

ABSTRACT The paper discusses how ‘new heritage’ is created in a fragile institutional context by analysing the example of the historical complex Tzari Mali Grad (TMG) in Bulgaria. The analysis builds upon Regina Bendix’s notion of ‘communities of patrons’ and further develops its analytical potential. The article focuses on the hierarchical community structures; on the transformation of patrons into leaders; on the dynamics of patrons’ engagement with heritage which we call ‘spiral of involvement’; and on the specifics of the fragile post-socialist context, which allows individual actors to substitute public institutions by restricting their power in heritage decision making. To explore the case, we triangulate qualitative and quantitative data. We conclude that this is an instrumental approach towards heritage, focused on the needs of the present rather than on the heritage’s authenticity. This contextual specificity allows individual patrons to become heritage leaders and to cultivate communities of patrons.

COVID-19 in Fiji

With the implementation of strict quarantine and adjunct health measures, Fiji was largely spared from the COVID-19 pandemic in 2020. In April 2021, the arrival of the delta variant of SARS-CoV-2 resulted in a major outbreak that is only now under control. The country enacted swift countermeasures to reduce population movement, but the delta variant gained a foothold in the peri-urban areas of the largest city, Suva, and from there spread to other major urban centres of the largest island, Viti Levu. A very effective vaccination campaign has since been implemented resulting in a rate of two-dose vaccination, above 70%, that exceeds many higher income countries and, as of late October 2021, the numbers of COVID-19 cases were in decline. Fijian life, especially in rural settings, is based on hierarchical but supportive community structures and the responsibilities that flow from community membership, for example, attending large funerals, mitigated against the organised and systematic control response instituted by the Fijian Ministry of Health and Medical Services, including vaccination, and the other Government agencies. A positive consequence of the outbreak has been the development of new capabilities, and the distribution of molecular diagnostic technologies that can be exploited to help control other infectious diseases.

Multiscale community detection in Cytoscape.

Detection of community structure has become a fundamental step in the analysis of biological networks with application to protein function annotation, disease gene prediction, and drug discovery. This recent impact creates a need to make these techniques and their accompanying visualization schemes available to a broad range of biologists. Here we present a service-oriented, end-to-end software framework, CDAPS (Community Detection APplication and Service), that integrates the identification, annotation, visualization, and interrogation of multiscale network communities, accessible within the popular Cytoscape network analysis platform. With novel design principles, CDAPS addresses unmet new challenges, such as identifying hierarchical community structures, comparison of outputs generated from diverse network resources, and easy deployment of new algorithms, to facilitate community-sourced science. We demonstrate that the CDAPS framework can be applied to high-throughput protein-protein interaction networks to gain novel insights, such as the identification of putative new members of known protein complexes.

Bounds on maximum concurrent flow in random bipartite graphs

Algorithms that exploit the duality between maximum concurrent flow and sparse cuts in a graph can effectively discover hierarchical community structures in social networks. We analyze the maximum concurrent flow on random bipartite graphs and compare the results to two bounds based on graph structure.. The minimum degree bound is based on a sparse cut that removes a node of minimum degree. The other is the “gridlock” bound, where all edges are saturated with flow. We show that the gridlock bound is more constraining when the minimum degree of the graph is sufficiently large, and provide a way to calculate this bound on graphs of diameter three, which occurs with high probability on larger random bipartite graphs.

Integrating Geovisual Analytics with Machine Learning for Human Mobility Pattern Discovery

Understanding human movement patterns is of fundamental importance in transportation planning and management. We propose to examine complex public transit travel patterns over a large-scale transit network, which is challenging since it involves thousands of transit passengers and massive data from heterogeneous sources. Additionally, efficient representation and visualization of discovered travel patterns is difficult given a large number of transit trips. To address these challenges, this study leverages advanced machine learning methods to identify time-varying mobility patterns based on smart card data and other urban data. The proposed approach delivers a comprehensive solution to pre-process, analyze, and visualize complex public transit travel patterns. This approach first fuses smart card data with other urban data to reconstruct original transit trips. We use two machine learning methods, including a clustering algorithm to extract transit corridors to represent primary mobility connections between different regions and a graph-embedding algorithm to discover hierarchical mobility community structures. We also devise compact and effective multi-scale visualization forms to represent the discovered travel behavior dynamics. An interactive web-based mapping prototype is developed to integrate advanced machine learning methods with specific visualizations to characterize transit travel behavior patterns and to enable visual exploration of transit mobility patterns at different scales and resolutions over space and time. The proposed approach is evaluated using multi-source big transit data (e.g., smart card data, transit network data, and bus trajectory data) collected in Shenzhen City, China. Evaluation of our prototype demonstrates that the proposed visual analytics approach offers a scalable and effective solution for discovering meaningful travel patterns across large metropolitan areas.

Hierarchical benchmark graphs for testing community detection algorithms.

Hierarchical organization is an important, prevalent characteristic of complex systems; to understand their organization, the study of the underlying (generally complex) networks that describe the interactions between their constituents plays a central role. Numerous previous works have shown that many real-world networks in social, biologic, and technical systems present hierarchical organization, often in the form of a hierarchy of community structures. Many artificial benchmark graphs have been proposed to test different community detection methods, but no benchmark has been developed to thoroughly test the detection of hierarchical community structures. In this study, we fill this vacancy by extending the Lancichinetti-Fortunato-Radicchi (LFR) ensemble of benchmark graphs, adopting the rule of constructing hierarchical networks proposed by Ravasz and Barabási. We employ this benchmark to test three of the most popular community detection algorithms and quantify their accuracy using the traditional mutual information and the recently introduced hierarchical mutual information. The results indicate that the Ravasz-Barabási-Lancichinetti-Fortunato-Radicchi (RB-LFR) benchmark generates a complex hierarchical structure constituting a challenging benchmark for the considered community detection methods.

Scalably revealing the dynamics of soft community structure in complex networks

Revealing the dynamics of community structure is of great concern for scientists from many fields. Specifically, how to quantify the dynamic details of soft community structure is a very interesting topic. In this paper, the authors propose a novel framework to study the scalable dynamic behavior of the soft community structure. First, the authors model the Potts dynamics to detect community structure using a “soft” Markov process. Then the soft stability of in a multiscale view is proposed to naturally uncover the local uniform behavior of spin values across multiple hierarchical levels. Finally, a new partition index is developed to detect fuzzy communities based on the stability and the dynamical information. Experiments on the both synthetically generated and real-world networks verify that the framework can be used to uncover hierarchical community structures effectively and efficiently.

Hierarchical mutual information for the comparison of hierarchical community structures in complex networks.

The quest for a quantitative characterization of community and modular structure of complex networks produced a variety of methods and algorithms to classify different networks. However, it is not clear if such methods provide consistent, robust, and meaningful results when considering hierarchies as a whole. Part of the problem is the lack of a similarity measure for the comparison of hierarchical community structures. In this work we give a contribution by introducing the hierarchical mutual information, which is a generalization of the traditional mutual information and makes it possible to compare hierarchical partitions and hierarchical community structures. The normalized version of the hierarchical mutual information should behave analogously to the traditional normalized mutual information. Here the correct behavior of the hierarchical mutual information is corroborated on an extensive battery of numerical experiments. The experiments are performed on artificial hierarchies and on the hierarchical community structure of artificial and empirical networks. Furthermore, the experiments illustrate some of the practical applications of the hierarchical mutual information, namely the comparison of different community detection methods and the study of the consistency, robustness, and temporal evolution of the hierarchical modular structure of networks.

An improved multi-objective evolutionary algorithm for simultaneously detecting separated and overlapping communities

In our previous work, a multi-objective evolutionary algorithm (MEA_CDPs) was proposed for detecting separated and overlapping communities simultaneously. However, MEA_CDPs has a couple of defects, like individuals cannot be transformed to community structure by the decoder when the quality of community structure is lower certain thresholds, many vertices with weak overlapping nature are identified as overlapping nodes, and the objective functions can not control the ratio of separated nodes to overlapping nodes. Therefore, in this paper, to overcome these defects, we improve MEA_CDPs by designing more efficient objective functions. We also extend MEA_CDPs' capability in detecting hierarchical community structures. The improved algorithm is named as iMEA_CDPs. In the experiments, a set of computer-generated networks are first used to test the effect of parameters in iMEA_CDPs, and then four real-world networks are used to validate the performance of iMEA_CDPs. The experimental results show that iMEA_CDPs outperforms MEA_CDPs. Moreover, compared with MEA_CDPs, iMEA_CDPs can detect various kinds of overlapping and hierarchical community structures.

An integer programming approach and visual analysis for detecting hierarchical community structures in social networks

Detecting community structures in social networks is a very important task in social network analysis as these community structures explain relationships among individuals and can be used to predict social behavior. The relationship among subcommunities in each community can further be identified as hierarchical community structures, in which each super node at each hierarchical level represents a nested structure of communities or nodes. Most previous studies attempting to detect hierarchical community structures focused on new metaheuristics that are computationally efficient but do not guarantee the optimal community partition. As a result, this work applies a novel integer programming (IP) approach to detect hierarchical community structures in social networks. This approach has flexible community capacity limits, does not limit the community numbers at different levels, and maximizes a quality measure for hierarchical community partition. The proposed IP approach can use existing software solvers to detect hierarchical community structures without implementing an algorithm. Visual analysis of experimental results shows that the proposed model with different settings for level numbers can analyze reasonable and sophisticated hierarchical community structures, such that the relationships between communities at different levels can be elucidated clearly.

Novel Dynamic Evolution Model Based on Improved Cellular Automata in Hierarchical Complex Networks

Modeling of network structures has been a popular issue in the study of complex networks. The use of complex network theory in modeling complex networks has gradually become a trend. In this study, the current research status of the modeling of complex networks was synthesized based on the BA scale-free network model. The advantages and disadvantages of most mainstream models were integrated, and a complex network model was established from the view of hierarchical community structures. Furthermore, a dynamic evolution network model was built by establishing a new evolution rule of cellular automata. Simulation experiments were conducted to validate the model. Results revealed that the network exhibits a power-law behavior and has a clear, controlled community structure. The model can suitably describe the characteristics of complex network topologies.

Post-processing hierarchical community structures: Quality improvements and multi-scale view

Dense sub-graphs of sparse graphs ( communities), which appear in most real-world complex networks, play an important role in many contexts. Most existing community detection algorithms produce a hierarchical structure of communities and seek a partition into communities that optimizes a given quality function. We propose new methods to improve the results of any of these algorithms. First we show how to optimize a general class of additive quality functions (containing the modularity, the performance, and a new similarity based quality function which we propose) over a larger set of partitions than the classical methods. Moreover, we define new multi-scale quality functions which make it possible to detect different scales at which meaningful community structures appear, while classical approaches find only one partition.

Graphical models based hierarchical probabilistic community discovery in large-scale social networks

Real-world social networks, while disparate in nature, often comprise of a set of loose clusters (a.k.a. communities), in which members are better connected to each other than to the rest of the network. In addition, such communities are often hierarchical, reflecting the fact that some communities are composed of a few smaller, sub-communities. Discovering the complicated hierarchical community structure can gain us deeper understanding about the networks and the pertaining communities. This paper describes a hierarchical Bayesian model based scheme namely hierarchical social network-pachinko allocation model (HSN-PAM), for discovering probabilistic, hierarchical communities in social networks. This scheme is powered by a previously developed hierarchical Bayesian model. In this scheme, communities are classified into two categories: super-communities and regular-communities. Two different network encoding approaches are explored to evaluate this scheme on research collaborative networks, including CiteSeer. The experimental results demonstrate that HSN-PAM is effective for discovering hierarchical community structures in large-scale social networks.