Fast Stochastic Block Partitioning using a Single Commodity Machine

Abstract

Network (graph) is a powerful way of representing the relationship among different kinds of entities in a socio-technical system. Network partitioning is a problem that has its applications in social networking, traffic and communication networks, biological networks, etc. Exact partitioning of a network is an NP-hard problem. There exist different state-of-the-art relaxation techniques for approximating network partitioning with varying accuracy. However, with the exponential growth of information technology and social media, the sizes of real-world networks now have reached millions to billions of vertices and edges. Processing such massive networks require fast and efficient algorithms. A good domain-specific heuristic can make a big change in the execution time of an algorithm while maintaining accuracy in effective sense-making from network data. Stochastic Block Partitioning (SBP) is one of the state-of-the-art relaxation techniques that combines a stochastic model and an information-theoretic approach for network partitioning. An SBP algorithm of sub-quadratic run time complexity is given as the baseline algorithm for MIT GraphChallenge competition, which uses OpenMP based parallelism for scaling. In this work, we improve the performance of the baseline algorithm to several folds for a single (multi-core) commodity machine. We incorporate a refinement of the greedy agglomerative heuristic and modify the OpenMP based parallelism with more programmer-level control to gain up to 10–fold speedup over the baseline algorithm.