A Distributed Graph Partitioning Algorithm for Processing Large Graphs

Abstract

To address the challenge of processing large-scale graphs, researchers have paid much attention to distributed approaches. The quality of graph partitioning plays a key role in the performance of distributed algorithms, in respect of workload balance and communication cost. However, few of existing graph partitioning algorithms are capable of partitioning large graphs on distributed memory systems. In this paper, we propose a distributed balanced graph partitioning algorithm that is suitable for general distributed graph computation frameworks, called BS (Bulk Swap), which is based on a scatter-gather local search scheme and the simulated annealing technique. BS takes the advantage of the BSP graph computation model which can process bulk data efficiently. Experimental analysis shows BS can produce good partitions with high efficiency on both real-world and synthetic graphs.