A Balanced Vertex Cut Partition Method in Distributed Graph Computing

Abstract

Graph computing plays an important role in mining data at large scale. Partition is the primary step when we process large graph in a distributed system. A good partition has less communication and memory cost as well as more balanced load to take advantage of the whole system. Traditional edge cut methods introduce large communication cost for realistic power law graphs. Current vertex cut methods perform poorly with little consideration on load balance especially for online streaming vertex cut partition. In this paper, we formulate the total cost (partition cost, communication cost and computing cost) of graph computing especially that in iterating algorithms and analyze the cost of current partitioning methods. In addition, we explore a novel vertex cut method to ensure lower total cost. It has more balanced load with fewer communications. Experiments show that our method outperforms in state of the art graph computing frameworks at an average of 10 percent.