Arbor: Efficient Large-Scale Graph Data Computing Model

Abstract

Graph data is the default data organization mechanism used in large-scale Social Network Service (SNS) applications. Traditional graph data computing models are used to dig out useful hidden information inside the data. However, the ever growing data volume is adding more and more pressures. To retrieve and discover the information, the system has to introduce a larger number of data iterations. This makes the data analysis operations becoming slower. To speed up these operations on large-scale graph data, recent research works focus on developing efficient parallel iteration processing strategies. However, the synchronization requirements between successive iterations can severely jeopardize the effectiveness of parallel operations. In this paper, we propose a novel large-scale graph data processing model, Arbor, to address these issues. Arbor substitutes time-constrained synchronization operations with nontime-constrained control message transmissions to increase the degree of parallelism. Furthermore, it develops a new graph data organization format, which can not only save storage space, but also accelerate graph data processing operations. We compare Arbor with other graph processing models using a large-scale experimental graph data, and the results show that it outperforms the state-of-the-art systems.