Partitioning Graph Databases by Using Access Patterns

Abstract

With the emergence of large scale social networks such as Twitter, Facebook, Linkedin and Google+ the growing trends of big data is becoming much clearer. In addition to massive storage requirements for this highly connected big data, efficient mechanisms for processing this data are also needed. The inadequacy of traditional solutions such as relational database management systems for processing highly connected data causes the people to head towards graph databases. Graph databases are able to handle billions of nodes and relationships on a single machine but the high growing rate of social data are already pushing their limits. In this work, we consider partitioning of graph databases in order to increase throughput of a graph database system. For this purpose we design and implement a framework that both partitions a graph database and provides a fully functional distributed graph database system. We concentrate on access pattern based partitioning. In our experiments access pattern based partitioning outperforms unbiased partitioning that only depends on static structure of the graph. We evaluate our results on real world datasets of the Erdos Web-Graph Project and the Pokec social network.