Efficient Graph Analytics in Python for Large-Scale Data Science

Abstract

Graph analytics is important in data science research, where Python is nowadays the most popular language among data analysts. It facilitates many packages for graph analytics. However, those packages are either too specific or cannot work on graphs that cannot fit into the main memory. Moreover, it is hard to handle new graph algorithms or even customize existing ones according to the analyst’s need. In this paper, we develop a general graph C++ function based on a semiring algorithm including two math operators. The function can help solve many graph problems. It also works for graphs that cannot fit in the main memory. Our function is developed in C++, but it can be easily called in Python. Experimental comparison with state-of-art Python packages show that our C++ function has comparative performance for both small and large graphs.