Efficient Distributed Approaches to Core Maintenance on Large Dynamic Graphs

Abstract

As a fundamental problem in graph analysis, core decomposition aims to compute the core numbers of vertices in a given graph. It is a powerful tool for mining important graph structures. For dynamic graphs with real-time updates of vertices/edges, core maintenance has been utilized to update the core numbers of vertices. The previous approaches to core maintenance face challenges in terms of storage and efficiency. In this article, we investigate distributed approaches to core maintenance on a pregel-like system, which is a famous graph computing system. We first design a core decomposition algorithm to obtain core numbers of vertices in a given graph. Based on it, a distributed batch-stream combined algorithm (DBCA) is devised to efficiently maintain the core numbers when vertex/edge updates happen. In particular, we introduce a new task assignment strategy to DBCA based on diversity of the edge-cores of updated edges. To ensure that DBCA can accurately process core maintenance, we develop a message interaction protocol to resolve the problem of crosstalk among different tasks. Comprehensive experiments have been conducted on real/synthetic graphs, more specifically, in two typical distributed environments built on Supercomputing Center and Alibaba Cloud. The experiment results demonstrate that our proposed algorithms are efficient and scalable.