Impact of RDMA communication on the performance of distributed BFS algorithm

Abstract

Remote Direct Memory Access(RDMA) technology offers a promising solution to the memory wall problem in high speed communication. In this paper, we analyze the performance of RDMA and alternative communication models for the utilization of distributed breadth first search (BFS) traversal on large full tree graphs. First, we present the scalability limitations of the shared-memory multi-threading solution along with a sequential benchmark. We then compare the performance of socket-based distributed and RDMA-based distributed communication models on nine different graph structures. The results illustrate that distributed solutions are more viable than shared memory models for the given test cases. Among the distributed models, RDMA achieves 28% lower latency than the socket communication for graphs having 22.3M vertices on eight peer nodes. RDMA also achieves 98% lower latency than the sequential algorithm.