RDMA-driven MongoDB: An approach of RDMA enhanced NoSQL paradigm for large-Scale data processing

Abstract

With the rapid development of big data and data center networks, NoSQL database has won great popularity for its excellent performance in accelerating the performance of many online and offline big data applications, such as HBase, Cassandra and MongoDB. However, due to massive and frequent Create/Update/Retrieval/Delete (CURD) operations, the traditional TCP/IP protocol stack has difficulty to provide the required request rates and response latency for the large-scale NoSQL system. For example, large-scale data migration or synchronization among multiple clusters in a data center results in competition for network bandwidth with high delay. To mitigate such transmission bottleneck, we propose an approach of RDMA-driven document NoSQL Paradigm’ RDMA_Mongo, based on MongoDB. The performance of CURD operations is enhanced by one-sided Remote Direct Memory Access (RDMA) primitives (such as RDMA Read/Write) without involving the TCP/IP stack or CPU. Evaluation under RDMA-enabled network demonstrates that RDMA_Mongo significantly improves the CURD performance, compared with plain MongoDB. The results show that the average insert throughput increases by approximately 30%, the average delete throughput by over 30%, the update by up to 17% and the query throughput by 15% when facing large-scale data requests.