Efficient Remote Memory Paging for Disaggregated Memory Systems

Abstract

Memory disaggregation has attracted increasing attention in recent years because it is a cost-efficient approach to scale memory capacity for applications in a data center. However, the latency of remote memory access is a major concern in disaggregated memory systems. This paper presents VANDI, a virtual memory paging mechanism that allows applications to use remote memory pools transparently. VANDI enables effective data caching and prefetching mechanisms to address the problem of high access latency in disaggregated memory systems. VANDI exploits a low-complexity cache replacement strategy to optimize the asynchronous staging queue so that the remote write latency can be significantly reduced. VANDI can also prefetch data in multi-granularity from a remote memory pool in an adaptive manner, and thus further improves the hit rate of the local cache to reduce the read latency of remote memory. Our extensive experiments using micro-benchmarks show that VANDI can improve the performance of typical remote paging system–Infiniswap by up to 15 $$\times $$ –102 $$\times $$ . VANDI can also improve the performance of state-of-the-art disaggregated memory system–Valet by 1.2 $$\times $$ –2.7 $$\times $$ . For typical machine learning workloads, VANDI can achieve 20% to 80% performance improvement compared with the state-of-the-art Valet.