Design and implementation of bandwidth-aware memory placement and migration policies for heterogeneous memory systems

Abstract

Heterogeneous memory systems that comprise memory nodes based on widely-different device technologies (e.g., DRAM and nonvolatile memory (NVM)) are emerging in various computing domains ranging from high-performance to embedded computing. Despite the extensive prior work on architectural and system software support for heterogeneous memory systems, relatively little work has been done to investigate the OS-level memory placement and migration policies that consider the bandwidth differences of heterogeneous memory nodes. To bridge this gap, this work investigates the design and implementation of memory placement and migration policies for bandwidth-intensive applications on heterogeneous memory systems. Specifically, we propose three bandwidth-aware memory placement policies (i.e., bandwidth-aware interleave, random, and local policies) and a bandwidth-aware memory migration policy and implement the proposed policies in the Linux kernel. Through our quantitative evaluation based on real system software and hardware stacks, we demonstrate that the bandwidth-aware memory placement and migration policies achieve significantly higher performance than the conventional bandwidth-oblivious policies across a wide range of the DRAM-to-NVM bandwidth ratios when executing bandwidth-intensive workloads.