A unified hybrid memory system for scalable deep learning and big data applications

Abstract

Emerging non-volatile memory (NVM) technologies are of dynamic random access memory (DRAM)-like, high capacity, and low cost, at the expense of slower bandwidth and higher read/write latency compared to DRAM. Typically, NVM finds its primary application in serving as an extension of conventional DRAM to create hybrid memory systems tailored to non-uniform memory access (NUMA) architectures. This strategic integration offers the potential for high performance, enhanced capacity efficiency, and a favorable balance of cost considerations. Traditional NUMA memory management policies distribute data uniformly across both DRAM and NVM, overlooking the inherent performance gap between these heterogeneous memory systems. This challenge becomes particularly pronounced when provisioning resources for deep learning and big data applications in hybrid memory systems. To tackle the performance issues in the hybrid memory systems, we propose and develop a unified memory system, UniRedl, which automatically optimizes data migration between DRAM and NVM based on data access patterns and computation graphs of applications. To improve application performance, we provide a new memory allocation strategy named HiLowAlloc. We further design two data migration strategies in UniRedl, Idle Migration and Dynamic Migration, for management of hybrid memory systems. Specifically, Idle Migration aims to manage data placed in DRAM, while Dynamic Migration manages data saved in NVM. The experimental results demonstrate that on average UniRedl improves application performance by 33.2%, 20.6%, 19.0%, and 17.5% compared to the traditional NUMA, NUMA with anb, BMPM, and OIM, respectively. It also achieves 52.0%, 34.3%, 30.6%, 22.1% on average improvement in data locality against the state-of-the-art solutions.