A High-bandwidth High-capacity Hybrid 3D Memory for GPUs

Abstract

GPUs execute thousands of active threads simultaneously, requiring high memory bandwidth to handle multiple memory requests efficiently. The memory bandwidth in GPUs has always been increasing, but it is still insufficient for the demands of fine-grained threads, necessitating a higher memory bandwidth. Important workloads like deep learning and data analytics demand terabytes of data processing, necessitating high memory capacity and bandwidth to avoid performance overheads. True-3D stacking of non-volatile memory layers on GPUs can provide the required higher bandwidth and capacity, enhancing performance and energy efficiency. We propose a high-bandwidth high-capacity hybrid 3D memory (H3DM) that doubles bandwidth through true-3D integration compared to the baseline GPU architecture and affords 272 GB of total memory capacity by stacking 8 PCM layers (each of 32 GB) and two DRAM layers (each of 8 GB).