Designing Virtual Memory System of MCM GPUs

Abstract

Multi-Chip Module (MCM) designs have emerged as a key technique to scale up a GPU’s compute capabilities in the face of slowing transistor technology. However, the disaggregated nature of MCM GPUs with many chiplets connected via in-package interconnects leads to non-uniformity. We explore the implications of MCM’s non-uniformity on the GPU’s virtual memory. We quantitatively demonstrate that an MCM-aware virtual memory system should aim to 1 leverage aggregate TLB capacity across chiplets while limiting accesses to L2 TLB on remote chiplets, 2 reduce accesses to page table entries resident on a remote chiplet’s memory during page walks. We propose MCM-aware GPU virtual memory (MGvm) that leverages static analysis techniques, previously used for thread and data placement, to map virtual addresses to chiplets and to place the page tables. At runtime, MGvm balances its objective of limiting the number of remote L2 TLB lookups with that of reducing the number of remote page table accesses to achieve good speedups (52%, on average) across diverse application behaviors.