Fast and efficient automatic memory management for GPUs using compiler-assisted runtime coherence scheme

Abstract

Exploiting the performance potential of GPUs requires managing the data transfers to and from them efficiently which is an error-prone and tedious task. In this paper, we develop a software coherence mechanism to fully automate all data transfers between the CPU and GPU without any assistance from the programmer. Our mechanism uses compiler analysis to identify potential stale accesses and uses a runtime to initiate transfers as necessary. This allows us to avoid redundant transfers that are exhibited by all other existing automatic memory management proposals. We integrate our automatic memory manager into the X10 compiler and runtime, and find that it not only results in smaller and simpler programs, but also eliminates redundant memory transfers. Tested on eight programs ported from the Rodinia benchmark suite it achieves (i) a 1.06× speedup over hand-tuned manual memory management, and (ii) a 1.29× speedup over another recently proposed compiler-runtime automatic memory management system. Compared to other existing runtime-only and compiler-only proposals, it also transfers 2.2× to 13.3× less data on average.