Designing High-Performance In-Memory Key-Value Operations with Persistent GPU Kernels and OpenSHMEM

Abstract

Graphics Processing Units (GPUs) are popular for their massive parallelism and high bandwidth memory and are being increasingly used in data-intensive applications. In this context, GPU-based In-Memory Key-Value (G-IMKV) Stores have been proposed to take advantage of GPUs’ capability to achieve high-throughput indexing operations. The state-of-the-art implementations batch requests on the CPU at the server before launching a compute kernel to process operations on the GPU. They also require explicit data movement operations between the CPU and GPU. However, the startup overhead of compute kernel launches and memory copies limit the throughput of these frameworks unless operations are batched into large groups.