Enabling Scalable and Extensible Memory-Mapped Datastores in Userspace

Abstract

Exascale workloads are expected to incorporate data-intensive processing in close coordination with traditional physics simulations. These emerging scientific, data-analytics and machine learning applications need to access a wide variety of datastores in flat files and structured databases. Programmer productivity is greatly enhanced by mapping datastores into the application process's virtual memory space to provide a unified “in-memory” interface. Currently, memory mapping is provided by system software primarily designed for generality and reliability. However, scalability at high concurrency is a formidable challenge on exascale systems. Also, there is a need for extensibility to support new datastores potentially requiring HPC data transfer services. In this article, we present <i>UMap</i> , a scalable and extensible userspace service for memory-mapping datastores. Through decoupled queue management, concurrency aware adaptation, and dynamic load balancing, <i>UMap</i> enables application performance to scale even at high concurrency. We evaluate <i>UMap</i> in data-intensive applications, including sorting, graph traversal, database operations, and metagenomic analytics. Our results show that <i>UMap</i> as a userspace service outperforms an optimized kernel-based service across a wide range of intra-node concurrency by 1.22-1.9 <inline-formula><tex-math notation="LaTeX">${\times}$</tex-math></inline-formula> . We performed two case studies to demonstrate <i>UMap</i> 's extensibility. First, a new datastore residing in remote memory is incorporated into <i>UMap</i> as an application-specific plugin. Second, we present a persistent memory allocator <i>Metall</i> built atop <i>UMap</i> for unified storage/memory.