Parallelizing Shared File I/O Operations of NVM File System for Manycore Servers

June-Hyung Kim,Sungyong Park,Safdar Jamil,Chang-Gyu Lee,Youngjae Kim

doi:10.1109/access.2021.3054905

June-Hyung Kim, Sungyong Park + Show 3 more

Open Access

https://doi.org/10.1109/access.2021.3054905

Copy DOI

Abstract

NOVA, a state-of-the-art non-volatile memory (NVM) file system, has limited performance due to its coarse-grained per-file lock when multiple threads perform I/Os to a shared file in a manycore environment. For instance, a writer lock blocks other threads attempting to access the same file, although they access different regions of a file. When multiple threads reading the same file share a cache line containing a reader counter, performance can be significantly degraded due to cache consistency protocol as we increase the number of readers. This paper proposes a fine-grained segment-based range lock (SRL) that divides a file into multiple segments and manages a lock variable dynamically for each segment. Consequently, write operations can be parallelized without blocking unless there is a conflict in accessing the same range in a file. Moreover, SRL maintains a reader counter per segment that allows multiple reader threads to perform read operations without causing a performance bottleneck. We evaluated an SRL-based NOVA on an Intel Optane DC persistent memory (PM) manycore server. The benchmarking results showed that the average write throughput of the SRL-based NOVA is $3\times $ higher than the original NOVA, and the average read throughput scales linearly, while the original NOVA does not scale.

Highlights

In the last decade, distinct non-volatile memory (NVM) devices, such as phase-change memory (PCM) [1], resistive memory (ReRAM) [2] and 3D-Xpoint [3], have been studied and commercialized for various applications
We evaluated the scalability of segment-based range lock (SRL)-based NOVA using FxMark [15] on a manycore server with 56 cores and Intel
NOVA guarantees consistency of the file system using per-inode logging in NVM, where each log records every modification made to the files or directories

Summary

INTRODUCTION

Distinct non-volatile memory (NVM) devices, such as phase-change memory (PCM) [1], resistive memory (ReRAM) [2] and 3D-Xpoint [3], have been studied and commercialized for various applications. NOVA uses a counter-based readers-writer (RW) lock that allows multiple threads to read concurrently from a file while only one thread can write. This becomes another cause of performance bottleneck when the modification of a single shared RW lock variable invalidates the CPU cache lines of other read threads. We proposed a fine-grained segment-based range lock (SRL) and apply it to NOVA to improve the scalability of shared file I/Os on manycore servers. With a shared file read workload, the SRL-based NOVA scales linearly as the number of cores increases, while both original NOVA and NOVA with the interval tree-based range lock do not scale. We provided extensive evaluation results on a manycore machine with 56 CPU cores and Intel Optane DC PM modules, and showed that the SRL-based NOVA outperformed other NOVA versions

BACKGROUND

DYNAMIC SEGMENT LOCK MEMORY OVERHEAD

GUARANTEEING CONSISTENCY

EXPERIMENTAL ENVIRONMENT