A Multiagent Reinforcement Learning-Assisted Cache Cleaning Scheme for DM-SMR

Abstract

To support non-sequential writes, Persistent Cache (PC) is constructed in DM-SMR drive. However, PC cleaning introduces drastic performance degradation and enlarges tail latencies. In this paper, we propose to utilize Reinforcement Learning (RL) to mitigate the long-tail latency of PC cleaning. Our scheme uses lightweight Q-learning method to monitor and learn the idle time of I/O workloads, based on which PC cleaning is intelligently guided, thus maximally exploit idle time between requests and hiding tail latency from normal requests. In addition, a multi-agent RL scheme with clustering algorithm is adopted to further mitigate the tail latencies and adapt to variable workloads. We emulate a DM-SMR drive inside a Linux device driver to implement our proposed scheme. According to the experimental results, our scheme can effectively reduce the tail latency by 59.45% at 99.9th percentile and the average latency by 48.75% compared with a typical SMR design.