Improving storage performance of high-performance computing systems by using the non-volatile buffer cache

Abstract

ABSTRACT With the emerging applications of the 4th industrial revolution, such as IoT (Internet-of-Things), big data, and machine learning, the role of high-performance computing systems is becoming increasingly important. One of the significant performance hurdles in these systems is storage devices as the access time of disk drives is limited to tens of milliseconds. Although buffer cache alleviates the speed gap between memory and storage, data in buffer cache may be lost if the system crashes before updated data is reflected to storage. Thus, pdflush is used to periodically flush the updated data to storage. However, pdflush degrades the buffer cache performance significantly as most writes are directly transferred to storage. We show that pdflush accounts for 64–78% of total write traffic to storage. To resolve this problem, we present a new buffer cache architecture that adopts non-volatile memory to maintain updated data. Experimental results show that the proposed buffer cache reduces storage write traffic by 40%, thereby eliminating most of pdflush overhead.