Performance optimization method for distributed storage system based on hybrid storage device

Abstract

In the era of big data and artificial intelligence, the storage data of data centers is growing rapidly. Big data storage systems with high-performance computing require high-speed data access and greater storage capacity. However, traditional storage technology based on hard disk media (HDDs) has been difficult to meet the current demand for high-performance storage, solid state drives (SSDs)’ cost will be a lot for storing massive amounts of data. Therefore, in response to the above problems, this paper proposes a hierarchical hybrid storage technology for distributed file system, including an adaptive file migration strategy and client caching mechanism. Through experimental verification, the proposed technology fully combines the high I/O throughput characteristics of SSDs and the large capacity characteristics of disks, thereby providing a cost-effective high-performance distributed storage solution, and when processing real-time data, it has higher access performance.