Device and Placement Aware Framework to optimize Single Failure Recoveries and Reads for Erasure Coded Storage System with Heterogeneous Storage Devices

Abstract

Erasure codes are widely used in cloud storage systems, such as in Google File System and Windows Azure. However, most of existing erasure codes focus on homogeneous storage device, but ignore that heterogeneous devices are in majority in cloud storage system. In this paper, we propose a new erasure code framework termed Device Placement Aware Framework (DPAF), to integrate existing erasure codes to generate DPAF-Codes, in order to gain good performance on heterogeneous storage devices. The key insight is to detect the device performance to generate a series of coefficients, and use these coefficients to choose proper devices to construct DPAF-Codes. We utilize simulated annealing algorithm to optimize the selection process, in order to maintain the balance among devices by considering the coefficients. The experiment results show that, our proposed DPAF-Codes gain up to 70.3%, 48.7%, and 48.5% improvements on single failure recovery, normal read, and degraded read speed, compared to RS code and LRC code with different configurations.