The Efficient Erasure code with Low Recovery-Overhead and degraded reads optimization

Abstract

At present, the distributed storage systems are being adopted by more and more Internet companies because of their reliability and efficiency. In the distributed storage systems, some fault-tolerant technologies are often used to ensure the data reliability. Since the erasure code technology can store more data with lower storage space, it is often used as one of fault-tolerant technologies by some distributed storage systems. However, the existing erasure code technology has many problems. The large amount of repair overhead and I/O overhead consumed to recover data after data loss has become the biggest obstacle in actual use. This paper starts from the RS code, and constructs a new packet repair SLRC code by using the coding idea of stripe set, which can effectively reduce the repair cost required for single-node data recovery. At the same time, for the structure of SLRC code, a new data storage method, Curve storage, can effectively reduce the extra I/O overhead required for SLRC code degraded read operation. The experimental horizontal comparison with the LRC code and the RS code shows that the new method can effectively reduce the single node repair overhead and the additional I/O overhead required for the degraded read operation.