Efficient Exact Regenerating Codes for Byzantine Fault Tolerance in Distributed Networked Storage

Abstract

Today's large-scale distributed storage systems are commonly built using commodity software and hardware. As a result, crash-stop and Byzantine failures in such systems become more and more prevalent. In the literature, regenerating codes have been shown to be a more efficient way to disperse information across multiple storage nodes and recover from crash-stop failures. In this paper, we propose a novel decoding design of product-matrix constructed regenerating codes in conjunction with integrity check that allows exact regeneration of failed nodes and data reconstruction in the presence of Byzantine failures. A progressive decoding mechanism is incorporated in both procedures to leverage computation performed thus far. Unlike previous works, our new regenerating code decoding has the advantage that its building blocks, such as Reed-Solomon codes and standard cryptographic hash functions, are relatively well-understood because of their widespread applications. The fault tolerance and security properties of the proposed schemes are also analyzed. In addition, the performance of the proposed schemes, in terms of the average number of access nodes and the reconstruction failure probability versus the node failure probability, are also evaluated by Monte Carlo simulations.