Minimum Storage Regenerating Codes Research Articles

Minimum Storage Regenerating Codes for Scalable Distributed Storage

Regenerating codes (RGCs) have recently been proposed to reduce the repair traffic of ( $n,k$ ) erasure-coded distributed storage systems. Moreover, RGCs can also be used in a scalable distributed storage scenario where $n$ is increased (decreased) to upgrade (degrade) redundancy while maintaining the maximum distance separable property of erasure codes. In this paper, we propose a new application of minimum storage regenerating (MSR) codes in storage scalability. The connection between repairing invalid nodes and adding new nodes suggests that the two processes can be unified in the same framework. We consider both single and multiple node situations, and two methods for constructing multiple nodes are proposed: concurrent and sequential. We focus on proving the achieved capability of concurrent MSR that can consume minimum traffic for generating multiple nodes. Because concurrent MSR is sensitive to both the number of helpers and added nodes, sequential methods make scalable MSR generalizable. The examples show that the scalable MSR codes have the same advantage of saving network traffic as repairing failures.

Explicit Minimum Storage Regenerating Codes

In distributed storage, a file is stored in a set of nodes and protected by erasure-correcting codes. Regenerating code is a type of code with two properties: first, it can reconstruct the entire file in the presence of any r node erasures for some specified integer r; second, it can efficiently repair an erased node from any subset of remaining nodes with a given size. In the repair process, the amount of information transmitted from each node normalized by the storage size per node is termed repair bandwidth (fraction). When the storage size per node is minimized, the repair bandwidth is lower bounded by 1/r, where r is the number of parity nodes. A code attaining this lower bound is said to have optimal repair. We consider codes with minimum storage size per node and optimal repair, called minimum storage regenerating (MSR) codes. In particular, if an MSR code has r parities and any r erasures occur, then by transmitting all the information from the remaining nodes, the original file can be reconstructed. On the other hand, if only one erasure occurs, only a fraction of 1/r of the information in each remaining node needs to be transmitted. If we view each node as a vector or a column over some field, then the code forms a 2-D array. Given the length of the column l and the number of parities r, we explicitly construct the high-rate MSR codes. The number of systematic nodes of our construction is (r + 1) log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">r</sub> l, which is longer than previously known results. Besides, we construct the MSR codes with other desirable properties: first, the codes with low complexity when the information is updated, and second, the codes with low access or storage node I/O cost during repair.

On the Non-Existence of Minimum Storage Regenerating Codes With Repair-by-Transfer Property

Regenerating codes are the family of distributed storage codes that achieve the optimal trade-off between storage overhead and the bandwidth consumption for the repair of a single node failure. Recent research reveals that, with certain regenerating codes, the amount of disk read for the repair can be reduced to as much as the repair bandwidth (i.e., the amount of data to be transferred), which makes encoding at the helper nodes unnecessary. In literature, such a property is often referred to as the repair-by-transfer property. Searching for regenerating codes with the repair-by-transfer property has been of great interest. In this manuscript, we prove that there does not exist a minimum storage regenerating (MSR) code with the repair-by-transfer property for the case of $k\ge 3$ , $\beta , where $k$ is the number of storage nodes required to reconstruct the original file, $d$ is the number of helper nodes, and $\beta$ is the number of symbols transmitted from each helper node.

A Systematic Piggybacking Design for Minimum Storage Regenerating Codes

Piggybacking is an efficient method to decrease the repair bandwidth of maximum distance separable codes. In this paper, in order to reduce the repair bandwidth of parity nodes of the known minimum storage regenerating (MSR) codes with high rate, which is usually the whole amount of the original data, i.e., the maximal, a new systematic piggybacking design is proposed through an in-depth analysis of the design of piggybacking. As a result, new MSR codes are obtained with almost optimal repair bandwidth of parity nodes while retaining the optimal repair bandwidth of systematic nodes. Furthermore, MSR codes with balanced download during node repair process are presented based on the new piggybacking design.

A New Repair Strategy for the Hadamard Minimum Storage Regenerating Codes for Distributed Storage Systems

The newly presented (k + m, k) Hadamard minimum storage regenerating (MSR) codes are a class of high rate storage codes with optimal repair property for single node failure. In this paper, we propose a new simple optimal repair strategy for (k + m, k) Hadamard MSR codes, which can considerably reduce the computation compared with the original one during the node repair.

Construction of one special minimum storage regenerating code when α=2

Regenerating codes is one class of erasure codes for distributed storage. A [n, k, d, α, β] regenerating code can recover the original message from any k out of n distributed storage nodes and repair any failed node from other arbitrary d nodes. α is the number of symbols stored in one node and β is the number of symbols downloaded by a newcomer node. Minimum storage regenerating (MSR) code is a regenerating code attaining the minimal storage requirement. In this paper, we address the design of a special MSR code where α = 2 and the first fragment of a node stores original symbol (we call it hybrid systematic MSR code or HS-MSR code). We point out that there exists no exact-repair construction when k ⩾ 5. A simple exact linear construction is given when k = 2. Furthermore, by relaxing the condition of connecting any d nodes, we investigate the properties of quasi-cyclic regenerating code and propose a unified construction such that when k ⩾ 3, repairing process can be achieved with less repair bandwidth by repair-by-transfer form.

A Unified Form of Exact-MSR Codes via Product-Matrix Frameworks

Regenerating codes represent a class of block codes applicable for distributed storage systems. The [n, k, d] regenerating code has data recovery capability while possessing arbitrary k out of n code fragments, and supports the capability for code fragment regeneration through the use of other arbitrary d fragments, for k ≤ d ≤ n - 1. Minimum storage regenerating (MSR) codes are a subset of regenerating codes containing the minimal size of each code fragment. The first explicit construction of MSR codes that can perform exact regeneration (named exact-MSR codes) for d ≥ 2k - 2 has been presented via a product-matrix framework. This paper addresses some of the practical issues on the construction of exact-MSR codes. The major contributions of this paper include as follows. A new product-matrix framework is proposed to directly include all feasible exact-MSR codes for d ≥ 2k - 2. The mechanism for a systematic version of exact-MSR code is proposed to minimize the computational complexities for the process of message-symbol remapping. Two practical forms of encoding matrices are presented to reduce the size of the finite field.

Linear Exact-Repair Construction of Hybrid MSR Codes in Distributed Storage Systems

An [n, k, d]-hybrid minimum storage regenerating (HMSR) codes is one class of minimum storage regenerating (MSR) codes where each storage node stores two symbols in a distributed storage system (DSS). Due to the ability of reducing repair bandwidth and minimizing disk I/O cost, HMSR codes are proper to be used in practice. In this letter, we prove that there exists no linear exact-repair HMSR codes when k ≥ 5. A construction of [6,3,4]-HMSR codes is devised in F 2 . Focused on [4,2,3]-HMSR codes, a repair-by-transfer construction and the corresponding regenerating algorithm are proposed.

Optimal Exact-Regenerating Codes for Distributed Storage at the MSR and MBR Points via a Product-Matrix Construction

Regenerating codes are a class of distributed storage codes that allow for efficient repair of failed nodes, as compared to traditional erasure codes. An <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$[n, k, d]$</tex></formula> regenerating code permits the data to be recovered by connecting to any <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$k$</tex></formula> of the <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$</tex></formula> nodes in the network, while requiring that a failed node be repaired by connecting to any <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$d$</tex></formula> nodes. The amount of data downloaded for repair is typically much smaller than the size of the source data. Previous constructions of exact-regenerating codes have been confined to the case <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n=d+1$</tex> </formula> . In this paper, we present optimal, explicit constructions of (a) Minimum Bandwidth Regenerating (MBR) codes for all values of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$[n, k, d]$</tex></formula> and (b) Minimum Storage Regenerating (MSR) codes for all <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$[n, k, d\geq 2k-2]$</tex></formula> , using a new product-matrix framework. The product-matrix framework is also shown to significantly simplify system operation. To the best of our knowledge, these are the first constructions of exact-regenerating codes that allow the number <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$</tex></formula> of nodes in the network, to be chosen independent of the other parameters. The paper also contains a simpler description, in the product-matrix framework, of a previously constructed MSR code with <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$[n=d+1, k, d\geq 2k-1]$</tex></formula> .

Interference Alignment in Regenerating Codes for Distributed Storage: Necessity and Code Constructions

Regenerating codes are a class of recently developed codes for distributed storage that, like Reed-Solomon codes, permit data recovery from any arbitrary k of n nodes. However regenerating codes possess in addition, the ability to repair a failed node by connecting to any arbitrary d nodes and downloading an amount of data that is typically far less than the size of the data file. This amount of download is termed the repair bandwidth. Minimum storage regenerating (MSR) codes are a subclass of regenerating codes that require the least amount of network storage; every such code is a maximum distance separable (MDS) code. Further, when a replacement node stores data identical to that in the failed node, the repair is termed as exact. The four principal results of the paper are (a) the explicit construction of a class of MDS codes for d = n-1 >= 2k-1 termed the MISER code, that achieves the cut-set bound on the repair bandwidth for the exact-repair of systematic nodes, (b) proof of the necessity of interference alignment in exact-repair MSR codes, (c) a proof showing the impossibility of constructing linear, exact-repair MSR codes for d < 2k-3 in the absence of symbol extension, and (d) the construction, also explicit, of MSR codes for d = k+1. Interference alignment (IA) is a theme that runs throughout the paper: the MISER code is built on the principles of IA and IA is also a crucial component to the non-existence proof for d < 2k-3. To the best of our knowledge, the constructions presented in this paper are the first, explicit constructions of regenerating codes that achieve the cut-set bound.