Maximum Distance Separable Array Research Articles

Extended EVENODD+ and STAR+ With Asymptotic Optimal Update and Efficient Decoding

The increasing capacity of storage devices and the growing frequency of correlated failures demands better fault tolerance by using maximum distance separable (MDS) array codes with triple-parity. Although many constructions of triple-parity MDS array codes have been proposed, they either have large update complexity or have large encoding/decoding complexity or only support specified parameters. In this paper, we propose two classes of triple-parity MDS array codes, called extended EVENODD+ and STAR+ that both have asymptotically optimal update complexity and lower encoding/decoding complexity. We show that the existing extended EVENODD and STAR are a special case of our extended EVENODD+ and STAR+, respectively. Moreover, we show that our extended EVENODD+ and STAR+ have strictly less encoding/decoding/update complexity than that of extended EVENODD and STAR for most parameters.

A Generic Transformation for Optimal Node Repair in MDS Array Codes Over F2

For high-rate linear systematic maximum distance separable (MDS) codes, most early constructions could initially optimally repair all the systematic nodes but not all the parity nodes. Fortunately, this issue was first solved by Li et al. in (IEEE Trans. Inform. Theory, 64(9), 6257-6267, 2018), where a transformation that can convert any nonbinary MDS array code into another one with desired properties was proposed. However, the transformation does not work for binary MDS array codes. In this paper, we address this issue by proposing another generic transformation that can convert any $[n, k]$ binary MDS array code into a new one, which endows any $r=n-k\ge2$ chosen nodes with optimal repair bandwidth and optimal rebuilding access properties, and at the same time, preserves the normalized repair bandwidth/rebuilding access for the remaining $k$ nodes under some conditions. As two immediate applications, we show that 1) by applying the transformation multiple times, any binary MDS array code can be converted into one with optimal rebuilding access for all nodes, 2) any binary MDS array code with optimal repair bandwidth or optimal rebuilding access for the systematic nodes can be converted into one with the corresponding optimality property for all nodes.

Two Classes of Binary MDS Array Codes With Asymptotically Optimal Repair for Any Single Column

An $m\times (k+r)$ binary maximum distance separable (MDS) array code contains $k$ information columns and $r$ parity columns with each entry being a bit, where any $k$ out of $k+r$ columns can recover the $k$ information columns. When there is a failed column, it is critical to minimize the repair bandwidth that is the total number of bits downloaded from $d$ out of $k+r-1$ surviving columns in repairing the failed column. In this article, we first propose two explicit constructions of binary MDS array codes that have asymptotically optimal repair bandwidth for any information column, where $r\geq 2$ and $d=k+r-1$ for the first construction, and $r\geq 4$ is an even number and $d=k+\frac {r}{2}-1$ for the second construction. By applying a generic transformation for the proposed two classes of binary MDS array codes, we then obtain two classes of new binary MDS array codes that also have optimal repair bandwidth for any parity column and asymptotically optimal repair bandwidth for any information column.

Maximum Distance Separable Array Codes Allowing Partial Collaboration

This letter considers the problem of repairing multiple node failures through partial collaboration in a distributed storage system (DSS). In the repair process, each failed node firstly connects to $d\geq k$ alive nodes to download data, and then exchanges data with other repairing nodes. Partial collaboration allows each failed node to only connect to some (not all) of the other repairing nodes to exchange data. Constructions of partially collaborative regenerating codes with $d=k$ at the minimum-storage regime have been studied before.We propose a code construction using the maximum distance separable (MDS) array codes to achieve $d>k$ , and show that the constructed code asymptotically approaches the minimum storage repair point as the number of failed nodes grows.

Exact-Repair Codes With Partial Collaboration in Distributed Storage Systems

The partially collaborative repair of multiple node failures in distributed storage systems is investigated. An exact-repair code is constructed using a bilinear form, which achieves the minimum-bandwidth partially collaborative repair (MBPCR) point. When the storage capacity is minimum, the problem of repairing multi-node failures using a Maximum Distance Separable (MDS) array code is also investigated. The minimum-storage partially collaborative repair (MSPCR) problem with same number of repairing and reconstruction nodes has been studied before. In this paper, the scenario where the number of repairing nodes is greater than that of reconstruction nodes and the storage capacity is already minimal for partial collaboration is considered. An MDS array code is constructed, which asymptotically achieves the MSPCR point. Further, we show that the given code construction could not always have a regular collaborative structure.

Towards Practical Private Information Retrieval From MDS Array Codes

Private information retrieval (PIR) is the problem of privately retrieving one out of $M$ original files from $N$ severs, i.e., each individual server learns nothing about the file that the user is requesting. Usually, the $M$ files are replicated or encoded by a maximum distance separable (MDS) code and then stored across the $N$ servers. Compared to mere replication, MDS coded servers can significantly reduce the storage overhead. Particularly, PIR from minimum storage regenerating (MSR) coded servers can simultaneously reduce the repair bandwidth when repairing failed servers. Existing PIR schemes from MSR coded servers either require large sub-packetization levels or are not capacity-achieving. In this paper, a PIR protocol from MDS array codes is proposed, subsuming PIR from MSR coded servers as a special case. Particularly, the case of non-colluding, honest-but-curious servers is considered. The retrieval rate of the new PIR protocol achieves the capacity of PIR from MDS/MSR coded servers. By choosing different MDS array codes, the new PIR protocol can have some advantages when compared with existing protocols, e.g., 1) small sub-packetization, 2) (near-) optimal repair bandwidth, 3) implementable over the binary field $\mathbf{F}_2$.

A New Design of Binary MDS Array Codes With Asymptotically Weak-Optimal Repair

Binary maximum distance separable (MDS) array codes are a special class of erasure codes for distributed storage that not only provides fault tolerance with minimum storage redundancy but also achieves low computational complexity. They are constructed by encoding $k$ information columns into $r$ parity columns, in which each element in a column is a bit, such that any $k$ out of the $k+r$ columns suffice to recover all information bits. In addition to providing fault tolerance, it is critical to improve repair performance in practical applications. Specifically, if a single column fails, then our goal is to minimize the repair bandwidth by downloading the least amount of bits from $d$ healthy columns, where $k\leq d\leq k+r-1$ . If one column of an MDS code is failed, it is known that we need to download at least $1/(d-k+1)$ fraction of the data stored in each of the $d$ healthy columns. If this lower bound is achieved for the repair of the failure column from accessing arbitrary $d$ healthy columns, we say that the MDS code has optimal repair. However, if such lower bound is only achieved by $d$ specific healthy columns, then we say the MDS code has weak-optimal repair. In this paper, we propose two explicit constructions of binary MDS array codes with more parity columns (i.e., $r\geq 3$ ) that achieve asymptotically weak-optimal repair, where $k+1\leq d\leq k+\lfloor (r-1)/2\rfloor $ , and “asymptotic” means that the repair bandwidth achieves the minimum value asymptotically in $d$ . Codes in the first construction have odd number of parity columns and asymptotically weak-optimal repair for anyone information failure, while codes in the second construction have even number of parity columns and asymptotically weak-optimal repair for any one column failure.

Binary MDS Array Codes With Optimal Repair

Consider a binary maximum distance separable (MDS) array code composed of an $m\times (k+r)$ array of bits with $k$ information columns and $r$ parity columns, such that any $k$ out of $k+r$ columns suffice to reconstruct the $k$ information columns. Our goal is to provide {\em optimal repair access} for binary MDS array codes, meaning that the bandwidth triggered to repair any single failed information or parity column is minimized. In this paper, we propose a generic transformation framework for binary MDS array codes, using EVENODD codes as a motivating example, to support optimal repair access for $k+1\le d \le k+r-1$, where $d$ denotes the number of non-failed columns that are connected for repair; note that when $d<k+r-1$, some of the chosen $d$ columns in repairing a failed column are specific. In addition, we show how our transformation framework applies to an example of binary MDS array codes with asymptotically optimal repair access of any single information column and enables asymptotically or exactly optimal repair access for any column. Furthermore, we present a new transformation for EVENODD codes with two parity columns such that the existing efficient repair property of any information column is preserved and the repair access of parity column is optimal.

HVD code: a class of MDS array codes for tolerating triple disk failures

In this study, the authors propose a class of exclusive OR (XOR)-based maximum distance separable (MDS) array codes, referred to as horizontal-vertical-diagonal (HVD) codes, which can tolerate triple disk failures with optimal update complexity. The HVD code has a similar data/parity layout to the horizontal-vertical (HV) code and has many figures of merits as the HV code. Also, an efficient recovery algorithm is proposed, which can be implemented to accelerate the recovery process of double disk failures by making full use of all kinds of parities in the HVD code. The authors evaluate the encoding/decoding efficiency of the HVD code by comparing the number of XORs and the number of disk reads with other MDS array codes. Results show that the HVD code inherits most of the merits of the HV code and requires less number of disk reads when recovering single disk failure.

A class of binary MDS array codes with asymptotically weak-optimal repair

Binary maximum distance separable (MDS) array codes contain $k$ information columns and $r$ parity columns in which each entry is a bit that can tolerate $r$ arbitrary erasures. When a column in an MDS code fails, it has been proven that we must download at least half of the content from each helper column if $k+1$ columns are selected as the helper columns. If the lower bound is achieved such that the $k+1$ helper columns can be selected from any $k+r-1$ surviving columns, then the repair is an optimal repair. Otherwise, if the lower bound is achieved with $k+1$ specific helper columns, the repair is a weak-optimal repair. This paper proposes a class of binary MDS array codes with $k\geq~3$ and $r\geq~2$ that asymptotically achieve weak-optimal repair of an information column with $k+1$ helper columns. We show that there exist many encoding matrices such that the corresponding binary MDS array codes can asymptotically achieve weak-optimal repair for repairing any information column.

A New Construction and an Efficient Decoding Method for Rabin-Like Codes

Array codes have been widely used in communication and storage systems. To reduce computational complexity, one important property of the array codes is that only exclusive OR operations are used in the encoding and decoding processes. Cauchy Reed–Solomon codes, Rabin-like codes, and circulant Cauchy codes are existing Cauchy maximum-distance separable (MDS) array codes that employ Cauchy matrices over finite fields, circular permutation matrices, and circulant Cauchy matrices, respectively. All these codes can correct any number of failures; however, a critical drawback of existing codes is the high decoding complexity. In this paper, we propose a new construction of Rabin-like codes based on a quotient ring with a cyclic structure. The newly constructed Rabin-like codes have more supported parameters (prime $p$ is extended to an odd number), such that the world sizes of them are more flexible than the existing Cauchy MDS array codes. An efficient decoding method using LU factorization of the Cauchy matrix can be applied to the newly constructed Rabin-like codes. It is shown that the decoding complexity of the proposed approach is less than that of existing Cauchy MDS array codes. Hence, the Rabin-like codes based on the new construction are attractive to distributed storage systems.

Explicit Constructions of High-Rate MDS Array Codes With Optimal Repair Bandwidth

Maximum distance separable (MDS) codes are optimal error-correcting codes in the sense that they provide the maximum failure tolerance for a given number of parity nodes. Suppose that an MDS code with k information nodes and r = n - k parity nodes is used to encode data in a distributed storage system. It is known that if h out of the n nodes are inaccessible and d surviving (helper) nodes are used to recover the lost data, then we need to download at least h/(d + h - k) fraction of the data stored in each of the helper nodes (Dimakis et al., 2010 and Cadambe et al., 2013). If this lower bound is achieved for the repair of any h erased nodes from any d helper nodes, we say that the MDS code has the (h, d)-optimal repair property. We study high-rate MDS array codes with the optimal repair property (also known as minimum storage regenerating codes, or MSR codes). Explicit constructions of such codes in the literature are only available for the cases where there are at most three parity nodes, and these existing constructions can only optimally repair a single node failure by accessing all the surviving nodes. In this paper, given any r and n, we present two explicit constructions of MDS array codes with the (h, d)-optimal repair property for all h r and k d n - h simultaneously. Codes in the first family can be constructed over any base field F as long as |F| ≥ sn, where s = lcm(1, 2, . .. , r). The encoding, decoding, repair of failed nodes, and update procedures of these codes all have low complexity. Codes in the second family have the optimal access property and can be constructed over any base field F as long as |F| ≥ n+1. Moreover, both code families have the optimal error resilience capability when repairing failed nodes. We also construct several other related families of MDS codes with the optimal repair property.

XI-Code: A Family of Practical Lowest Density MDS Array Codes of Distance 4

Designing the lowest density maximum-distance separable (MDS) array codes has gained much attention in recent years due to the optimal redundancy and minimum update penalty (lowest density) of such codes. However, the existing lowest density MDS array codes of distance 4 have extremely strict constraints on the code length, which makes them impractical. In particular, most of them require the code length to be $p$ (or $p - 1$ ), where $p$ is a prime that satisfies: i) $3|(p-1)$ and ii) 2 is primitive in ${\mathrm {GF}} (p)$ . In this paper, we propose a new family of the lowest density MDS array codes of distance 4, called XI-Code. It has the properties of: 1) being capable of correcting both triple erasures and a single error combined with one erasure; 2) having code length of either $p$ or $p + 1$ with $p$ being an odd prime number; 3) achieving optimality in encoding and update; and 4) achieving optimality in erasure decoding for certain erasure patterns and near optimality for other erasure patterns. It is worth mentioning that XI-Code is the first discovered family of the lowest density MDS array codes of distance 4 that supports the code length of $p$ or $p + 1$ with $p$ being an odd prime.

On the MDS Condition of Blaum–Bruck–Vardy Codes With Large Number Parity Columns

Binary maximum distance separable (MDS) array codes are widely used in storage systems. EVENODD code and row-diagonal parity (RDP) code are two well-known binary MDS array codes with two parity columns. The Blaum–Bruck–Vardy (BBV) code is an extension of the double-erasure-correcting EVENODD code. It is known that the BBV code is always MDS for three parity columns, and sufficient conditions for up to eight parity columns to be MDS are also known. However, the MDS condition for more than eight parity columns is an open problem since then. In this letter, we study the MDS condition of BBV code and give a sufficient MDS condition of the BBV code with more than eight parity columns.

Two New Classes of 2-Parity MDS Array Codes with Optimal Repair

Two-parity maximum distance separable (MDS) array codes achieve the lowest storage overhead for tolerating two erasures. In this letter, we propose two new classes of two-parity MDS array codes over finite field $\mathbb {F}_{2}$ , with which each single node failure can be recovered by reading the minimum amount of data from each surviving node.

Lowest Density MDS Array Codes of Distance 3

We present a new family of maximum-distance separable (MDS) array codes of distance 3, called S(ymmetry)-Code, which has the properties of: 1) optimality in encoding, decoding and update; 2) code length of either $p$ or $p - 1$ with $p$ being an odd prime; 3) requiring less I/O cost than most existing lowest density MDS array codes during single-erasure recovery; and 4) approaching the lower bound in I/O cost of single-erasure recovery when the code length is small.

Lowest‐density MDS array codes for reliable Smart Meter networks

AbstractIn this paper, we introduce a lowest‐density maximum‐distance separable (MDS) array code, which is applied to a Smart Meter network to introduce reliability. By treating the network as distributed storage with multiple sources, information can be exchanged between the nodes in the network allowing each node to store parity symbols relating to data from other nodes. A lowest‐density MDS array code is then applied to make the network robust against outages, ensuring low overhead and data transfers. We show the minimum amount of overhead required to be able to recover from r node erasures in an n node network and explicitly design an optimal array code with lowest density. In contrast to existing codes, this one has no restrictions on the number of nodes or erasures it can correct. Furthermore, we consider incomplete networks where all nodes are not connected to each other. This limits the exchange of data for purposes of redundancy, and we derive conditions on the minimum node degree that allow lowest‐density MDS codes to exist. We also present an explicit code design for incomplete networks that is capable of correcting two node failures. Copyright © 2015 John Wiley &amp; Sons, Ltd.

Irregular MDS Array Codes

In this paper, we extend the concept of maximum-distance separable (MDS) array codes to a larger class of codes, where the array columns contain a variable number of data and parity symbols and the codewords cannot be arranged, in general, in a regular array structure with equal column length. These new codes, named irregular MDS array codes, find applications in problems of distributed data storage with multiple sources of information generating data at unequal rates. We solve the problem of finding optimal parity symbol allocations that achieve minimum redundancy for a given level of protection against block erasures. We provide a classification of irregular MDS array codes according to the parameters of their parity symbol allocation and we show how regular MDS array codes are a special case of this wider class. We derive necessary and sufficient conditions for such irregular array codes to be MDS and extend the concept of the lowest density generator matrix. Finally, we show how a simple constructive method allows to design irregular lowest density MDS array codes with alphabet size independent of the size of the array columns.

LDPC Codes for 2D Arrays

Binary codes over 2D arrays are very useful in data storage, where each array column represents a storage device or unit that may suffer failure. In this paper, we propose a new framework for probabilistic construction of codes on 2D arrays. Instead of a pure combinatorial erasure model used in traditional array codes, we propose a mixed combinatorial-probabilistic model of limiting the number of column failures, and assuming a binary erasure channel in each failing column. For this model, we give code constructions and detailed analysis that allow sustaining a large number of column failures with graceful degradation in the fraction of erasures correctable in failing columns. Another advantage of the new framework is that it uses low-complexity iterative decoding. The key component in the analysis of the new codes is to analyze the decoding graphs induced by the failed columns, and infer the decoding performance as a function of the code design parameters, as well as the array size and failure parameters. A particularly interesting class of codes, called probabilistically maximum distance separable (MDS) array codes, gives fault-tolerance that is equivalent to traditional MDS array codes. The results also include a proof that the 2D codes outperform standard 1D low-density parity-check codes.

An Improved Sub-Packetization Bound for Minimum Storage Regenerating Codes

Distributed storage systems employ codes to provide resilience to failure of multiple storage disks. In particular, an (n, k) maximum distance separable (MDS) code stores k symbols in n disks such that the overall system is tolerant to a failure of up to n - k disks. However, access to at least k disks is still required to repair a single erasure. To reduce repair bandwidth, array codes are used where the stored symbols or packets are vectors of length ℓ. The MDS array codes have the potential to repair a single erasure using a fraction 1/(n - k) of data stored in the remaining disks. We introduce new methods of analysis, which capitalize on the translation of the storage system problem into a geometric problem on a set of operators and subspaces. In particular, we ask the following question: for a given (n, k), what is the minimum vector-length or subpacketization factor ℓ required to achieve this optimal fraction? For exact recovery of systematic disks in an MDS code of low redundancy, i.e., k/n > 1/2, the best known explicit codes have a subpacketization factor ℓ, which is exponential in k. It has been conjectured that for a fixed number of parity nodes, it is in fact necessary for ℓ to be exponential in k. In this paper, we provide a new log-squared converse bound on k for a given ℓ, and prove that k ≤ 2 log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> I(log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">δ</sub> ℓ + 1), for an arbitrary number of parity nodes r = n - k, where δ = r/(r - 1).