Minimum Bandwidth Regenerating Research Articles

Single node repair algorithm for a multimedia cloud storage system based on network coding

Multimedia is inconvenient to use, difficult to maintain, and redundant in data storage. In order to solve the above problems and apply cloud storage to the integration of university teaching resources, this paper designs a virtualized cloud storage platform for university multimedia classrooms. The platform has many advantages, such as reducing the initial investment in multimedia classrooms, simplifying management tasks, making maximum use of actual resources and easy access to resources. Experiments and analysis show the feasibility and effectiveness of the platform. Aiming at the problems of the single-node repair algorithm of the existing multimedia cloud storage system, the limited domain is large, the codec complexity is high, the disk I/O (Input/Output) cost is high, the storage overhead and the repair bandwidth are unbalanced, and a network coding-based approach is proposed. Multimedia cloud storage. System single node repair algorithm. The algorithm stores the grouped multimedia file data in groups in the system, and performs XOR (exclusive OR) on the data in the group on the GF(2) finite field. When some nodes fail, the new node only needs to be connected. Two to three non-faulty nodes in the same group can accurately repair the data in the failed node. Theoretical analysis and simulation results show that the algorithm can reduce the complexity and repair of the codec, and reduce the disk I/O overhead. In this case, the storage cost of the algorithm is consistent with the storage cost based on the minimum storage regeneration code algorithm, and the repair bandwidth cost is close to the minimum bandwidth regeneration code algorithm.

PB: A Product-Bitmatrix Construction to Reduce the Complexity of XOR Operations of PM-MSR and PM-MBR Codes over GF 2 w

Edge computing, as an emerging computing paradigm, aims to reduce network bandwidth transmission overhead while storing and processing data on edge nodes. However, the storage strategies required for edge nodes are different from those for existing data centers. Erasure code (EC) strategies have been applied in some decentralized storage systems to ensure the privacy and security of data storage. Product-matrix (PM) regenerating codes (RGCs) as a state-of-the-art EC family are designed to minimize the repair bandwidth overhead or minimize the storage overhead. Nevertheless, the high complexity of the PM framework contains more finite-domain multiplication operations than classical ECs, which heavily consumes computational resources at the edge nodes. In this paper, a theoretical derivation of each step of the PM minimum storage regeneration (PM-MSR) and PM minimum bandwidth regeneration (PM-MBR) codes is performed and the XOR complexity over finite fields is analyzed. On this basis, a new construct called product bitmatrix (PB) is designed to reduce the complexity of XOR operations in the PM framework, and two heuristics are used to further reduce the XOR numbers of the PB-MSR and PB-MBR codes, respectively. The evaluation results show that the PB construction significantly reduces the XOR number compared to the PM-MSR, PM-MBR, Reed–Solomon (RS), and Cauchy RS codes while retaining optimal performance and reliability.

Private Information Retrieval Schemes With Product-Matrix MBR Codes

A private information retrieval (PIR) scheme allows a user to retrieve a file from a database without revealing any information on the file being requested. As of now, PIR schemes have been proposed for several kinds of storage systems, including replicated and MDS-coded systems. However, the problem of constructing PIR schemes on regenerating codes has been sparsely considered. A regenerating code is a storage code whose codewords are distributed among nodes, enabling efficient storage of files, as well as low-bandwidth retrieval of files and repair of nodes. Minimum-bandwidth regenerating (MBR) codes define a family of regenerating codes allowing a node repair with optimal bandwidth. Rashmi, Shah, and Kumar obtained a large family of MBR codes using the product-matrix (PM) construction. In this work, a new PIR scheme over PM-MBR codes is designed. The inherent redundancy of the PM structure is used to reduce the download communication complexity of the scheme. A lower bound on the PIR capacity of MBR-coded PIR schemes is derived, showing an interesting storage space vs. PIR rate trade-off compared to existing PIR schemes with the same reconstruction capability. The present scheme also outperforms a recent PM-MBR PIR construction of Dorkson and Ng.

Beyond the MDS Bound in Distributed Cloud Storage

Regenerating code is a class of distributed storage codes that can optimally trade the bandwidth with the amount of data stored per node to repair a failed node. There are two extreme points in the optimal regenerating trade-off curve, which correspond to minimum-storage regenerating (MSR) and minimum-bandwidth regenerating (MBR). Recently, Reed-Solomon (RS) code based regenerating codes (RS-RC) were constructed under the product-matrix framework. It can also achieve the maximum distance separable (MDS) property in code regeneration and reconstruction. However, in case that the network is hostile and the storage nodes could be compromised or packets be modified, the storage capacity and the bandwidth required to regenerate or reconstruct the original file can be significantly affected. In this paper, we propose Hermitian code based regenerating codes (H-RC) by developing constructions under the product-matrix framework for minimum storage regenerating (H-MSR) and the minimum bandwidth regenerating (H-MBR). We also propose data regeneration and reconstruction algorithms for both H-MSR and H-MBR codes under both error-free and hostile networks. We demonstrate that the proposed algorithms can also successfully determine the erroneous decodings in hostile networks. Theoretical evaluation shows that our proposed H-RC can detect and correct more errors in hostile networks well beyond the RS-RC with the same code rate. Our analysis shows that the proposed H-RC have lower computational complexity than the RS-RC for both code regeneration and code reconstruction.

Efficient Storage Scaling for MBR and MSR Codes

Due to the explosive growth of storage demands, distributed storage systems need to support storage scaling efficiently. Recent work optimizes scaling in a decentralized manner for Reed-solomon coded storage systems. In this paper, we focus on storage scaling for storage systems with regenerating codes and design two efficient scaling algorithms for minimum bandwidth regenerating (MBR) and minimum storage regenerating (MSR) codes. We integrate these two scaling algorithms into Hadoop Distributed File System (HDFS), and the experiments on Amazon EC2 show that the scaling bandwidth can be reduced up to 75% and 43.8% over the centralized scaling.

Minimum Bandwidth Regenerating Codes Based on Cyclic VFR Codes

Minimum Bandwidth Regenerating Codes Based on Cyclic VFR Codes

Decoding and Repair Schemes for Shift-XOR Regenerating Codes

Decoding and repair schemes are proposed for shift-exclusive-or (shift-XOR) product-matrix (PM) regenerating codes, which outperform the existing schemes in terms of both communication and computation costs. In particular, for the shift-XOR minimum bandwidth regenerating (MBR) codes, our decoding and repair schemes have the optimal transmission bandwidth and can be implemented in-place without extra storage space for intermediate XOR results. Technically, our schemes involve an in-place algorithm for solving a system of shift-XOR equations, called shift-XOR elimination , which does not have the bandwidth overhead generated by shift operations as in the previous zigzag algorithm and has lower computation complexities compared with the zigzag algorithm. The decoding and repair of shift-XOR MBR/MSR codes are decomposed into a sequence of systems of shift-XOR equations, and hence can be solved by a sequence of calls to the shift-XOR elimination. As the decompositions of the decoding and repair depend only on the PM construction, but not the specific shift and XOR operations, our decoding and repair schemes can be extended to other MBR/MSR codes using the PM construction. Due to its fundamental role, the shift-XOR elimination is of independent interest.

Multilevel Diversity Coding with Secure Regeneration: Separate Coding Achieves the MBR Point.

The problem of multilevel diversity coding with secure regeneration (MDC-SR) is considered, which includes the problems of multilevel diversity coding with regeneration (MDC-R) and secure regenerating code (SRC) as special cases. Two outer bounds are established, showing that separate coding can achieve the minimum-bandwidth-regeneration (MBR) point of the achievable normalized storage-capacity repair-bandwidth trade-off regions for the general MDC-SR problem. The core of the new converse results is an exchange lemma, which can be established using Han’s subset inequality.

New results on multilevel diversity coding with secure regeneration

The problem of multilevel diversity coding with secure regeneration is revisited. Under the assumption that the eavesdropper can access the repair data for all compromised storage nodes, Shao el al. provided a precise characterization of the minimum-bandwidth-regeneration (MBR) point of the achievable normalized storage-capacity repair-bandwidth tradeoff region. In this paper, it is shown that the MBR point of the achievable normalized storage-capacity repair-bandwidth tradeoff region remains the same even if we assume that the eavesdropper can access the repair data for some compromised storage nodes (type 2 compromised nodes) but only the data contents of the remaining compromised nodes (type 1 compromised nodes), as long as the number of type 1 compromised nodes is no greater than that of type 2 compromised nodes.

A Study on Universally Good Fractional Repetition Codes

Fractional repetition (FR) codes are introduced with the aim of facilitating node repairs in distributed storage systems, i.e., they enable uncoded exact repairs at the minimum bandwidth regenerating (MBR) point. In this letter, we investigate universally good FR (UGFR) codes, which can achieve the storage capacity of MBR codes. We show that UGFR codes are optimal with respect to a bound on the reconstruction degree, which is the number of nodes required for data retrieval. We also present a construction of UGFR codes by establishing a relation with regular packings.

Architecture-aware coding for distributed storage: Repairable block failure resilient codes

In large scale distributed storage systems (DSS) deployed in cloud computing, correlated failures resulting in simultaneous failure (or, unavailability) of blocks of nodes are common. In such scenarios, the stored data or a content of a failed node can only be reconstructed from the available live nodes belonging to the available blocks. To analyze the resilience of the system against such block failures, this work introduces the framework of Block Failure Resilient (BFR) codes, wherein the data (e.g., a file in DSS) can be decoded by reading out from a same number of codeword symbols (nodes) from a subset of available blocks of the underlying codeword. Further, repairable BFR codes are introduced, wherein any codeword symbol in a failed block can be repaired by contacting a subset of remaining blocks in the system. File size bounds for repairable BFR codes are derived, and the trade-off between per node storage and repair bandwidth is analyzed, and the corresponding minimum storage regenerating (BFR-MSR) and minimum bandwidth regenerating (BFR-MBR) points are derived. Explicit codes achieving the two operating points for a special case of parameters are constructed, wherein the underlying regenerating codewords are distributed to BFR codeword symbols according to combinatorial designs. Finally, BFR locally repairable codes (BFR-LRC) are introduced, an upper bound on the resilience is derived and optimal code construction are provided by a concatenation of Gabidulin and MDS codes. Repair efficiency of BFR-LRC is further studied via the use of BFR-MSR/MBR codes as local codes. Code constructions achieving optimal resilience for BFR-MSR/MBR-LRCs are provided for certain parameter regimes. Overall, this work introduces the framework of block failures along with optimal code constructions, and the study of architecture-aware coding for distributed storage systems.

Locally minimum storage regenerating codes in distributed cloud storage systems

In distributed cloud storage systems, inevitably there exist multiple node failures at the same time. The existing methods of regenerating codes, including minimum storage regenerating (MSR) codes and minimum bandwidth regenerating (MBR) codes, are mainly to repair one single or several failed nodes, unable to meet the repair need of distributed cloud storage systems. In this paper, we present locally minimum storage regenerating (LMSR) codes to recover multiple failed nodes at the same time. Specifically, the nodes in distributed cloud storage systems are divided into multiple local groups, and in each local group (4, 2) or (5, 3) MSR codes are constructed. Moreover, the grouping method of storage nodes and the repairing process of failed nodes in local groups are studied. Theoretical analysis shows that LMSR codes can achieve the same storage overhead as MSR codes. Furthermore, we verify by means of simulation that, compared with MSR codes, LMSR codes can reduce the repair bandwidth and disk I/O overhead effectively.

Optimal Construction of Regenerating Code Through Rate-Matching in Hostile Networks

Regenerating code is a class of distributed storage codes that can optimally trade the bandwidth required to repair a failed node with the amount of data stored per node. There are two optimal points in the regeneration tradeoff curve: the minimum storage regeneration code and the minimum bandwidth regeneration code. However, in hostile networks where the storage nodes may be compromised, the storage capacity of the network can be significantly affected. In this paper, we propose two optimal regenerating code constructions through rate-matching to combat this kind of adversarial attacks in hostile networks. We first develop a two-layer rate-matched regenerating code construction. By matching the parameters of the full rate code and the partial rate code, we can optimize the overall storage efficiency while maintaining the corrupted node detection probability. Through comprehensive analysis, we show that the two-layer rate-matched regenerating code can achieve 70% higher storage efficiency than the universally resilient regenerating code. We then propose an optimal $m$ -layer regenerating code construction. While the principle remains the same as the two-layer code, it is designed to optimize the total number of detectable corrupted nodes of $m$ layers from which the errors can be corrected under the constraint of any given code efficiency. Compared with the universally resilient regenerating code with the same rate, our $m$ -layer code can detect 50% more corrupted nodes.

Toward Optimal Secure Distributed Storage Systems With Exact Repair

Distributed storage systems in the presence of a wiretapper are considered. A distributed storage system (DSS) is parameterized by three parameters (n, k,d), in which a file stored across n distributed nodes, can be recovered from any k out of n nodes. If a node fails, any d out of (n-1) nodes help in the repair of the failed node. For such a (n,k,d)-DSS, two types of wiretapping scenarios are investigated: (a) Type-I (node) adversary which can wiretap the data stored on any l<k nodes; and a more severe (b) Type-II (repair data) adversary which can wiretap the contents of the repair data that is used to repair a set of l failed nodes over time. The focus of this work is on the practically relevant setting of exact repair regeneration in which the repair process must replace a failed node by its exact replica. We make new progress on several non-trivial instances of this problem which prior to this work have been open. The main contribution of this paper is the optimal characterization of the secure storage-vs-exact-repair-bandwidth tradeoff region of a (n,k,d)-DSS, with n<=4 and any l<k in the presence of both Type-I and Type-II adversaries. While the problem remains open for a general (n,k,d)-DSS with n>4, we present extensions of these results to a (n, n-1,n-1)-DSS, in presence of a Type-II adversary that can observe the repair data of any l=(n-2) nodes. The key technical contribution of this work is in developing novel information theoretic converse proofs for the Type-II adversarial scenario. From our results, we show that in the presence of Type-II attacks, the only efficient point in the storage-vs-exact-repair-bandwidth tradeoff is the MBR (minimum bandwidth regenerating) point. This is in sharp contrast to the case of a Type-I attack in which the storage-vs-exact-repair-bandwidth tradeoff allows a spectrum of operating points beyond the MBR point.

Optimal Heterogeneous Distributed Storage Regenerating Code at Minimum Remote-Repair Bandwidth Regenerating Point

© 2016 Jian Xu et al. 529 http://dx.doi.org/10.4218/etrij.16.0115.0412 Recently, a product-matrix (PM) framework was proposed to construct optimal regenerating codes for homogeneous distributed storage systems (DSSs). In this paper, we propose an extended PM (EPM) framework for coding of heterogeneous DSSs having different repair bandwidths but identical storage capacities. Based on the EPM framework, an explicit construction of minimum remote-repair bandwidth regenerating (MRBR) codes is presented for a specific heterogeneous DSS, where two geographically different datacenters with associated storage nodes are deployed. The data reconstruction and regeneration properties of the MRBR code are proved strictly. For the purpose of demonstration, an example implementation of MRBR code is provided. The presented MRBR code is the first optimal strict-regenerating code for heterogeneous DSSs. In addition, our proposed EPM framework can be applied to homogeneous systems also.

Optimal Fractional Repetition Codes Based on Graphs and Designs

Fractional repetition (FR) codes is a family of codes for distributed storage systems (DSSs) that allow for uncoded exact repairs having the minimum repair bandwidth. However, in contrast to minimum bandwidth regenerating (MBR) codes, where an arbitrary set of a certain size of available nodes is used for a node repair, the repairs with FR codes are table based. This usually allows to store more data compared with MBR codes. In this paper, we consider bounds on the FR capacity, which is the maximum amount of data that can be stored using an FR code. Optimal FR codes which attain these bounds are presented. The constructions of these FR codes are based on combinatorial designs and on families of regular and biregular graphs. These constructions of FR codes for given parameters raise some interesting questions in graph theory. These questions and some of their solutions are discussed in this paper. In addition, based on a connection between FR codes and batch codes, we propose a new family of codes for DSS, namely, FR batch codes, which have the properties of batch codes and FR codes simultaneously. These are the first codes for DSS which allow for uncoded efficient exact repairs and load balancing which can be performed by several users in parallel. Other concepts related to FR codes are also discussed.

Novel Repair-by-Transfer Codes and Systematic Exact-MBR Codes with Lower Complexities and Smaller Field Sizes

The <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$(n,k,d)$</tex> <mathgraphic fileref="chung-ieq1-2297109.gif" graphicformat="GIF"/></formula> regenerating code is a class of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$(n,k)$</tex><mathgraphic fileref="chung-ieq2-2297109.gif" graphicformat="GIF"/> </formula> erasure codes with the capability to recover a lost code fragment from other <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$d$</tex><mathgraphic fileref="chung-ieq3-2297109.gif" graphicformat="GIF"/></formula> existing code fragments. In this paper, we focus on the design of exact regenerating codes at minimum bandwidth regenerating (MBR) points. For <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$d=n-1$</tex><mathgraphic fileref="chung-ieq4-2297109.gif" graphicformat="GIF"/> </formula> , a class of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$(n,k,d=n-1)$</tex> <mathgraphic fileref="chung-ieq5-2297109.gif" graphicformat="GIF"/></formula> exact-MBR codes, termed as repair-by-transfer codes, have been developed in prior work to avoid arithmetic operations in node repairing process. The first result of this paper presents a new class of repair-by-transfer codes via congruent transformations. As compared with prior works, the advantages of proposed codes include: i) the minimum of field size is significantly reduced from <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n \atopwithdelims ()2$</tex> <mathgraphic fileref="chung-ieq6-2297109.gif" graphicformat="GIF"/></formula> to <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$ </tex><mathgraphic fileref="chung-ieq7-2297109.gif" graphicformat="GIF"/></formula> ; ii) the encoding complexity is decreased from <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n^4$</tex> <mathgraphic fileref="chung-ieq8-2297109.gif" graphicformat="GIF"/></formula> to <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX"> $n^3$</tex><mathgraphic fileref="chung-ieq9-2297109.gif" graphicformat="GIF"/></formula> . Our simulation results show that the proposed code achieves faster operations than the prior approach does under large <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$</tex><mathgraphic fileref="chung-ieq10-2297109.gif" graphicformat="GIF"/> </formula> . The second result of this paper presents a new form of coding matrix for product-matrix exact-MBR codes. The proposed coding matrix includes the following advantages: i) the minimum of finite field size is reduced from <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n-k+d$</tex><mathgraphic fileref="chung-ieq11-2297109.gif" graphicformat="GIF"/> </formula> to <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$</tex> <mathgraphic fileref="chung-ieq12-2297109.gif" graphicformat="GIF"/></formula> ; ii) the fast Reed-Solomon erasure coding algorithms can be applied on the proposed exact-MBR codes to reduce the time complexities.

Codes Between MBR and MSR Points With Exact Repair Property

In this paper distributed storage systems with exact repair are studied. A construction for regenerating codes between the minimum storage regenerating (MSR) and the minimum bandwidth regenerating (MBR) points is given. To the best of author's knowledge, no previous construction of exact-regenerating codes between MBR and MSR points is done except in the work by Tian et al. On contrast to their work, the methods used here are elementary. In this paper it is shown that in the case that the parameters $n$, $k$, and $d$ are close to each other, the given construction is close to optimal when comparing to the known functional repair capacity. This is done by showing that when the distances of the parameters $n$, $k$, and $d$ are fixed but the actual values approach to infinity, the fraction of the performance of constructed codes with exact repair and the known capacity of codes with functional repair, approaches to one. Also a simple variation of the constructed codes with almost the same performance is given.

Codes With Local Regeneration and Erasure Correction

Regenerating codes and codes with locality are two coding schemes that have recently been proposed, which in addition to ensuring data collection and reliability, also enable efficient node repair. In a situation where one is attempting to repair a failed node, regenerating codes seek to minimize the amount of data downloaded for node repair, while codes with locality attempt to minimize the number of helper nodes accessed. This paper presents results in two directions. In one, this paper extends the notion of codes with locality so as to permit local recovery of an erased code symbol even in the presence of multiple erasures, by employing local codes having minimum distance >2. An upper bound on the minimum distance of such codes is presented and codes that are optimal with respect to this bound are constructed. The second direction seeks to build codes that combine the advantages of both codes with locality as well as regenerating codes. These codes, termed here as codes with local regeneration, are codes with locality over a vector alphabet, in which the local codes themselves are regenerating codes. We derive an upper bound on the minimum distance of vector-alphabet codes with locality for the case when their constituent local codes have a certain uniform rank accumulation property. This property is possessed by both minimum storage regeneration (MSR) and minimum bandwidth regeneration (MBR) codes. We provide several constructions of codes with local regeneration which achieve this bound, where the local codes are either MSR or MBR codes. Also included in this paper, is an upper bound on the minimum distance of a general vector code with locality as well as the performance comparison of various code constructions of fixed block length and minimum distance.

Secure Regenerating Codes Based on Rashmi-Shah-Kumar MBR Codes

SUMMARY In this paper, we present a construction of (n,k,d,m )s ecure regenerating codes for distributed storage systems against eavesdroppers that can observe either data stored in at most m storage nodes or downloaded data for repairing at most m failed nodes in a network where m < k ≤ d ≤ n − 1. The (n,k,d,m) secure regenerating code is based on an (n,k,d) minimum bandwidth regenerating (MBR) code, which was proposed by Rashmi, Shah and Kumar as optimal exact-regenerating codes, for all values of the parameters (n,k,d). The (n,k,d,m) secure regenerating codes have the security as a secret sharing scheme such that even if an eavesdropper knows either data stored in at most m storage nodes or downloaded data for repairing at most m failed nodes, no information about data leaks to the eavesdropper.