Locally Recoverable Codes Research Articles

Curve-lifted codes for local recovery using lines

AbstractIn this paper, we introduce curve-lifted codes over fields of arbitrary characteristic, inspired by Hermitian-lifted codes over $$\mathbb {F}_{2^r}$$ F 2 r . These codes are designed for locality and availability, and their particular parameters depend on the choice of curve and its properties. Due to the construction, the numbers of rational points of intersection between curves and lines play a key role. To demonstrate that and generate new families of locally recoverable codes (LRCs) with high availabilty, we focus on norm-trace-lifted codes.

Optimal (r,δ)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(r,\\delta )$$\\end{document}-LRCs from monomial-Cartesian codes and their subfield-subcodes

We study monomial-Cartesian codes (MCCs) which can be regarded as (r,δ)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(r,\\delta )$$\\end{document}-locally recoverable codes (LRCs). These codes come with a natural bound for their minimum distance and we determine those giving rise to (r,δ)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(r,\\delta )$$\\end{document}-optimal LRCs for that distance, which are in fact (r,δ)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(r,\\delta )$$\\end{document}-optimal. A large subfamily of MCCs admits subfield-subcodes with the same parameters of certain optimal MCCs but over smaller supporting fields. This fact allows us to determine infinitely many sets of new (r,δ)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(r,\\delta )$$\\end{document}-optimal LRCs and their parameters.

Practical Design Considerations for Wide Locally Recoverable Codes (LRCs)

Most of the data in large-scale storage clusters is erasure coded. At exascale, optimizing erasure codes for low storage overhead, efficient reconstruction, and easy deployment is of critical importance. Locally recoverable codes (LRCs) have deservedly gained central importance in this field, because they can balance many of these requirements. In our work, we study wide LRCs; LRCs with large number of blocks per stripe and low storage overhead. These codes are a natural next step for practitioners to unlock higher storage savings, but they come with their own challenges. Of particular interest is their reliability , since wider stripes are prone to more simultaneous failures. We conduct a practically minded analysis of several popular and novel LRCs. We find that wide LRC reliability is a subtle phenomenon that is sensitive to several design choices, some of which are overlooked by theoreticians, and others by practitioners. Based on these insights, we construct novel LRCs called Uniform Cauchy LRCs , which show excellent performance in simulations and a 33% improvement in reliability on unavailability events observed by a wide LRC deployed in a Google storage cluster. We also show that these codes are easy to deploy in a manner that improves their robustness to common maintenance events. Along the way, we also give a remarkably simple and novel construction of distance-optimal LRCs (other constructions are also known), which may be of interest to theory-minded readers.

Optimal Data Decoding Strategies for Product-Coded Sequential Media Recording Via Latin Squares

In sequential media recording, such as found in tape data storage, data are typically accessed by reading signals from the beginning to the end of the recording medium in a streaming mode, which makes it unsuitable for random write accesses. However, sequential media is usually the preferred method for cold data storage in long-term data retention scenarios. In this study, we propose to increase the durability of sequential media by encoding data with locally recoverable codes (LRCs) both within and across different storage units creating a multidimensional product code external to the product code already implemented in the hardware. This way, local LRCs provide protection across the medium, whereas the LRCs spanning across different cartridges will provide global protection, which requires multiple drives and cartridges to help with the local decoding. Since global decoding is extremely costly, we require data allocation strategies that would use library resources efficiently. More specifically, we propose optimal data decoding and allocation strategies that would ensure minimum repair latency/communication cost in a global decoding setting for a given data request when local data reconstruction is not an option. We furthermore conduct analysis to find closed-form expressions and/or bounds on the overall latency/communication cost for product-coded systems under maintenance.

Optimal quaternary $$(r,\delta )$$-locally recoverable codes: their structures and complete classification

Locally recoverable codes (LRCs) have been introduced as a family of erasure codes that support the repair of a failed storage node by contacting a small number of other nodes in the cluster. Boosted by their applications in distributed storage, LRCs have attracted a lot of attention in recent literature since the concept of codes with locality r was introduced by Gopalan et al. in 2012. Aiming to recover the data from several concurrent node failures, the concept of r-locality was later generalized as $$(r, \delta )$$ -locality by Prakash et al. An $$(r, \delta )$$ -LRCs in which every code symbol has $$(r, \delta )$$ -locality is said to be optimal if it achieves the Singleton-like bound with equality. In present paper, we are interested in optimal $$(r, \delta )$$ -LRCs over small fields, more precisely, over quaternary field. We study their parity-check matrices or generator matrices, using the properties of projective space. The classification of optimal quaternary $$(r,\delta )$$ -LRCs and their explicit code constructions are proposed by examining all possible parameters.

Multiple-Layer Integrated Interleaved Codes: A Class of Hierarchical Locally Recoverable Codes

The traditional definition of Integrated Interleaved (II) codes generally assumes that the component nested codes are either Reed-Solomon (RS) or shortened Reed-Solomon codes. By taking general classes of codes, we present a recursive construction of Extended Integrated Interleaved (EII) codes into multiple layers, a problem that brought attention in literature for II codes. The multiple layer approach allows for a hierarchical scheme where each layer of the code provides for a different locality. In particular, we present the erasure-correcting capability of the new codes and we show that they are ideally suited as Locally Recoverable codes (LRC) due to their hierarchical locality and the small finite field required by the construction. Properties of the multiple layer EII codes, like their minimum distance and dimension, as well as their erasure decoding algorithms, parity-check matrices and performance analysis, are provided and illustrated with examples. Finally, we will observe that the parity-check matrices of high layer EII codes have low density.

A function field approach toward good polynomials for further results on optimal LRC codes

Because of the recent applications to distributed storage systems, researchers have introduced a new class of block codes, i.e., locally recoverable (LRC) codes. LRC codes can recover information from erasure(s) by accessing a small number of erasure-free code symbols and increasing the efficiency of repair processes in large-scale distributed storage systems. In this context, Tamo and Barg first gave a breakthrough by cleverly introducing a good polynomial notion. Constructing good polynomials for locally recoverable codes achieving Singleton-type bound (called optimal codes) is challenging and has attracted significant attention in recent years. This article aims to increase our knowledge of good polynomials for optimal LRC codes. Using tools from algebraic function fields and Galois theory, we continue investigating those polynomials and studying them by developing the Galois theoretical approach initiated by Micheli in 2019. Specifically, we push further the study of a crucial parameter G(f) (of a given polynomial f), which measures how much a polynomial is “good” in the sense of LRC codes. We provide some characterizations of polynomials with minimal Galois groups and prove some properties of finite fields where polynomials exist with a specific size of Galois groups. We also present some explicit shapes of polynomials with small Galois groups. For some particular polynomials f, we give the exact formula of G(f).

Good polynomials for optimal LRC of low locality

According to a magnific method due to I. Tamo and A. Barg, a class of polynomials over finite fields, called good polynomials, was introduced and used to construct optimal Locally Recoverable Codes (LRC), which have been developed and exploited in distributed storage. An important derived algebraic problem is, for a given finite field $$\mathbb {F}_q$$ and a fixed integer r, to find a polynomial of degree $$r+1$$ that is constant on as many subsets of $$\mathbb {F}_q$$ as possible of size $$r+1$$ . Compared to the literature on this topic, our main contribution is introducing a new parameter that measures how “good” a polynomial is in the sense of LRC. Our new approach allows us to characterize completely good polynomials of a low degree over finite fields and, next, to derive new constructions of such polynomials, leading to optimal LRC with new flexible localities. Specifically, several good polynomials of degree at most 6 are studied and described precisely in this paper.

Single-Server Private Information Retrieval Schemes are Equivalent to Locally Recoverable Coding Schemes

The Private Information Retrieval (PIR) problem has recently attracted a significant interest in the information-theory community. In this problem, a client wants to download one or more messages belonging to a database while protecting the identity of the downloaded message(s). In this article, we focus on the scenarios in which (i) the entire database is stored on a single server and (ii) the client has prior side information, namely a subset of messages unknown to the server. Such prior side information is necessary to enable efficient private information retrieval in the single server scenario. In the last decade, there has also been a significant interest in Locally Recoverable Codes (LRCs), a class of storage codes in which each symbol can be recovered from a limited number of other symbols. More recently, there is an interest in cooperative locally recoverable codes, i.e., codes in which multiple symbols can be recovered from a small set of other code symbols. The central problem in this context is given a set of code parameters to design an LRC scheme that includes a locally recoverable code along with encoding, decoding, and repair functions. The paper establishes an equivalence between the single-server PIR schemes and LRC schemes. In particular, we present explicit algorithms that transform a given PIR scheme into an LRC scheme and vice versa. We show that (i) PIR schemes for retrieving a single message are equivalent to classical LRC schemes; and (ii) PIR schemes for retrieving multiple messages are equivalent to cooperative LRC schemes. These equivalence results allow us to recover upper bounds on the download rate for PIR schemes, and to obtain a novel rate upper bound on cooperative LRC schemes. Our results cover schemes based on both linear and non-linear codes.

Cyclic and Convolutional Codes With Locality

Locally recoverable (LRC) codes and their variants have been extensively studied in recent years. In this paper we focus on cyclic constructions of LRC codes and derive conditions on the zeros of the code that support the property of hierarchical locality. As a result, we obtain a general family of hierarchical LRC codes for a new range of code parameters. We also observe that our approach enables one to represent an LRC code in quasicyclic form, and use this representation to construct tail-biting convolutional LRC codes with locality. Among other results, we extend the general approach to cyclic codes with locality to multidimensional cyclic codes, yielding new families of LRC codes with availability, and construct a family of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula> -ary cyclic hierarchical LRC codes of unbounded length.

Cyclic LRC-LCD Codes With Hierarchical Locality

Locally recoverable (LRC) codes and linear complementary dual (LCD) codes are both useful in distributed storage systems. In this letter, we establish a connection between LRC codes and LCD codes. We derive some conditions on the construction of cyclic codes with $(r,\delta)$ locality and codes with hierarchical locality so that they are also LCD codes. Some constructed codes are optimal. These codes may be useful in simultaneously providing local recoverability and security against certain attacks in distributed storage systems.

A New Construction of Optimal (r, δ) Locally Recoverable Codes

Locally recoverable codes (LRCs) have a great significance in distributed storage systems, and have received considerable attention in recent years. In particular, it is a challenging task to construct optimal $(r,\delta)$ -LRCs, meaning $(r,\delta)$ -LRCs whose minimum distances attain Singleton-type bound. In this letter, we investigate the construction of a family of optimal $(r,\delta)$ -LRCs via generalized Reed-Solomon codes (GRS codes). Our strategy is to equip parity-check matrices for optimal $(r,\delta)$ -LRCs with the Vandermonde structure. Furthermore, based on these new optimal $(r,\delta)$ -LRCs we present a family of optimal locally recoverable codes with hierarchical locality (H-LRCs). The parameters of our results are not covered in the literature.

RS-like locally recoverable codes with intersecting recovering sets

A recovering set for a coordinate position i in a code is a set Ri of other coordinate positions such that the value at the ith position can be recovered by accessing the values at coordinate positions in Ri. A locally recoverable (LRC) code with multiple recovering sets is a code in which for every coordinate position there are more than one recovering set. Such codes have been generally studied with the assumption that the recovering sets are pairwise disjoint. Recently Kruglik et al. [1] have presented a construction of LRC codes in which the recovering sets of a coordinate need not be disjoint. In this paper, we present a construction of such type of codes by using the construction of RS-like LRC codes. Further, using a bound given in [1], we have obtained a bound on the rate of the codes from the present construction. Also, we have presented a sufficient condition for a cyclic code over a finite field to be an LRC code with intersecting recovering sets.

Codes With Minimum Bandwidth Cooperative Local Regeneration

Locally recoverable codes (LRCs) are known to reduce the repair locality, i.e., the number of nodes accessed during node repair, in distributed storage systems. Regenerating codes, on the other hand, offer a decreased repair bandwidth, which is the repair traffic incurred during node repairs. Locally regenerating codes (LRGCs), which are constructed via intertwining regenerating codes of smaller block lengths, simultaneously offer a small repair locality and repair bandwidth. In this letter, we construct a family of LRGCs where the constituent codes are bandwidth-efficient, minimum bandwidth cooperative regenerating codes. The newly constructed LRGCs are optimal with respect to both minimum distance and rate.

Optimal Cyclic Codes With Hierarchical Locality

By introducing several levels of recoverability for locally recoverable codes (LRCs), LRCs with hierarchical locality (H-LRCs) are defined for correcting different numbers of erasures. There are two major ingredients in this paper. The first is to investigate some properties and existence conditions of optimal H-LRCs with respect to a generalized Singleton-like bound for H-LRCs. The second ingredient is to propose several constructions of optimal H-LRCs by employing cyclic codes. Notably, the parameters of these optimal H-LRCs are flexible.

Locally recoverable J-affine variety codes

A locally recoverable (LRC) code is a code over a finite field Fq such that any erased coordinate of a codeword can be recovered from a small number of other coordinates in that codeword. We construct LRC codes correcting more than one erasure, which are subfield-subcodes of some J-affine variety codes. For these LRC codes, we compute localities (r,δ) that determine the minimum size of a set R‾ of positions so that any δ−1 erasures in R‾ can be recovered from the remaining r coordinates in this set. We also show that some of these LRC codes with lengths n≫q are (δ−1)-optimal.

Constructions of Locally Recoverable Codes Which are Optimal

Let $q$ be a prime power and $\mathbb F_q$ be the finite field of size $q$. In this paper we provide a Galois theoretical framework that allows to produce good polynomials for the Tamo and Barg construction of optimal locally recoverable codes (LRC). Using our approach we construct new good polynomials and then optimal LRCs with new parameters. The existing theory of good polynomials fits entirely in our new framework. The key advantage of our method is that we do not need to rely on arithmetic properties of the pair $(q,r)$, where $r$ is the locality of the code.

A Tight Rate Bound and Matching Construction for Locally Recoverable Codes With Sequential Recovery From Any Number of Multiple Erasures

By a locally recoverable code (LRC), we will in this paper, mean a linear code in which a given code symbol can be recovered by taking a linear combination of at most $r$ other code symbols with $r << k$. A natural extension is to the local recovery of a set of $t$ erased symbols. There have been several approaches proposed for the handling of multiple erasures. The approach considered here, is one of sequential recovery meaning that the $t$ erased symbols are recovered in succession, each time contacting at most $r$ other symbols for assistance in recovery. Under the constraint that each erased symbol be recoverable by contacting at most $r$ other code symbols, this approach is the most general and hence offers maximum possible code rate. We characterize the maximum possible rate of an LRC with sequential recovery for any $r \geq 3$ and $t$. We do this by first deriving an upper bound on code rate and then going on to construct a {\em binary} code that achieves this optimal rate. The upper bound derived here proves a conjecture made earlier relating to the structure (but not the exact form) of the rate bound. Our approach also permits us to deduce the structure of the parity-check matrix of a rate-optimal LRC with sequential recovery. The parity-check matrix in turn, leads to a graphical description of the code. The construction of a binary code having rate achieving the upper bound derived here makes use of this description. Interestingly, it turns out that a subclass of binary codes that are both rate and block-length optimal, correspond to graphs known as Moore graphs that are regular graphs having the smallest number of vertices for a given girth. A connection with Tornado codes is also made in the paper.

Relaxing the Constraints on Locally Recoverable Erasure Codes by Finite Field Element Variation

To enable the continued scaling of distributed storage, locally recoverable (LRC) erasure codes are necessary to substantially reduce the number of symbols to access for failure recovery. Codes based on parity splitting and generalized integrated interleaved codes are two categories of LRC codes that achieve good tradeoffs on locality, redundancy, and complexity. The parity check and nesting matrices in these codes are modified to improve the complexity and locality. On the other hand, these structural modifications add constraints on the code parameters. This letter proposes to relax the constraints by exploiting different finite field elements, field constructions, and independent entries in the matrices. The computation complexity of finding the maximum allowed values of the parameters has been reduced by orders of magnitudes through taking advantage of isomorphic mapping and conjugacy classes, as well as developing a decomposed searching method. Significant improvements on the applicable code parameters have been achieved for both codes using the proposed schemes.

Extended Integrated Interleaved Codes Over Any Field With Applications to Locally Recoverable Codes

Integrated Interleaved (II) and Extended Integrated Interleaved (EII) codes are a versatile alternative for Locally Recoverable (LRC) codes, since they require fields of relatively small size. II and EII codes are generally defined over Reed-Solomon type of codes. A new comprehensive definition of EII codes is presented, allowing for EII codes over any field, and in particular, over the binary field $GF(2)$ . The traditional definition of II and EII codes is shown to be a special case of the new definition. Improvements over previous constructions of LRC codes, in particular, for binary codes, are given, as well as cases meeting an upper bound on the minimum distance. Properties of the codes are presented as well, in particular, an iterative decoding algorithm on rows and columns generalizing the iterative decoding algorithm of product codes. Two applications are also discussed: one is finding a systematic encoding of EII codes such that the parity symbols have a balanced distribution on rows, and the other is the problem of ordering the symbols of an EII code such that the maximum length of a correctable burst is achieved.