Low-density Parity-check Ensembles Research Articles

Role of the Product λ′(0)ρ′(1) in Determining LDPC Code Performance

The objective of this work is to analyze the importance of the product λ ′ ( 0 ) ρ ′ ( 1 ) in determining low density parity check (LDPC) code performance, as far as its influence on the weight distribution function and on the decoding thresholds. This analysis is based on the 2006 paper by Di et al., as far as the weight distribution function is concerned, and on the 2018 paper by Vatta et al., regarding the LDPC decoding thresholds. In particular, the first paper Di et al. analyzed the relation between the above mentioned product and the minimum weight of an ensemble of random LDPC codewords, whereas in the second some analytical upper bounds to the LDPC decoding thresholds were determined. In the present work, besides analyzing the performance of an ensemble of LDPC codes through the outcomes of Di et al.’s 2006 paper, we give the relation between one of the upper bounds found in Vatta et al.’s 2018 paper and the above mentioned product λ ′ ( 0 ) ρ ′ ( 1 ) , thus showing its role in also determining an upper bound to LDPC decoding thresholds.

LDPC Codes Over the $q$ -ary Multi-Bit Channel

In this paper, we introduce a new channel model termed as the $q$ -ary multi-bit channel. This channel models a memory device, where $q$ -ary symbols ( $q=2^{s}$ ) are stored in the form of current/voltage levels. The symbols are read in a measurement process, which provides a symbol bit in each measurement step, starting from the most significant bit. An error event occurs when not all the symbol bits are known. To deal with such error events, we use GF( $q$ ) low-density parity-check (LDPC) codes and analyze their decoding performance. We start with iterative-decoding threshold analysis and derive optimal edge-label distributions for maximizing the decoding threshold. We later move to a finite-length iterative-decoding analysis and propose an edge-labeling algorithm for the improved decoding performance. We then provide a finite-length maximum-likelihood decoding analysis for both the standard non-binary random ensemble and LDPC ensembles. Finally, we demonstrate by simulations that the proposed edge-labeling algorithm improves the finite-length decoding performance by orders of magnitude.

Finite-Length Analysis of Spatially-Coupled Regular LDPC Ensembles on Burst-Erasure Channels

Regular spatially-coupled low-density parity-check ensembles have gained significant interest, since they were shown to universally achieve the capacity of binary memoryless channels under low-complexity belief-propagation decoding. In this paper, we focus primarily on the performance of these ensembles over binary channels affected by bursts of erasures. We first develop an analysis of the finite length performance for a single burst per code word and no errors otherwise. We first assume that the burst erases a complete spatial position, modeling for instance node failures in distributed storage. We provide new tight lower bounds for the block erasure probability (PB) at finite block length and bounds on the coupling parameter for being asymptotically able to recover the burst. We further show that expurgating the ensemble can improve the block erasure probability by several orders of magnitude. Later we extend our methodology to more general channel models. In a first extension, we consider bursts that can start at a random location in the code word and span across multiple spatial positions. Besides the finite length analysis, we determine by means of density evolution the maximum correctable burst length. In a second extension, we consider the case where in addition to a single burst, random bit erasures may occur. Finally, we consider a block erasure channel model which erases each spatial position independently with some probability p, potentially introducing multiple bursts simultaneously. All results are verified using Monte-Carlo simulations.

Spatial Coupling as a Proof Technique and Three Applications

The aim of this paper is to show that spatial coupling can be viewed not only as a means to build better graphical models, but also as a tool to better understand uncoupled models. The starting point is the observation that some asymptotic properties of graphical models are easier to prove in the case of spatial coupling. In such cases, one can then use the so-called interpolation method to transfer known results for the spatially coupled case to the uncoupled one. Our main use of this framework is for Low-density parity check (LDPC) codes, where we use interpolation to show that the average entropy of the codeword conditioned on the observation is asymptotically the same for spatially coupled as for uncoupled ensembles. We give three applications of this result for a large class of LDPC ensembles. The first one is a proof of the so-called Maxwell construction stating that the MAP threshold is equal to the area threshold of the BP GEXIT curve. The second is a proof of the equality between the BP and MAP GEXIT curves above the MAP threshold. The third application is the intimately related fact that the replica symmetric formula for the conditional entropy in the infinite block length limit is exact.

Bounds on the Belief Propagation Threshold of Non-Binary LDPC Codes

We consider low-density parity-check (LDPC) code ensembles over non-binary Galois fields when used for transmission over arbitrary discrete memoryless channels. Belief propagation decoding for these codes has been shown to achieve excellent results. However, computing the decoding threshold using density evolution is usually impractical, since one needs to propagate multi-dimensional probability distributions, and Monte Carlo simulations are required instead. By considering the evolution of the message Bhattacharyya parameter and the message expected value parameter, we derive a simple lower bound on the performance of the algorithm. This bound applies for both regular and irregular non-binary LDPC ensembles.

New Nonasymptotic Channel Coding Theorems for Structured Codes

New nonasymptotic random coding theorems (with error probability $\epsilon $ and finite block length $n$ ) based on Gallager parity check ensemble and general parity check ensembles are derived in this paper. The resulting nonasymptotic achievability bounds, when combined with nonasymptotic equipartition properties developed in this paper, can be easily computed. Analytically, these nonasymptotic achievability bounds are shown to be asymptotically tight up to the second order of the coding rate as $n$ goes to infinity with either constant or subexponentially decreasing $\epsilon $ in the case of Gallager parity check ensemble, and to imply that low density parity check (LDPC) codes be capacity-achieving in the case of LDPC ensembles. Numerically, they are also compared favorably, for finite $n$ and $\epsilon $ of practical interest, with existing nonasymptotic achievability bounds in the literature.

Asymptotic bounds on the decoding error probability for two ensembles of LDPC codes

Two ensembles of low-density parity-check (LDPC) codes with low-complexity decoding algorithms are considered. The first ensemble consists of generalized LDPC codes, and the second consists of concatenated codes with an outer LDPC code. Error exponent lower bounds for these ensembles under the corresponding low-complexity decoding algorithms are compared. A modification of the decoding algorithm of a generalized LDPC code with a special construction is proposed. The error exponent lower bound for the modified decoding algorithm is obtained. Finally, numerical results for the considered error exponent lower bounds are presented and analyzed.

Variable Length Lossy Coding Using an LDPC Code

In this paper, a new variable length coding scheme using a low density parity check (LDPC) code is proposed for the lossy compression of general i.i.d. finite sources. It is proved that the proposed scheme achieves the rate-distortion function asymptotically for an LDPC ensemble. For our setting, Miyake-Muramatsu already proposed an asymptotically optimal LDPC coding scheme. In their scheme, a source sequence is first vector-quantized using an LDPC matrix and then it is compressed losslessly by fixed length coding with another LDPC matrix. However, it is not shown whether their scheme can attain good performance practically. This is mainly because of the difficulty of lossless coding by a fixed length code. In the proposed scheme, the lossless compression is performed by arithmetic coding instead of the fixed length code. Combined with vector-quantization using the reinforced belief propagation, the proposed scheme attains performance near the rate-distortion function practically with time complexity roughly equal to <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">${\bf O}(n\log n)$</tex></formula> for length <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n$</tex></formula> source sequences.

Bounds on the Error Probability of Block Codes over the q-Ary Erasure Channel

In this paper, tight bounds on the block error probability of linear block codes over order-q finite fields for the q-ary erasure channel, under maximum-likelihood (ML) decoding, are developed. Upper bounds are obtained for uniform parity-check ensembles, sparse parity-check ensembles, general parity-check ensembles (e.g., Gallager regular nonbinary low-density parity-check ensembles), and for any given linear code with known distance spectrum. The tightness of the upper bounds is confirmed both by the comparison with simple lower bounds and, for Gallager low-density parity-check ensembles, by extensive Monte Carlo simulations. Exploiting the derived bounds, it is shown how already for short blocks and small q>2 sparse ensembles attain block error probabilities close to those of idealized maximum distance separable (MDS) codes, down to low error probabilities, whereas in the same regime binary codes show visible losses with respect to the Singleton bound. Thanks to the accurate performance estimates, the developed bounds can support the design of near-optimum erasure correcting codes with short and moderate lengths.

Performance Analysis of Iterative Decoding Algorithms with Memory over Memoryless Channels

Density evolution is often used to determine the performance of an ensemble of low-density parity-check (LDPC) codes under iterative message-passing algorithms. Conventional density evolution techniques over memoryless channels are based on the assumption that messages at iteration l are only a function of the messages at iteration l -1 and possibly the channel output. This assumption is valid for many algorithms such as standard belief propagation (BP) and min-sum (MS) algorithms. However, there are other important iterative algorithms such as successive relaxation (SR) versions of BP and MS, and differential decoding with binary message passing (DD-BMP) algorithm of Mobini et al., for which this assumption is not valid. The reason is the introduction of memory in these algorithms. In this work, we propose a model for iterative decoding algorithms with memory which covers SR and DD-BMP algorithms as special cases. Based on this model, we derive a Bayesian network for iterative algorithms with memory over memoryless channels and use this representation to analyze the performance of the algorithms using density evolution. The density evolution technique is developed based on truncating the memory of the decoding process and approximating it with a finite order Markov process, and can be implemented efficiently. As an example, we apply our technique to analyze the performance of DD-BMP on regular LDPC code ensembles, and make a number of interesting observations with regard to the performance/complexity tradeoff of DD-BMP in comparison with BP and MS algorithms. The model presented in this paper is based on certain simplifying assumptions about the memory structure of iterative algorithms such as the existence of memory only at the output of variable nodes in the code's Tanner graph rather than at both outputs of variable and check nodes. The Bayesian network framework introduced here however, can still be used to analyze the more general scenarios.

Threshold Saturation via Spatial Coupling: Why Convolutional LDPC Ensembles Perform So Well over the BEC

Convolutional low-density parity-check (LDPC) ensembles, introduced by Felström and Zigangirov, have excellent thresholds and these thresholds are rapidly increasing functions of the average degree. Several variations on the basic theme have been proposed to date, all of which share the good performance characteristics of convolutional LDPC ensembles. We describe the fundamental mechanism that explains why “convolutional-like” or “spatially coupled” codes perform so well. In essence, the spatial coupling of individual codes increases the belief-propagation (BP) threshold of the new ensemble to its maximum possible value, namely the maximum a posteriori (MAP) threshold of the underlying ensemble. For this reason, we call this phenomenon “threshold saturation.” This gives an entirely new way of approaching capacity. One significant advantage of this construction is that one can create capacity-approaching ensembles with an error correcting radius that is increasing in the blocklength. Although we prove the “threshold saturation” only for a specific ensemble and for the binary erasure channel (BEC), empirically the phenomenon occurs for a wide class of ensembles and channels. More generally, we conjecture that for a large range of graphical systems a similar saturation of the “dynamical” threshold occurs once individual components are coupled sufficiently strongly. This might give rise to improved algorithms and new techniques for analysis.

Finite-Length Scaling for Iteratively Decoded LDPC Ensembles

We investigate the behavior of iteratively decoded low-density parity-check (LDPC) codes over the binary erasure channel in the so-called ldquowaterfall region.rdquo We show that the performance curves in this region follow a simple scaling law. We conjecture that essentially the same scaling behavior applies in a much more general setting and we provide some empirical evidence to support this conjecture. The scaling law, together with the error floor expressions developed previously, can be used for a fast finite-length optimization.

A Combined Matrix Ensemble of Low-Density Parity-Check Codes for Correcting a Solid Burst Erasure

A new ensemble of low-density parity-check (LDPC) codes for correcting a solid burst erasure is proposed. This ensemble is an instance of a combined matrix ensemble obtained by concatenating some LDPC matrices. We derive a new bound on the critical minimum span ratio of stopping sets for the proposed code ensemble by modifying the bound for ordinary code ensemble. By calculating this bound, we show that the critical minimum span ratio of stopping sets for the proposed code ensemble is better than that of the conventional one with keeping the same critical exponent of stopping ratio for both ensemble. Furthermore from experimental results, we show that the average minimum span of stopping sets for a solid burst erasure of the proposed codes is larger than that of the conventional ones.

On the error-correcting capability of LDPC codes

We consider the ensemble of low-density parity-check (LDPC) codes introduced by Gallager [1]. The Zyablov-Pinsker majority-logic iterative algorithm [2] for decoding LDPC codes is analyzed on the binary symmetric channel. An analytical lower bound on the error-correcting capability ? max that grows linearly in the code block length is obtained.

Sharp Bounds for Optimal Decoding of Low-Density Parity-Check Codes

Consider communication over a binary-input memoryless output-symmetric channel with low density parity check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic formula for the average input-output conditional entropy per bit in the infinite block length limit. Montanari proved a lower bound for this entropy, in the case of LDPC ensembles with convex check degree polynomial, which matches the replica formula. Here we extend this lower bound to any irregular LDPC ensemble. The new feature of our work is an analysis of the second derivative of the conditional input-output entropy with respect to noise. A close relation arises between this second derivative and correlation or mutual information of codebits. This allows us to extend the realm of the interpolation method, in particular we show how channel symmetry allows to control the fluctuations of the overlap parameters.

Constrained Information Combining: Theory and Applications for LDPC Coded Systems

This paper tightens previous information combining bounds on the performance of iterative decoding of binary low-density parity-check (LDPC) codes over binary-input symmetric-output channels by tracking the probability of erroneous bit in conjunction with mutual information. Evaluation of the new bounds as well as of other known bounds on different LDPC ensembles demonstrates sensitivity of the finite dimensional iterative bounds to lambda <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> , the fraction of edges connected to degree 2 variable nodes

Unequal Error Protection Using Partially Regular LDPC Codes

In this paper, we propose a scheme to construct low-density parity-check (LDPC) codes that are suitable for unequal error protection (UEP). We derive density evolution (DE) formulas for the proposed unequal error protecting LDPC ensembles over the binary erasure channel (BEC). Using the DE formulas, we optimize the codes. For the finite-length cases, we compare our codes with some other LDPC codes, the time-sharing method, and a previous work on UEP using LDPC codes. Simulation results indicate the superiority of the proposed design methodology for UEP

Results on Punctured Low-Density Parity-Check Codes and Improved Iterative Decoding Techniques

This paper first introduces an improved decoding algorithm for low-density parity-check (LDPC) codes over binary-input-output-symmetric memoryless channels. Then some fundamental properties of punctured LDPC codes are presented. It is proved that for any ensemble of LDPC codes, there exists a puncturing threshold. It is then proved that for any rates R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> and R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> satisfying 0<R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> <R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> <1, there exists an ensemble of LDPC codes with the following property. The ensemble can be punctured from rate R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> to R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> resulting in asymptotically good codes for all rates R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> lesRlesR <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> . Specifically, this implies that rates arbitrarily close to one are achievable via puncturing. Bounds on the performance of punctured LDPC codes are also presented. It is also shown that punctured LDPC codes are as good as ordinary LDPC codes. For BEC and arbitrary positive numbers R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> <R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> <1, the existence of the sequences of punctured LDPC codes that are capacity-achieving for all rates R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1 </sub> lesRlesR <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> is shown. Based on the above observations, a method is proposed to design good punctured LDPC codes over a broad range of rates. Finally, it is shown that the results of this paper may be used for the proof of the existence of the capacity-achieving LDPC codes over binary-input-output-symmetric memoryless channels

On Achievable Rates and Complexity of LDPC Codes Over Parallel Channels: Bounds and Applications

A variety of communication scenarios can be modeled by a set of parallel channels. Upper bounds on the achievable rates under maximum-likelihood (ML) decoding, and lower bounds on the decoding complexity per iteration of ensembles of low-density parity-check (LDPC) codes are presented. The communication of these codes is assumed to take place over statistically independent parallel channels where the component channels are memoryless, binary-input, and output-symmetric. The bounds are applied to ensembles of punctured LDPC codes where the puncturing patterns are either random or possess some structure. Our discussion is concluded by a diagram showing interconnections between the new theorems and some previously reported results

Does the Performance of LDPC Codes Depend on the Channel?

In this letter, we discuss the performance of low-density parity-check (LDPC) codes on memoryless channels. Using a recently proposed analysis technique based on extrinsic information transfer (EXIT) charts, we present an interpretation of the known fact that the bit-error rate (BER) performance of an ensemble of LDPC codes shows little dependence on the specific memoryless channel. This result has been partially observed in the literature for symmetric channels and is here extended to asymmetric channels. We conjecture and demonstrate that the performance of an ensemble of LDPC codes depends primarily and solely on the mutual information (MI) between the input and the output of the channel. As a validation of this conjecture, we compare the performance of a few LDPC codes with various rates for five representative memoryless (both symmetric and asymmetric) channels, obtaining results in excellent agreement with the EXIT chart-based prediction