Low-density Parity-check Code Ensembles Research Articles

Performance Analysis of Iterative Decoding Algorithms with Memory over Memoryless Channels

Density evolution is often used to determine the performance of an ensemble of low-density parity-check (LDPC) codes under iterative message-passing algorithms. Conventional density evolution techniques over memoryless channels are based on the assumption that messages at iteration l are only a function of the messages at iteration l -1 and possibly the channel output. This assumption is valid for many algorithms such as standard belief propagation (BP) and min-sum (MS) algorithms. However, there are other important iterative algorithms such as successive relaxation (SR) versions of BP and MS, and differential decoding with binary message passing (DD-BMP) algorithm of Mobini et al., for which this assumption is not valid. The reason is the introduction of memory in these algorithms. In this work, we propose a model for iterative decoding algorithms with memory which covers SR and DD-BMP algorithms as special cases. Based on this model, we derive a Bayesian network for iterative algorithms with memory over memoryless channels and use this representation to analyze the performance of the algorithms using density evolution. The density evolution technique is developed based on truncating the memory of the decoding process and approximating it with a finite order Markov process, and can be implemented efficiently. As an example, we apply our technique to analyze the performance of DD-BMP on regular LDPC code ensembles, and make a number of interesting observations with regard to the performance/complexity tradeoff of DD-BMP in comparison with BP and MS algorithms. The model presented in this paper is based on certain simplifying assumptions about the memory structure of iterative algorithms such as the existence of memory only at the output of variable nodes in the code's Tanner graph rather than at both outputs of variable and check nodes. The Bayesian network framework introduced here however, can still be used to analyze the more general scenarios.

Maximum Likelihood Erasure Decoding of LDPC Codes: Pivoting Algorithms and Code Design

This paper investigates efficient maximum-likelihood (ML) decoding of low-density parity-check (LDPC) codes over erasure channels. A set of algorithms, referred to as pivoting algorithms, is developed. The aim is to limit the average number of pivots (or reference variables) from which all the other erased symbols are recovered iteratively. The suggested algorithms exhibit different trade-offs between complexity of the pivoting phase and average number of pivots. Moreover, a systematic procedure to design LDPC code ensembles for efficient ML decoding is proposed. Numerical results illustrate that the designed LDPC codes achieve a near-optimum performance (very close to the Singleton bound, at least down to a codeword error rate level 10-8) with an affordable decoding complexity. For one of the presented codes and algorithms, a software implementation has been developed which is capable to provide data rates above 1.5 Gbps on a commercial computing platform.

Optimal rate irregular low-density parity-check codes in binary erasure channel

In this study the authors design the optimal rate capacity approaching irregular low-density parity-check code ensemble over binary erasure channel, by using practical semi-definite programming approach. The method does not use any relaxation or any approximate solution unlike previous works. The simulation results include two parts. First, we present some codes and their degree distribution functions that their rates are close to the capacity. Second, the maximum achievable rate behaviour of codes in our method is illustrated through some figures.

Low-density parity-check codes based on Galois fields

Methods for constructing a mapping of the elements of a multiplicative group of a Galois field onto a symmetric group of permutation matrices are proposed. A technique minimizing the order of the symmetric group is suggested. The results are used for constructing an ensemble of low-density parity-check codes. The obtained code constructions are tested on an iterative belief propagation (sum-product) decoding algorithm on transmission of a code word through a binary channel with an additive Gaussian white noise.

Secret-Sharing LDPC Codes for the BPSK-Constrained Gaussian Wiretap Channel

The problem of secret sharing over the Gaussian wiretap channel is considered. A source and a destination intend to share secret information over a Gaussian channel in the presence of a wiretapper who observes the transmission through another Gaussian channel. Two constraints are imposed on the source-to-destination channel; namely, the source can transmit only binary phase shift keyed (BPSK) symbols, and symbol-by-symbol hard-decision quantization is applied to the received symbols of the destination. An error-free public channel is also available for the source and destination to exchange messages in order to help the secret sharing process. The wiretapper can perfectly observe all messages in the public channel. It is shown that a secret sharing scheme that employs a random ensemble of regular low density parity check (LDPC) codes can achieve the key capacity of the BPSK-constrained Gaussian wiretap channel asymptotically with increasing block length. To accommodate practical constraints of finite block length and limited decoding complexity, fixed irregular LDPC codes are also designed to replace the regular LDPC code ensemble in the proposed secret sharing scheme.

On Universal LDPC Code Ensembles Over Memoryless Symmetric Channels

A design of robust error-correcting codes that achieve reliable communication over various channels is of great theoretical and practical interest. Such codes are termed universal. This paper considers the universality of low-density parity-check (LDPC) code ensembles over families of memoryless binary-input output-symmetric (MBIOS) channels. Universality is considered both under belief-propagation (BP) and maximum-likelihood (ML) decoding. For the BP decoding case, we derive a density-evolution-based analytical method for designing LDPC code ensembles that are universal over various families of MBIOS channels. We also derive a necessary condition for universality of LDPC code ensembles under BP decoding; this condition is used to provide bounds on the universally achievable fraction of capacity. These results enable us to provide conditions for reliable/unreliable communications under BP decoding that are based on the Bhattacharyya parameter of the channel. For the ML decoding case, we prove that properly selected regular LDPC code ensembles are universally capacity-achieving for the set of equi-capacity MBIOS channels and extend this result to punctured regular LDPC code ensembles.

Weight Distributions of Regular Low-Density Parity-Check Codes Over Finite Fields

The average weight distribution of a regular low-density parity-check (LDPC) code ensemble over a finite field is thoroughly analyzed. In particular, a precise asymptotic approximation of the average weight distribution is derived for the small-weight case, and a series of fundamental qualitative properties of the asymptotic growth rate of the average weight distribution are proved. Based on this analysis, a general result, including all previous results as special cases, is established for the minimum distance of individual codes in a regular LDPC code ensemble.

Analysis of Error Floors of Non-binary LDPC Codes over MBIOS Channel

In this paper, we investigate the error floors of non-binary low-density parity-check (LDPC) codes transmitted over the memoryless binary-input output-symmetric (MBIOS) channels. We provide a necessary and sufficient condition for successful decoding of zigzag cycle codes over the MBIOS channel by the belief propagation decoder. We consider an expurgated ensemble of non-binary LDPC codes by using the above necessary and sufficient condition, and hence exhibit lower error floors. Finally, we show lower bounds of the error floors for the expurgated LDPC code ensembles over the MBIOS channel.

Analysis of Stopping Constellation Distribution for Irregular Non-binary LDPC Code Ensemble

The fixed points of the belief propagation decoder for non-binary low-density parity-check (LDPC) codes are referred to as stopping constellations. In this paper, we give the stopping constellation distributions for the irregular non-binary LDPC code ensembles defined over the general linear group. Moreover, we derive the exponential growth rate of the average number of the stopping constellation distributions in the limit of large code length.

New Sequences of Capacity Achieving LDPC Code Ensembles Over the Binary Erasure Channel

In this paper, new sequences <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$(\lambda ^{n},\rho ^{n})$</tex> </formula> of capacity achieving low-density parity-check (LDPC) code ensembles over the binary erasure channel (BEC) is introduced. These sequences include the existing sequences by Shokrollahi <etal xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"/> as a special case. For a fixed code rate <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$R$</tex></formula> , in the set of proposed sequences, Shokrollahi's sequences are superior to the rest of the set in that for any given value of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$n$</tex></formula> , their threshold is closer to the capacity upper bound <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$1- R$</tex></formula> . For any given <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$\delta $</tex></formula> , <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$0 &lt; \delta &lt; 1-R$</tex></formula> , however, there are infinitely many sequences in the set that are superior to Shokrollahi's sequences in that for each of them, there exists an integer number <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n_{0}$</tex> </formula> , such that for any <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$n &gt; n_{0}$</tex></formula> , the sequence <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$(\lambda ^{n},\rho ^{n})$</tex></formula> requires a smaller maximum variable node degree as well as a smaller number of constituent variable node degrees to achieve a threshold within <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$\delta $</tex></formula> -neighborhood of the capacity upper bound <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$1-R$</tex></formula> . Moreover, it is proven that the check-regular subset of the proposed sequences are asymptotically quasi-optimal, i.e., their decoding complexity increases only logarithmically with the relative increase of the threshold. A stronger result on asymptotic optimality of some of the proposed sequences is also established.

Performance Bounds for Erasure, List, and Decision Feedback Schemes With Linear Block Codes

A message independence property and some new performance upper bounds are derived in this work for erasure, list, and decision-feedback schemes with linear block codes transmitted over memoryless symmetric channels. Similar to the classical work of Forney, this work is focused on the derivation of some Gallager-type bounds on the achievable tradeoffs for these coding schemes, where the main novelty is the suitability of the bounds for both random and structured linear block codes (or code ensembles). The bounds are applicable to finite-length codes and to the asymptotic case of infinite block length, and they are applied to low-density parity-check code ensembles.

On the design of LDPC code ensembles for BIAWGN channels

Existing design methods for irregular Low-Density Parity-Check (LDPC) codes over the additive white Gaussian noise channel are based on using asymptotic analysis tools such as density evolution in an optimization process. Such a process is computationally expensive particularly when a large number of constituent variable node degrees are involved in the design. In this paper, we propose a systematic approach for the design of irregular LDPC codes. The proposed method, which is based on a pre-computed upper bound on the fraction of edges connected to variable nodes of degree 3, is considerably less complex than the conventional optimization approach. Through a number of examples, we demonstrate that using our method, ensembles with performance very close to those devised based on optimization, can be designed. In addition to having very good performance, the number of constituent variable node degrees in the designed ensembles is only three or four. This, in some cases, is much smaller than the corresponding number for optimization-based designs with similar performance.

Systematic design of low-density parity-check code ensembles for binary erasure channels

We propose a systematic method to design irregular low-density parity-check (LDPC) codes for binary erasure channels (BEC). Compared to the existing methods, which are based on the application of asymptotic analysis tools such as density evolution or Extrinsic Information Transfer (EXIT) charts in an optimization process, the proposed method is much simpler and faster. Through a number of examples, we demonstrate that the codes designed by the proposed method perform very closely to the best codes designed by optimization. An important property of the proposed designs is the flexibility to select the number of constituent variable node degrees P. The proposed designs include existing systematic designs as a special case with P = N - 1, where N is the maximum variable node degree. Compared to the existing systematic designs, for a given rate and a given ? > 0, the designed ensembles can have a threshold in ?-neighborhood of the capacity upper bound with smaller values of P and N. They can also achieve the capacity of the BEC as N, and correspondingly P and the maximum check node degree tend to infinity.

Decoding and Design of LDPC Codes for High-Order Modulations

A concatenated code model is proposed for high-order low-density parity-check (LDPC) coded modulations. A corresponding concatenated-code belief propagation (CCBP) decoding algorithm is derived for our proposed concatenated code. Moreover, the design of LDPC codes under the CCBP decoding is developed using extrinsic information transfer (EXIT) charts. Compared with other algorithms, the CCBP method provides an excellent parallel decoding process, and the EXIT-based design method offers highly accurate LDPC code ensembles. Simulation results show that the performance of the proposed CCBP algorithm is superior to that of the conventional belief propagation decoding within a wide range of modulation orders, and the EXIT-based method can design capacity-approaching LDPC codes for high-order modulations.

On Universal Properties of Capacity-Approaching LDPC Code Ensembles

This paper is focused on the derivation of some universal properties of capacity-approaching low-density parity-check (LDPC) code ensembles whose transmission takes place over memoryless binary-input output-symmetric (MBIOS) channels. Properties of the degree distributions, graphical complexity and the number of fundamental cycles in the bipartite graphs are considered via the derivation of information-theoretic bounds. These bounds are expressed in terms of the target block/ bit error probability and the gap (in rate) to capacity. Most of the bounds are general for any decoding algorithm, and some others are proved under belief propagation (BP) decoding. Proving these bounds under a certain decoding algorithm, validates them automatically also under any sub-optimal decoding algorithm. A proper modification of these bounds makes them universal for the set of all MBIOS channels which exhibit a given capacity. Bounds on the degree distributions and graphical complexity apply to finite-length LDPC codes and to the asymptotic case of an infinite block length. The bounds are compared with capacity-approaching LDPC code ensembles under BP decoding, and they are shown to be informative and are easy to calculate. Finally, some interesting open problems are considered.

A Combined Matrix Ensemble of Low-Density Parity-Check Codes for Correcting a Solid Burst Erasure

A new ensemble of low-density parity-check (LDPC) codes for correcting a solid burst erasure is proposed. This ensemble is an instance of a combined matrix ensemble obtained by concatenating some LDPC matrices. We derive a new bound on the critical minimum span ratio of stopping sets for the proposed code ensemble by modifying the bound for ordinary code ensemble. By calculating this bound, we show that the critical minimum span ratio of stopping sets for the proposed code ensemble is better than that of the conventional one with keeping the same critical exponent of stopping ratio for both ensemble. Furthermore from experimental results, we show that the average minimum span of stopping sets for a solid burst erasure of the proposed codes is larger than that of the conventional ones.

Probabilistic Analysis of Linear Programming Decoding

We initiate the probabilistic analysis of linear programming (LP) decoding of low-density parity-check (LDPC) codes. Specifically, we show that for a random LDPC code ensemble, the linear programming decoder of Feldman et al. succeeds in correcting a constant fraction of errors with high probability. The fraction of correctable errors guaranteed by our analysis surpasses previous non-asymptotic results for LDPC codes, and in particular exceeds the best previous finite-length result on LP decoding by a factor greater than ten. This improvement stems in part from our analysis of probabilistic bit-flipping channels, as opposed to adversarial channels. At the core of our analysis is a novel combinatorial characterization of LP decoding success, based on the notion of a generalized matching. An interesting by-product of our analysis is to establish the existence of ``probabilistic expansion'' in random bipartite graphs, in which one requires only that almost every (as opposed to every) set of a certain size expands, for sets much larger than in the classical worst-case setting.

Efficient Serial Message-Passing Schedules for LDPC Decoding

Conventionally, in each low-density parity-check (LDPC) decoding iteration all the variable nodes and subsequently all the check nodes send messages to their neighbors (flooding schedule). An alternative, more efficient, approach is to update the nodes' messages serially (serial schedule). A theoretical analysis of serial message passing decoding schedules is presented. In particular, the evolution of the computation tree under serial scheduling is analyzed. It shows that the tree grows twice as fast in comparison to the flooding schedule's one, indicating that the serial schedule propagates information twice as fast in the code's underlying graph. Furthermore, an asymptotic analysis of the serial schedule's convergence rate is done using the density evolution (DE) algorithm. Applied to various ensembles of LDPC codes, it shows that for long codes the serial schedule is expected to converge in half the number of iterations compared to the standard flooding schedule, when working near the ensemble's threshold. This observation is generally proved for the binary erasure channel (BEC) under some natural assumptions. Finally, an accompanying concentration theorem is proved.

Performance of Standard Irregular LDPC Codes under Maximum Likelihood Decoding

In this paper, we derive an upper bound for the average block error probability of a standard irregular low-density parity-check (LDPC) code ensemble under the maximum-likelihood (ML) decoding. Moreover, we show that the upper bound asymptotically decreases polynomially with the code length. Furthermore, when we consider several regular LDPC code ensembles as special cases of standard irregular ones over an additive white Gaussian noise channel, we numerically show that the signal-to-noise ratio (SNR) thresholds at which the proposed bound converges to zero as the code length tends to infinity are smaller than those for a bound provided by Miller et al. We also give an example of a standard irregular LDPC code ensemble which has a lower SNR threshold than a given regular LDPC code ensemble.

Approximately Lower Triangular Ensembles of LDPC Codes With Linear Encoding Complexity

The complexity of brute-force encoding of low-density parity-check (LDPC) codes is proportional to the square value of the block length. Richardson and Urbanke have proposed efficient encoding algorithms for LDPC codes. These algorithms permute the parity-check matrix of the code iteratively, such that it becomes approximately lower triangular. We propose a new approach for efficient encoding of LDPC codes in which we modify the code ensemble to force an approximate lower triangular structure, thus eliminating the need to apply the algorithms of Richardson and Urbanke in this ensemble. We prove that the new ensemble has the same asymptotic threshold as the corresponding standard ensemble. The new ensemble can be used for linear time encoding of an arbitrary code profile. Computer simulations confirm that the performances of the standard and new ensembles are also very similar when using finite length codes