Memoryless Binary-input Output-symmetric Research Articles

On Decoding Irregular Tanner Codes With Local-Optimality Guarantees

We consider decoding of binary linear Tanner codes using message-passing iterative decoding and linear-programming (LP) decoding in memoryless binary-input output-symmetric (MBIOS) channels. We present new certificates that are based on a combinatorial characterization for the local optimality of a codeword in irregular Tanner codes with respect to any MBIOS channel. This characterization is a generalization of (Arora , Proc. ACM Symp. Theory of Computing, 2009) and (Vontobel, Proc. Inf. Theory and Appl. Workshop, 2010) and is based on a conical combination of normalized weighted subtrees in the computation trees of the Tanner graph. These subtrees may have any finite height h (even equal or greater than half of the girth of the Tanner graph). In addition, the degrees of local-code nodes in these subtrees are not restricted to two (i.e., these subtrees are not restricted to skinny trees). We prove that local optimality in this new characterization implies maximum-likelihood (ML) optimality and LP optimality, and show that a certificate can be computed efficiently. We also present a new message-passing iterative decoding algorithm, called normalized weighted min-sum (NWMS). NWMS decoding is a belief-propagation (BP) type algorithm that applies to any irregular binary Tanner code with single parity-check local codes (e.g., low-density and high-density parity-check codes). We prove that if a locally optimal codeword with respect to height parameter h exists (whereby notably h is not limited by the girth of the Tanner graph), then NWMS decoding finds this codeword in h iterations. The decoding guarantee of the NWMS decoding algorithm applies whenever there exists a locally optimal codeword. Because local optimality of a codeword implies that it is the unique ML codeword, the decoding guarantee also provides an ML certificate for this codeword. Finally, we apply the new local-optimality characterization to regular Tanner codes, and prove lower bounds on the noise thresholds of LP decoding in MBIOS channels. When the noise is below these lower bounds, the probability that LP decoding fails to decode the transmitted codeword decays doubly exponentially in the girth of the Tanner graph.

Error bounds for repeat-accumulate codes decoded via linear programming

We examine regular and irregular repeat-accumulate (RA) codes with repetition degrees which are all even. For these codes and with a particular choice of an interleaver, we give an upper bound on the decoding error probability of a linear-programming based decoder which is an inverse polynomial in the block length. Our bound is valid for any memoryless, binary-input, output-symmetric (MBIOS) channel. This result generalizes the bound derived by Feldman et al., which was for regular RA($2$) codes.

On Universal LDPC Code Ensembles Over Memoryless Symmetric Channels

A design of robust error-correcting codes that achieve reliable communication over various channels is of great theoretical and practical interest. Such codes are termed universal. This paper considers the universality of low-density parity-check (LDPC) code ensembles over families of memoryless binary-input output-symmetric (MBIOS) channels. Universality is considered both under belief-propagation (BP) and maximum-likelihood (ML) decoding. For the BP decoding case, we derive a density-evolution-based analytical method for designing LDPC code ensembles that are universal over various families of MBIOS channels. We also derive a necessary condition for universality of LDPC code ensembles under BP decoding; this condition is used to provide bounds on the universally achievable fraction of capacity. These results enable us to provide conditions for reliable/unreliable communications under BP decoding that are based on the Bhattacharyya parameter of the channel. For the ML decoding case, we prove that properly selected regular LDPC code ensembles are universally capacity-achieving for the set of equi-capacity MBIOS channels and extend this result to punctured regular LDPC code ensembles.

Successive Maximization for Systematic Design of Universally Capacity Approaching Rate-Compatible Sequences of LDPC Code Ensembles over Binary-Input Output-Symmetric Memoryless Channels

A systematic construction of capacity achieving low-density parity-check (LDPC) code ensemble sequences over the Binary Erasure Channel (BEC) has been proposed by Saeedi et al. based on a method, here referred to as Successive Maximization (SM). In SM, the fraction of degree-i nodes are successively maximized starting from i = 2 with the constraint that the ensemble remains convergent over the channel. In this paper, we propose SM to design universally capacity approaching rate-compatible LDPC code ensemble sequences over the general class of Binary-Input Output-Symmetric Memoryless (BIOSM) channels. This is achieved by first generalizing the SM method to other BIOSM channels to design a sequence of capacity approaching ensembles called the parent sequence. The SM principle is then applied to each ensemble within the parent sequence, this time to design rate-compatible puncturing schemes. As part of our results, we extend the stability condition which was previously derived for degree-2 variable nodes to other variable node degrees as well as to the case of rate-compatible codes. Consequently, we prove that using the SM principle, one is able to design universally capacity achieving rate-compatible LDPC code ensemble sequences over the BEC. Unlike the previous results in the literature, the proposed SM approach is naturally extendable to other BIOSM channels. The performance of the rate-compatible schemes designed based on our method is comparable to those designed by optimization.

Analysis of Error Floors of Non-binary LDPC Codes over MBIOS Channel

In this paper, we investigate the error floors of non-binary low-density parity-check (LDPC) codes transmitted over the memoryless binary-input output-symmetric (MBIOS) channels. We provide a necessary and sufficient condition for successful decoding of zigzag cycle codes over the MBIOS channel by the belief propagation decoder. We consider an expurgated ensemble of non-binary LDPC codes by using the above necessary and sufficient condition, and hence exhibit lower error floors. Finally, we show lower bounds of the error floors for the expurgated LDPC code ensembles over the MBIOS channel.

Capacity-Achieving Codes With Bounded Graphical Complexity and Maximum Likelihood Decoding

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In this paper, the existence of capacity-achieving codes for memoryless binary-input output-symmetric (MBIOS) channels under maximum-likelihood (ML) decoding with bounded graphical complexity is investigated. Graphical complexity of a code is defined as the number of edges in the graphical representation of the code per information bit and is proportional to the decoding complexity per information bit per iteration under iterative decoding. Irregular repeat-accumulate (IRA) codes are studied first. Utilizing the asymptotic average weight distribution (AAWD) of these codes and invoking Divsalar's bound on the binary-input additive white Gaussian noise (BIAWGN) channel, it is shown that simple nonsystematic IRA ensembles outperform systematic IRA and regular low-density parity-check (LDPC) ensembles with the same graphical complexity, and are at most 0.124 dB away from the Shannon limit. However, a conclusive result as to whether these nonsystematic IRA codes can really achieve capacity cannot be reached. Motivated by this inconclusive result, a new family of codes is proposed, called low-density parity-check and generator matrix (LDPC-GM) codes, which are serially concatenated codes with an outer LDPC code and an inner low-density generator matrix (LDGM) code. It is shown that these codes can achieve capacity on any MBIOS channel using ML decoding and also achieve capacity on any BEC using belief propagation (BP) decoding, both with bounded graphical complexity. Moreover, it is shown that, under certain conditions, these capacity-achieving codes have linearly increasing minimum distances and achieve the asymptotic Gilbert–Varshamov bound for all rates. </para>

On Universal Properties of Capacity-Approaching LDPC Code Ensembles

This paper is focused on the derivation of some universal properties of capacity-approaching low-density parity-check (LDPC) code ensembles whose transmission takes place over memoryless binary-input output-symmetric (MBIOS) channels. Properties of the degree distributions, graphical complexity and the number of fundamental cycles in the bipartite graphs are considered via the derivation of information-theoretic bounds. These bounds are expressed in terms of the target block/ bit error probability and the gap (in rate) to capacity. Most of the bounds are general for any decoding algorithm, and some others are proved under belief propagation (BP) decoding. Proving these bounds under a certain decoding algorithm, validates them automatically also under any sub-optimal decoding algorithm. A proper modification of these bounds makes them universal for the set of all MBIOS channels which exhibit a given capacity. Bounds on the degree distributions and graphical complexity apply to finite-length LDPC codes and to the asymptotic case of an infinite block length. The bounds are compared with capacity-approaching LDPC code ensembles under BP decoding, and they are shown to be informative and are easy to calculate. Finally, some interesting open problems are considered.

The Capacity of Finite Abelian Group Codes Over Symmetric Memoryless Channels

The capacity of finite Abelian group codes over symmetric memoryless channels is determined. For certain important examples, such as m -PSK constellations over additive white Gaussian noise (AWGN) channels, with m a prime power, it is shown that this capacity coincides with the Shannon capacity; i.e., there is no loss in capacity using group codes. (This had previously been known for binary-linear codes used over binary-input output-symmetric memoryless channels.) On the other hand, a counterexample involving a three-dimensional geometrically uniform constellation is presented in which the use of Abelian group codes leads to a loss in capacity. The error exponent of the average group code is determined, and it is shown to be bounded away from the random-coding error exponent, at low rates, for finite Abelian groups not admitting Galois field structure.

Capacity Achieving LDPC Codes Through Puncturing

The performance of punctured low-definition parity-check (LDPC) codes under maximum-likelihood (ML) decoding is studied in this correspondence via deriving and analyzing their average weight distributions (AWDs) and the corresponding asymptotic growth rate of the AWDs. In particular, it is proved that capacity-achieving codes of any rate and for any memoryless binary-input output-symmetric (MBIOS) channel under ML decoding can be constructed by puncturing some original LDPC code with small enough rate. Moreover, it is shown that the gap to capacity of all the punctured codes can be the same as the original code with a small enough rate. Conditions under which puncturing results in no rate loss with asymptotically high probability are also given in the process. These results show high potential for puncturing to be used in designing capacity-achieving codes, and in rate-compatible coding under any MBIOS channel.

Incremental Redundancy Cooperative Coding for Wireless Networks: Cooperative Diversity, Coding, and Transmission Energy Gains

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> We study an <emphasis emphasistype="boldital">incremental redundancy</emphasis> (IR) cooperative coding scheme for wireless networks. To exploit the distributed spatial diversity we propose a cluster-based collaborating strategy for a quasi-static Rayleigh-fading channel model. Our scheme allows for enhancing the reliability performance of a direct communication over a single hop. The collaborative cluster consists of <emphasis><formula formulatype="inline"> <tex>$M-1$</tex></formula></emphasis> nodes between the sender and the destination. The transmitted message is encoded using a mother code which is partitioned into <emphasis><formula formulatype="inline"><tex>$M$</tex></formula></emphasis> blocks each assigned to one of <emphasis><formula formulatype="inline"><tex>$M$</tex> </formula></emphasis> transmission slots. In the first slot, the sender broadcasts its information by transmitting the first block, and its helpers attempt to decode this message. In the remaining slots, each of the next <emphasis><formula formulatype="inline"><tex>$M-1$</tex></formula></emphasis> blocks is sent either through a helper which has successfully decoded the message or directly by the sender where a dynamic schedule is based on the ACK-based feedback from the cluster. By employing powerful <emphasis emphasistype="boldital">good codes</emphasis> including turbo, low-density parity-check (LDPC), and repeat–accumulate (RA) codes, our approach illustrates the benefit of collaboration through not only a cooperation diversity gain but also a coding advantage. The basis of our error rate performance analysis is based on a derived code threshold for the Bhattacharyya distance which describes the behavior of good codes. The new simple code threshold is based on the modified Shulman–Feder bound and the relationship between the Bhattacharyya parameter and the channel capacity for an arbitrary binary-input symmetric-output memoryless channel. An average frame-error rate (FER) upper bound and its asymptotic (in signal-to-noise ratio (SNR)) version are derived as a function of the average fading channel SNRs and the code threshold. Based on the asymptotic bound, we investigate both the diversity, the coding, and the transmission energy gain in the high and moderate SNR regimes for three different scenarios: transmitter clustering, receiver clustering, and cluster hopping. We observe that the energy saving of the IR cooperative coding scheme is <emphasis emphasistype="boldital">universal</emphasis> for all good code families in the sense that the gain does not depend on the sender-to-destination distance and the code threshold. </para>

Pseudo-Codeword Analysis of Tanner Graphs From Projective and Euclidean Planes

We consider coded data transmission over a binary-input output-symmetric memoryless channel using a binary linear code. In order to understand the performance of maximum-likelihood (ML) decoding, one studies the codewords, in particular the minimal codewords, and their Hamming weights. In the context of linear programming (LP) decoding, one's attention needs to be shifted to the pseudo-codewords, in particular, to the minimal pseudo-codewords and their pseudo-weights. In this paper, we investigate some families of codes that have good properties under LP decoding, namely certain families of low-density parity-check (LDPC) codes that are derived from projective and Euclidean planes: we study the structure of their minimal pseudo-codewords and give lower bounds on their pseudo-weight. Besides this main focus, we also present some results that hold for pseudo-codewords and minimal pseudo-codewords of any Tanner graph, and we highlight how the importance of minimal pseudo-codewords under LP decoding varies depending on which binary-input output-symmetric memoryless channel is used.

Parity-Check Density Versus Performance of Binary Linear Block Codes: New Bounds and Applications

The moderate complexity of low-density parity-check (LDPC) codes under iterative decoding is attributed to the sparseness of their parity-check matrices. It is therefore of interest to consider how sparse parity-check matrices of binary linear block codes can be a function of the gap between their achievable rates and the channel capacity. This issue was addressed by Sason and Urbanke, and it is revisited in this paper. The remarkable performance of LDPC codes under practical and suboptimal decoding algorithms motivates the assessment of the inherent loss in performance which is attributed to the structure of the code or ensemble under maximum-likelihood (ML) decoding, and the additional loss which is imposed by the suboptimality of the decoder. These issues are addressed by obtaining upper bounds on the achievable rates of binary linear block codes, and lower bounds on the asymptotic density of their parity-check matrices as a function of the gap between their achievable rates and the channel capacity; these bounds are valid under ML decoding, and hence, they are valid for any suboptimal decoding algorithm. The new bounds improve on previously reported results by Burshtein and by Sason and Urbanke, and they hold for the case where the transmission takes place over an arbitrary memoryless binary-input output-symmetric (MBIOS) channel. The significance of these information-theoretic bounds is in assessing the tradeoff between the asymptotic performance of LDPC codes and their decoding complexity (per iteration) under message-passing decoding. They are also helpful in studying the potential achievable rates of ensembles of LDPC codes under optimal decoding; by comparing these thresholds with those calculated by the density evolution technique, one obtains a measure for the asymptotic suboptimality of iterative decoding algorithms

Capacity-Achieving Ensembles for the Binary Erasure Channel With Bounded Complexity

We present two sequences of ensembles of nonsystematic irregular repeat-accumulate (IRA) codes which asymptotically (as their block length tends to infinity) achieve capacity on the binary erasure channel (BEC) with bounded complexity per information bit. This is in contrast to all previous constructions of capacity-achieving sequences of ensembles whose complexity grows at least like the log of the inverse of the gap (in rate) to capacity. The new bounded complexity result is achieved by puncturing bits, and allowing in this way a sufficient number of state nodes in the Tanner graph representing the codes. We derive an information-theoretic lower bound on the decoding complexity of randomly punctured codes on graphs. The bound holds for every memoryless binary-input output-symmetric (MBIOS) channel and is refined for the binary erasure channel.

Typical performance of regular low-density parity-check codes over general symmetric channels

Typical performance of low-density parity-check (LDPC) codes over a general binary-input output-symmetric memoryless channel is investigated using methods of statistical mechanics. Theoretical framework for dealing with general symmetric channels is provided, based on which Gallager and MacKay-Neal codes are studied as examples of LDPC codes. It has been shown that the basic properties of these codes known for particular channels, including the property to potentially saturate Shannon's limit, hold for general symmetric channels. The binary-input additive-white-Gaussian-noise channel and the binary-input Laplace channel are considered as specific channel noise models.

Parity-check density versus performance of binary linear block codes over memoryless symmetric channels

We derive lower bounds on the density of parity-check matrices of binary linear codes which are used over memoryless binary-input output-symmetric (MBIOS) channels. The bounds are expressed in terms of the gap between the rate of these codes for which reliable communications is achievable and the channel capacity; they are valid for every sequence of binary linear block codes if there exists a decoding algorithm under which the average bit-error probability vanishes. For every MBIOS channel, we construct a sequence of ensembles of regular low-density parity-check (LDPC) codes, so that an upper bound on the asymptotic density of their parity-check matrices scales similarly to the lower bound. The tightness of the lower bound is demonstrated for the binary erasure channel by analyzing a sequence of ensembles of right-regular LDPC codes which was introduced by Shokrollahi, and which is known to achieve the capacity of this channel. Under iterative message-passing decoding, we show that this sequence of ensembles is asymptotically optimal (in a sense to be defined in this paper), strengthening a result of Shokrollahi. Finally, we derive lower bounds on the bit-error probability and on the gap to capacity for binary linear block codes which are represented by bipartite graphs, and study their performance limitations over MBIOS channels. The latter bounds provide a quantitative measure for the number of cycles of bipartite graphs which represent good error-correction codes.

Integer metrics for binary input symmetric output memoryless channels

Consider a coded communication system where antipodal signaling is used over symmetric memoryless channels. A mismatch decoder employing metrics which assume values in a finite set of integers is used. Two performance measures for achievable rates are considered, the generalized cutoff rate and the mismatch capacity. Two different integer metric assignments which maximize these criteria are found. The optimal metric assignment holds for any binary input symmetric output memoryless channels. It is calculated explicitly for the faded additive white Gaussian noise channel with and without diversity, an interference channel model and the additive "Cauchy" channel. The difference in terms of achievable rates between decoding using the optimal continuous metric and integer metric is calculated for these channels. The degradation for fairly low number of quantization levels is found to be small, e.g. for nine quantization levels degradation is less than 0.3 dB for all analyzed channels. The sensitivity of the generalized cutoff rate and the mismatch capacity to the exact location of the quantization levels is rather mild. The difference between the mismatch capacity corresponding to the optimal integer quantization and the mismatch capacity obtained by the quantization which optimizes the generalized cutoff rate is found to be quite small.