Symmetric Binary-input Memoryless Channels Research Articles

A Note on the Mixing Factor of Polar Codes.

Over binary-input memoryless symmetric (BMS) channels, the performance of polar codes under successive cancellation list (SCL) decoding can approach maximum likelihood (ML) algorithm when the list size L is greater than or equal to 2MF, where MF, known as mixing factor of code, represents the number of information bits before the last frozen bit. Recently, Yao et al. showed the upper bound of the mixing factor of decreasing monomial codes with length n=2m and rate R≤12 when m is an odd number; moreover, this bound is reachable. Herein, we obtain an achievable upper bound in the case of an even number. Further, we propose a new decoding hard-decision rule beyond the last frozen bit of polar codes under BMS channels.

Polar Coding for Non-Stationary Channels

The problem of polar coding for an arbitrary sequence of independent binary-input memoryless symmetric (BMS) channels $\left \{{W_{i}}\right \}_{i=1}^{N}$ is considered. Such a sequence of channels is referred to as a non-stationary sequence of channels and arises in applications where data symbols experience different and independent channel characteristics. The sequence of channels is assumed to be completely known to both the transmitter and the receiver (a coherent scenario). Also, at each code block transmission, each of the channels is used only once. In other words, a codeword of length $N$ is constructed and then the $i$ -th encoded bit is transmitted over $W_{i}$ . The goal is to operate at a rate $R$ close to the average of the symmetric capacities of $W_{i}$ ’s, denoted by $\overline {I}_{N}$ . To this end, we construct a polar coding scheme using Arikan’s channel polarization transform in combination with certain permutations at each polarization level and certain skipped operations. In particular, given a non-stationary sequence of BMS channels $\left \{{W_{i}}\right \}_{i=1}^{N}$ and $P_{e}$ , where $0 , we construct a polar code of length $N$ and rate $R$ guaranteeing a block error probability of at most $P_{e}$ for transmission over $\left \{{W_{i}}\right \}_{i=1}^{N}$ such that $N \leqslant \frac {\kappa }{(\overline {I}_{N}- R)^{\mu }}$ , where $\mu $ is a constant and $\kappa $ is a constant depending on $P_{e}$ and $\mu $ . We further show a numerical upper bound on $\mu $ that is: $\mu \leqslant 7.34$ for non-stationary binary erasure channels and $\mu \leqslant 8.54$ for general non-stationary BMS channels. The encoding and decoding complexities of the constructed polar code preserve $O(N \log N)$ complexity of Arikan’s polar codes. In an asymptotic sense, when coded bits are transmitted over a non-stationary sequence of BMS channels $\left \{{W_{i}}\right \}_{i=1}^{\infty }$ , our proposed scheme achieves the average symmetric capacity $\overline {I}(\left \{{W_{i}}\right \}_{i=1}^{\infty }) \,\,{\textstyle \stackrel {\mathrm{ def}}{=}}\,\,\lim _{N\rightarrow \infty } \frac {1}{N}\sum _{i=1}^{N} I(W_{i})$ , assuming that the limit exists.

A General Construction and Encoder Implementation of Polar Codes

Similar to existing codes, puncturing and shortening are two general ways to obtain an arbitrary code length and code rate for polar codes. When some of the coded bits are punctured or shortened, it is equivalent to a situation in which the underlying channels of the polar codes are different. Therefore, the quality of bit channels with puncturing or shortening differ from the original qualities, which can greatly affect the construction of polar codes. In this paper, a general construction of polar codes is studied in two aspects: 1) the theoretical foundation of the construction; and 2) the hardware implementation of polar codes encoders. In contrast to the original identical and independent binary-input, memoryless, symmetric (BMS) channels, these underlying BMS channels can be different. For binary erasure channel (BEC) channels, recursive equations can be employed assuming independent BMS channels. For all other channel types, the proposed general construction of polar codes is based on the existing Tal-Vardy's procedure. The symmetric property and the degradation relationship are shown to be preserved under the general setting, rendering the possibility of a modification of Tal-Vardy's procedure. Simulation results clearly show improved error performance with re-ordering using the proposed new procedures. In terms of hardware, a novel pruned folded encoder architecture is proposed which saves the computation for the beginning frozen bits. Implementation results show the pruned encoder achieve 28% throughput improvement.

Generalized Partial Orders for Polar Code Bit-Channels

We study partial orders (POs) for the synthesized bit-channels of polar codes. First, we give an alternative proof of an existing PO for bit-channels with the same Hamming weight and use the underlying idea to extend the bit-channel ordering to some additional cases. In particular, the bit-channel ordering for a given code block length is used to generate additional bit-channel ordering relationships for larger block lengths, generalizing previously known POs. Next, we consider POs especially for the binary erasure channel (BEC). We identify a symmetry property of the Bhattacharyya parameters of complementary bit-channel pairs on the BEC and provide a condition for the alignment of polarized sets of bit-channels for the BEC and general binary-input memoryless symmetric (BMS) channels. Numerical examples and further properties about the POs for the bit-channels with different Hamming weights are provided to illustrate the new POs. The bit-channels with universal ordering positions, which are independent of the channel erasure probability, are verified for all of the code block lengths. Finally, we show the threshold behavior of the Bhattacharyya parameters of some bit-channels by approximating the threshold values. The corresponding value for a bit-channel can be used to determine whether it is good or bad when the underlying channel is known.

Polar Code Design for Irregular Multidimensional Constellations

Polar codes, ever since their introduction, have been shown to be very effective for various wireless communication channels. This, together with their relatively low implementation complexity, has made them an attractive coding scheme for wireless communications. Polar codes have been extensively studied for use with binary-input symmetric memoryless channels but little is known about their effectiveness in other channels. In this paper, a novel methodology for designing multilevel polar codes that works effectively with arbitrary multidimensional constellations is presented. In order for this multilevel design to function, a novel set merging algorithm, able to label such constellations, is proposed. We then compare the error rate performance of our design with that of existing schemes and show that we were able to obtain unprecedented results in many cases over the previously known best techniques at relatively low decoding complexity.

On the Scaling Exponent of Polar Codes for Binary-Input Energy-Harvesting Channels

This paper investigates the scaling exponent of polar codes for binary-input energy-harvesting (EH) channels with infinite-capacity batteries. The EH process is characterized by a sequence of i.i.d. random variables with finite variances. The scaling exponent $\mu$ of polar codes for a binary-input memoryless channel (BMC) characterizes the closest gap between the capacity and non-asymptotic rates achieved by polar codes with error probabilities no larger than some non-vanishing $\varepsilon\in(0,1)$. It has been shown that for any $\varepsilon\in(0,1)$, the scaling exponent $\mu$ for any binary-input memoryless symmetric channel (BMSC) with $I(q_{Y|X})\in(0,1)$ lies between 3.579 and 4.714 , where the upper bound $4.714$ was shown by an explicit construction of polar codes. Our main result shows that $4.714$ remains to be a valid upper bound on the scaling exponent for any binary-input EH channel, i.e., a BMC subject to additional EH constraints. Our result thus implies that the EH constraints do not worsen the rate of convergence to capacity if polar codes are employed. The main result is proved by leveraging the following three existing results: scaling exponent analyses for BMSCs, construction of polar codes designed for binary-input memoryless asymmetric channels, and the save-and-transmit strategy for EH channels.

A Simple Proof of Maxwell Saturation for Coupled Scalar Recursions

Low-density parity-check (LDPC) convolutional codes (or spatially coupled codes) were recently shown to approach capacity on the binary erasure channel (BEC) and binary-input memoryless symmetric channels. The mechanism behind this spectacular performance is now called threshold saturation via spatial coupling. This new phenomenon is characterized by the belief-propagation threshold of the spatially coupled ensemble increasing to an intrinsic noise threshold defined by the uncoupled system. In this paper, we present a simple proof of threshold saturation that applies to a wide class of coupled scalar recursions. Our approach is based on constructing potential functions for both the coupled and uncoupled recursions. Our results actually show that the fixed point of the coupled recursion is essentially determined by the minimum of the uncoupled potential function and we refer to this phenomenon as Maxwell saturation. A variety of examples are considered including the density-evolution equations for: irregular LDPC codes on the BEC, irregular low-density generator-matrix codes on the BEC, a class of generalized LDPC codes with BCH component codes, the joint iterative decoding of LDPC codes on intersymbol-interference channels with erasure noise, and the compressed sensing of random vectors with independent identically distributed components.

On Achieving an Asymptotically Error-Free Fixed-Point of Iterative Decoding for Perfect A Priori Information

In this paper we provide necessary and sufficient conditions for constituent codes in (multiple) concatenated and graph-based coding schemes to achieve an asymptotically error-free iterative decoding fixed-point if the maximum possible a priori information is available. At least one constituent code in an iterative decoding scheme must satisfy these conditions in order to ensure an asymptotically vanishing bit error probability at the convergence point of the decoder. Our results are proved for arbitrary binary-input symmetric memoryless channels (BISMCs) and thus can be universally applied to many transmission scenarios. Specifically, using a factor graph framework, it is shown that non-inner codes in a serial concatenation or check nodes in generalized LDPC codes achieve perfect extrinsic information if and only if the minimum Hamming distance between codewords is two or greater. For inner codes in a serial concatenation, constituent codes in a parallel concatenation, or variable nodes in doubly-generalized LDPC codes the corresponding encoder condition for acquiring perfect extrinsic information is an infinite codeword weight for a weight-one input sequence. For this case we provide a general proof which holds for all linear encoders and BISMCs. We also show that these results can improve the performance of concatenated coding schemes.

Extremes of Error Exponents

This paper determines the range of feasible values of standard error exponents for binary-input memoryless symmetric channels of fixed capacity C and shows that extremes are attained by the binary symmetric and the binary erasure channel. The proof technique also provides analogous extremes for other quantities related to Gallager's E0 function, such as the cutoff rate, the Bhattacharyya parameter, and the channel dispersion.

Mathematical Programming Decoding of Binary Linear Codes: Theory and Algorithms

Mathematical programming is a branch of applied mathematics and has recently been used to derive new decoding approaches, challenging established but often heuristic algorithms based on iterative message passing. Concepts from mathematical programming used in the context of decoding include linear, integer, and nonlinear programming, network flows, notions of duality as well as matroid and polyhedral theory. This survey article reviews and categorizes decoding methods based on mathematical programming approaches for binary linear codes over binary-input memoryless symmetric channels.

Rate-Dependent Analysis of the Asymptotic Behavior of Channel Polarization

For a binary-input memoryless symmetric channel $W$, we consider the asymptotic behavior of the polarization process in the large block-length regime when transmission takes place over $W$. In particular, we study the asymptotics of the cumulative distribution $\mathbb{P}(Z_n \leq z)$, where $\{Z_n\}$ is the Bhattacharyya process defined from $W$, and its dependence on the rate of transmission. On the basis of this result, we characterize the asymptotic behavior, as well as its dependence on the rate, of the block error probability of polar codes using the successive cancellation decoder. This refines the original bounds by Ar{\i}kan and Telatar. Our results apply to general polar codes based on $\ell \times \ell$ kernel matrices. We also provide lower bounds on the block error probability of polar codes using the MAP decoder. The MAP lower bound and the successive cancellation upper bound coincide when $\ell=2$, but there is a gap for $\ell>2$.

Decay of Correlations for Sparse Graph Error Correcting Codes

The subject of this paper is transmission over a general class of binary-input memoryless symmetric channels using error correcting codes based on sparse graphs, namely, low-density generator-matrix and low-density parity-check codes. The optimal (or ideal) decoder based on the posterior measure over the code-bits and its relationship to the suboptimal belief propagation decoder are investigated. We consider the correlation (or covariance) between two code-bits, averaged over the noise realizations, as a function of the graph distance for the optimal decoder. Our main result is that this correlation decays exponentially fast for given low-density generator-matrix codes and a high enough noise parameter and also for given low-density parity-check codes and a low enough noise parameter. This has many consequences. Appropriate performance curves—called generalized extrinsic information transfer (GEXIT) functions—of the belief propagation and optimal decoders match in high/low noise regimes. This means that i...

Design of irregular LDPC codes with optimized performance-complexity tradeoff

The optimal performance-complexity tradeoff for error-correcting codes at rates strictly below the Shannon limit is a central question in coding theory. This paper proposes a numerical approach for the minimization of decoding complexity for long-block-length irregular low-density parity-check (LDPC) codes. The proposed design methodology is applicable to any binary-input memoryless symmetric channel and any iterative message-passing decoding algorithm with a parallel-update schedule. A key feature of the proposed optimization method is a new complexity measure that incorporates both the number of operations required to carry out a single decoding iteration and the number of iterations required for convergence. This paper shows that the proposed complexity measure can be accurately estimated from a density-evolution and extrinsic-information transfer chart analysis of the code. A sufficient condition is presented for convexity of the complexity measure in the variable edge-degree distribution; when it is not satisfied, numerical experiments nevertheless suggest that the local minimum is unique. The results presented herein show that when the decoding complexity is constrained, the complexity-optimized codes significantly outperform threshold-optimized codes at long block lengths, within the ensemble of irregular codes.

Performance of Polar Codes with the Construction using Density Evolution

Polar coding, proposed by Arikan, makes it possible to construct capacity-achieving codes for symmetric binary-input discrete memoryless channels, with low encoding and decoding complexity. Complexity of the originally proposed code construction method, however, grows exponentially in the block-length unless a channel is the binary erasure channel. Recently, the authors have proposed a new capacity-achieving code construction method with linear complexity in the block-length for arbitrary symmetric binary-input memoryless channels. In this letter, we evaluate performance of polar codes designed with the new construction method, and compare it with that of the codes constructed with another heuristic method with linear complexity proposed by Arikan.

Channel Polarization: A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> A method is proposed, called channel polarization, to construct code sequences that achieve the symmetric capacity <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$I(W)$</tex></formula></emphasis> of any given binary-input discrete memoryless channel (B-DMC) <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$W$</tex></formula></emphasis>. The symmetric capacity is the highest rate achievable subject to using the input letters of the channel with equal probability. Channel polarization refers to the fact that it is possible to synthesize, out of <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$N$</tex></formula></emphasis> independent copies of a given B-DMC <emphasis emphasistype="italic"><formula formulatype="inline"> <tex Notation="TeX">$W$</tex></formula></emphasis>, a second set of <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$N$</tex> </formula></emphasis> binary-input channels <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$\{W_N^{(i)}:1\le i\le N\}$</tex> </formula></emphasis> such that, as <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$N$</tex></formula></emphasis> becomes large, the fraction of indices <emphasis emphasistype="italic"><formula formulatype="inline"> <tex Notation="TeX">$i$</tex></formula></emphasis> for which <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$I(W_N^{(i)})$</tex></formula></emphasis> is near <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$1$</tex> </formula></emphasis> approaches <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$I(W)$</tex></formula></emphasis> and the fraction for which <emphasis emphasistype="italic"><formula formulatype="inline"> <tex Notation="TeX">$I(W_N^{(i)})$</tex></formula></emphasis> is near <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$0$</tex> </formula></emphasis> approaches <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$1-I(W)$</tex></formula></emphasis>. The polarized channels <emphasis emphasistype="italic"><formula formulatype="inline"> <tex Notation="TeX">$\{W_N^{(i)}\}$</tex></formula></emphasis> are well-conditioned for channel coding: one need only send data at rate <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$1$</tex></formula></emphasis> through those with capacity near <emphasis emphasistype="italic"><formula formulatype="inline"> <tex Notation="TeX">$1$</tex></formula></emphasis> and at rate <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$0$</tex></formula></emphasis> through the remaining. Codes constructed on the basis of this idea are called polar codes. The paper proves that, given any B-DMC <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$W$</tex></formula></emphasis> with <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$I(W)&gt; 0$</tex></formula></emphasis> and any target rate <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$R ≪ I(W)$</tex></formula></emphasis>, there exists a sequence of polar codes <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$\{{\Fraktur {C}}_n;n\ge 1\}$</tex> </formula></emphasis> such that <emphasis emphasistype="italic"><formula formulatype="inline"> <tex Notation="TeX">${\Fraktur {C}}_n$</tex></formula></emphasis> has block-length <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$N=2^n$</tex> </formula></emphasis>, rate <emphasis emphasistype="italic"><formula formulatype="inline"> <tex Notation="TeX">$\ge R$</tex></formula></emphasis>, and probability of block error under successive cancellation decoding bounded as <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$P_{e}(N,R) \le O(N^{-{1\over 4}})$</tex> </formula></emphasis> independently of the code rate. This performance is achievable by encoders and decoders with complexity <emphasis emphasistype="italic"><formula formulatype="inline"><tex Notation="TeX">$O(N\log N)$</tex></formula></emphasis> for each. </para>

Sharp Bounds on Generalized EXIT Functions

We consider communication over binary-input memoryless symmetric channels with low-density parity-check (LDPC) codes. The relationship between maximum a posteriori and belief propagation decoding is investigated using a set of correlation inequalities that first appeared in statistical mechanics of Gaussian spin glasses. We prove bounds on generalized extrinsic information transfer (EXIT) functions, that are believed to be tight, and discuss their relationship with the ones obtained by the interpolation method.

EXIT functions for binary input memoryless symmetric channels

Use of extrinsic information transfer (EXIT) functions, characterizing the amplification of mutual information between the input and output of the maximum a posteriori (MAP) decoder, significantly facilitates analysis of iterative coding schemes. Previously, EXIT functions derived for binary erasure channels (BECs) were used as an approximation for other channels. Here, we improve on this approach by introducing more accurate methods to construct EXIT functions for binary-input memoryless symmetric (BMS) channels. By defining an alternative pseudo-MAP decoder coinciding with the MAP decoder over BEC, we provide an expression for the EXIT functions of block codes over BEC. Furthermore, we draw a connection between the EXIT function over BEC and the EXIT function over the BMS channel under certain conditions. This is used for deriving accurate or approximate expressions of EXIT functions over BMS channels in certain scenarios

Raptor codes on binary memoryless symmetric channels

In this paper, we will investigate the performance of Raptor codes on arbitrary binary input memoryless symmetric channels (BIMSCs). In doing so, we generalize some of the results that were proved before for the erasure channel. We will generalize the stability condition to the class of Raptor codes. This generalization gives a lower bound on the fraction of output nodes of degree 2 of a Raptor code if the error probability of the belief-propagation decoder converges to zero. Using information-theoretic arguments, we will show that if a sequence of output degree distributions is to achieve the capacity of the underlying channel, then the fraction of nodes of degree 2 in these degree distributions has to converge to a certain quantity depending on the channel. For the class of erasure channels this quantity is independent of the erasure probability of the channel, but for many other classes of BIMSCs, this fraction depends on the particular channel chosen. This result has implications on the "universality" of Raptor codes for classes other than the class of erasure channels, in a sense that will be made more precise in the paper. We will also investigate the performance of specific Raptor codes which are optimized using a more exact version of the Gaussian approximation technique.

Extension of an entropy property for binary input memoryless symmetric channels

The channel output entropy property introduced by A.D. Wyner and J. Ziv (ibid., vol.IT-19, p.769-762, Nov.1973) for a binary symmetric channel is extended to arbitrary memoryless symmetric channels with binary inputs and discrete or continuous outputs. This yields lower bounds on the achievable information rates of these channels under constrained binary inputs. Using the interpretation of entropy as a measure of order and randomness, the authors deduce that output sequences of memoryless symmetric channels induced by binary inputs are of a higher degree of randomness if the redundancy of the input binary sequence is spread in memory rather than in one-dimensional asymmetry. It is of interest to characterize the general class of schemes for which this interpretation holds. >