Memoryless Symmetric Channels Research Articles

Reed–Muller Codes on BMS Channels Achieve Vanishing Bit-Error Probability for all Rates Below Capacity

This paper considers the performance of Reed-Muller (RM) codes transmitted over binary memoryless symmetric (BMS) channels under bitwise maximum-a-posteriori (bit-MAP) decoding. Its main result is that, for a fixed BMS channel, the family of binary RM codes can achieve a vanishing bit-error probability at rates approaching the channel capacity. This partially resolves a long-standing open problem that connects information theory and error-correcting codes. In contrast with the earlier result for the binary erasure channel, the new proof does not rely on hypercontractivity. Instead, it combines a nesting property of RM codes with new information inequalities relating the generalized extrinsic information transfer function and the extrinsic minimum mean-squared error.

Series of formulas for bhattacharyya parameters in the theory of polar codes

Bhattacharyya parameters are used in the theory of polar codes to determine positions of frozen and information bits. These parameters characterize rate of polarization of channels WN(i), 1 ≤ i ≤ N, which are constructed in a special way from the original channel W, where N = 2n is the channel length, n = 1, 2, .... In the case where W is a binary symmetric memoryless channel, we present two series of formulas for the parameters Z(WN(i)): for i = N - 2k + 1, 0 ≤ k ≤ n, and for i = N/2 - 2k + 1, 1 ≤ k ≤ n - 2. The formulas require of the order of $\binom{2^{n-k}+2^k-1}{2^k} 2^{2^k}$ addition operations for the first series and of the order of $\binom{2^{n-k-1}+2^k-1}{2^k} 2^{2^k}$ for the second. In the cases i = 1, N/4 + 1, N/2 + 1, N, the obtained expressions for the parameters have been simplified by computing the sums in them. We show potential generalizations for the values of i in the interval (N/4, N). We also study combinatorial properties of the polarizing matrix GN of a polar code with Arıkan’s kernel. In particular, we establish simple recurrence relations between rows of the matrices GN and GN/2.

A Note on the Mixing Factor of Polar Codes.

Over binary-input memoryless symmetric (BMS) channels, the performance of polar codes under successive cancellation list (SCL) decoding can approach maximum likelihood (ML) algorithm when the list size L is greater than or equal to 2MF, where MF, known as mixing factor of code, represents the number of information bits before the last frozen bit. Recently, Yao et al. showed the upper bound of the mixing factor of decreasing monomial codes with length n=2m and rate R≤12 when m is an odd number; moreover, this bound is reachable. Herein, we obtain an achievable upper bound in the case of an even number. Further, we propose a new decoding hard-decision rule beyond the last frozen bit of polar codes under BMS channels.

Sub-4.7 Scaling Exponent of Polar Codes

Polar codes approach channel capacity provably and empirically and are thereby a constituent code of the 5G standard. Compared to low-density parity-check codes, however, the performance of short-length polar codes have rooms for improvement that could hinder its adoption by a wider class of applications. As part of the program that addresses the performance issue at short length, it is crucial to understand how fast binary memoryless symmetric channels polarize. A number, called scaling exponent, was defined to measure the speed of polarization and several estimates of the scaling exponent were given in literature. As of 2022, the tightest overestimate is 4.714 made by Mondelli, Hassani, and Urbanke in 2015. We lower the overestimate to 4.63. The idea behind this improvement is that, instead of describing the relation between a channel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> and its children <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ◻ and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ⊛, we describe the relation between <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> and its grandchildren <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ◻◻, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ◻⊛, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ⊛◻, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ⊛⊛. By doing so, the evolution of channels becomes “less Markovian” and hence more tighter inequalities can be obtained.

Successive Cancellation Decoding of Single Parity-Check Product Codes: Analysis and Improved Decoding

A product code with single parity-check component codes can be described via the tools of a multi-kernel polar code, where the rows of the generator matrix are chosen according to the constraints imposed by the product code construction. Following this observation, successive cancellation decoding of such codes is introduced. In particular, the error probability of single parity-check product codes over binary memoryless symmetric channels under successive cancellation decoding is characterized. A bridge with the analysis of product codes introduced by Elias is also established for the binary erasure channel. Successive cancellation list decoding of single parity-check product codes is then described. For the provided example, simulations over the binary input additive white Gaussian channel show that successive cancellation list decoding outperforms belief propagation decoding applied to the code graph. Finally, the performance of the concatenation of a product code with a high-rate outer code is investigated via distance spectrum analysis. Examples of concatenations performing within 0.7 dB from the random coding union bound are provided.

An Information-Theoretic Perspective on Successive Cancellation List Decoding and Polar Code Design

This work identifies information-theoretic quantities that are closely related to the required list size on average for successive cancellation list (SCL) decoding to implement maximum-likelihood decoding over general binary memoryless symmetric (BMS) channels. It also provides upper and lower bounds for these quantities that can be computed efficiently for very long codes. For the binary erasure channel (BEC), we provide a simple method to estimate the mean accurately via density evolution. The analysis shows how to modify, e.g., Reed-Muller codes, to improve the performance when practical list sizes, e.g., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L\in {[{8, 1024}]}$ </tex-math></inline-formula> , are adopted. Exemplary constructions with block lengths <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N\in \{128,512\}$ </tex-math></inline-formula> outperform polar codes of 5G over the binary-input additive white Gaussian noise channel. It is further shown that there is a concentration around the mean of the logarithm of the required list size for sufficiently large block lengths, over discrete-output BMS channels. We provide the probability mass functions (p.m.f.s) of this logarithm, over the BEC, for a sequence of the modified RM codes with an increasing block length via simulations, which illustrate that the p.m.f.s concentrate around the estimated mean.

General Strong Polarization

Arıkan’s exciting discovery of polar codes has provided an altogether new way to efficiently achieve Shannon capacity. Given a (constant-sized) invertible matrix M , a family of polar codes can be associated with this matrix and its ability to approach capacity follows from the polarization of an associated [0, 1]-bounded martingale, namely its convergence in the limit to either 0 or 1 with probability 1. Arıkan showed appropriate polarization of the martingale associated with the matrix ( G 2 = ( 1 1 0 1) to get capacity achieving codes. His analysis was later extended to all matrices M that satisfy an obvious necessary condition for polarization. While Arıkan’s theorem does not guarantee that the codes achieve capacity at small blocklengths (specifically in length, which is a polynomial in ( 1ε ) where (ε) is the difference between the capacity of a channel and the rate of the code), it turns out that a “strong” analysis of the polarization of the underlying martingale would lead to such constructions. Indeed for the martingale associated with ( G 2 ) such a strong polarization was shown in two independent works (Guruswami and Xia (IEEE IT’15) and Hassani et al. (IEEE IT’14)), thereby resolving a major theoretical challenge associated with the efficient attainment of Shannon capacity. In this work we extend the result above to cover martingales associated with all matrices that satisfy the necessary condition for (weak) polarization. In addition to being vastly more general, our proofs of strong polarization are (in our view) also much simpler and modular. Key to our proof is a notion of local polarization that only depends on the evolution of the martingale in a single time step. We show that local polarization always implies strong polarization. We then apply relatively simple reasoning about conditional entropies to prove local polarization in very general settings. Specifically, our result shows strong polarization over all prime fields and leads to efficient capacity-achieving source codes for compressing arbitrary i.i.d. sources, and capacity-achieving channel codes for arbitrary symmetric memoryless channels. We show how to use our analyses to achieve exponentially small error probabilities at lengths inverse polynomial in the gap to capacity. Indeed we show that we can essentially match any error probability while maintaining lengths that are only inverse polynomial in the gap to capacity.

Iterative Programming of Noisy Memory Cells

In this paper, we study a model that mimics the programming operation of memory cells. This model was first introduced by Lastras-Montano <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> for continuous-alphabet channels, and later by Bunte and Lapidoth for discrete memoryless channels (DMC). Under this paradigm we assume that cells are programmed sequentially and individually. The programming process is modeled as transmission over a channel, such that it is possible to read the cell state in order to determine its programming success, and in case of programming failure, to reprogram the cell again. Reprogramming a cell can reduce the bit error rate, however this comes with the price of increasing the overall programming time and thereby affecting the writing speed of the memory. An <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iterative programming scheme</i> is an algorithm which specifies the number of attempts to program each cell. Given the programming channel and constraints on the average and maximum number of attempts to program a cell, we study programming schemes which maximize the number of bits that can be reliably stored in the memory. We extend the results by Bunte and Lapidoth and study this problem when the programming channel is either discrete-input memoryless symmetric channel (including the BSC,BEC, BI-AWGN) or the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Z$ </tex-math></inline-formula> channel. For the BSC and the BEC our analysis is also extended for the case where the error probabilities on consecutive writes are not necessarily the same. Lastly, we also study a related model which is motivated by the synthesis process of DNA molecules.

A Lower Bound on the Essential Interactive Capacity of Binary Memoryless Symmetric Channels

The essential interactive capacity of a discrete memoryless channel is defined in this paper as the maximal rate at which the transcript of any interactive protocol can be reliably simulated over the channel, using a deterministic coding scheme. In contrast to other interactive capacity definitions in the literature, this definition makes no assumptions on the order of speakers (which can be adaptive) and does not allow any use of private/public randomness; hence, the essential interactive capacity is a function of the channel model only. It is shown that the essential interactive capacity of any binary memoryless symmetric (BMS) channel is at least 0.0302 its Shannon capacity. To that end, we present a simple coding scheme, based on extended-Hamming codes combined with error detection, that achieves the lower bound in the special case of the binary symmetric channel (BSC). We then adapt the scheme to the entire family of BMS channels, and show that it achieves the same lower bound using extremes of the Bhattacharyya parameter.

Systematic Convolutional Low Density Generator Matrix Code

In this paper, we propose a systematic low density generator matrix (LDGM) code ensemble, which is defined by the Bernoulli process. We prove that, under maximum likelihood (ML) decoding, the proposed ensemble can achieve the capacity of binary-input output symmetric (BIOS) memoryless channels in terms of bit error rate (BER). The proof technique reveals a new mechanism, different from lowering down frame error rate (FER), that the BER can be lowered down by assigning light codeword vectors to light information vectors. The finite length performance is analyzed by deriving an upper bound and a lower bound, both of which are shown to be tight in the high signal-to-noise ratio (SNR) region. To improve the waterfall performance, we construct the systematic convolutional LDGM (SysConv-LDGM) codes by a random splitting process. The SysConv-LDGM codes are easily configurable in the sense that any rational code rate can be realized without complex optimization. As a universal construction, the main advantage of the SysConv-LDGM codes is their near-capacity performance in the waterfall region and predictable performance in the error-floor region that can be lowered down to any target as required by increasing the density of the uncoupled LDGM codes. Numerical results are also provided to verify our analysis.

Log-Logarithmic Time Pruned Polar Coding

A pruned variant of polar coding is proposed for binary erasure channels. For sufficiently small $\varepsilon>0$, we construct a series of capacity achieving codes with block length $N=\varepsilon^{-5}$, code rate $R=\text{Capacity}-\varepsilon$, error probability $P=\varepsilon$, and encoding and decoding time complexity $\text{bC}=O(\log\left|\log\varepsilon\right|)$ per information bit. The given per-bit complexity $\text{bC}$ is log-logarithmic in $N$, in $\text{Capacity}-R$, and in $P$; no known family of codes possesses this property. It is also the second lowest $\text{bC}$ after repeat-accumulate codes and their variants. While random codes and classical polar codes are the only two families of capacity-achieving codes whose $N$, $R$, $P$, and $\text{bC}$ were written down as explicit functions, our construction gives the third family. Then we generalize the result to: Fix a prime $q$ and fix a $q$-ary-input discrete symmetric memoryless channel. For sufficiently small $\varepsilon>0$, we construct a series of capacity achieving codes with block length $N=\varepsilon^{-O(1)}$, code rate $R=\text{Capacity}-\varepsilon$, error probability $P=\varepsilon$, and encoding and decoding time complexity $\text{bC}=O(\log\left|\log\varepsilon\right|)$ per information bit. The later construction gives the fastest family of capacity-achieving codes to date on those channels.

Polar Coding for Non-Stationary Channels

The problem of polar coding for an arbitrary sequence of independent binary-input memoryless symmetric (BMS) channels $\left \{{W_{i}}\right \}_{i=1}^{N}$ is considered. Such a sequence of channels is referred to as a non-stationary sequence of channels and arises in applications where data symbols experience different and independent channel characteristics. The sequence of channels is assumed to be completely known to both the transmitter and the receiver (a coherent scenario). Also, at each code block transmission, each of the channels is used only once. In other words, a codeword of length $N$ is constructed and then the $i$ -th encoded bit is transmitted over $W_{i}$ . The goal is to operate at a rate $R$ close to the average of the symmetric capacities of $W_{i}$ ’s, denoted by $\overline {I}_{N}$ . To this end, we construct a polar coding scheme using Arikan’s channel polarization transform in combination with certain permutations at each polarization level and certain skipped operations. In particular, given a non-stationary sequence of BMS channels $\left \{{W_{i}}\right \}_{i=1}^{N}$ and $P_{e}$ , where $0 , we construct a polar code of length $N$ and rate $R$ guaranteeing a block error probability of at most $P_{e}$ for transmission over $\left \{{W_{i}}\right \}_{i=1}^{N}$ such that $N \leqslant \frac {\kappa }{(\overline {I}_{N}- R)^{\mu }}$ , where $\mu $ is a constant and $\kappa $ is a constant depending on $P_{e}$ and $\mu $ . We further show a numerical upper bound on $\mu $ that is: $\mu \leqslant 7.34$ for non-stationary binary erasure channels and $\mu \leqslant 8.54$ for general non-stationary BMS channels. The encoding and decoding complexities of the constructed polar code preserve $O(N \log N)$ complexity of Arikan’s polar codes. In an asymptotic sense, when coded bits are transmitted over a non-stationary sequence of BMS channels $\left \{{W_{i}}\right \}_{i=1}^{\infty }$ , our proposed scheme achieves the average symmetric capacity $\overline {I}(\left \{{W_{i}}\right \}_{i=1}^{\infty }) \,\,{\textstyle \stackrel {\mathrm{ def}}{=}}\,\,\lim _{N\rightarrow \infty } \frac {1}{N}\sum _{i=1}^{N} I(W_{i})$ , assuming that the limit exists.

Spatially coupled turbo-coded continuous phase modulation: asymptotic analysis and optimization

For serially or parallel concatenated communication systems, spatial coupling techniques enable to improve the threshold of these systems under iterative decoding using belief propagation (BP). For the case of low-density parity-check (LDPC) codes, it has been shown that, under some asymptotic assumptions, spatially coupled ensembles have BP thresholds that approach the bitwise maximum a posteriori (MAP) threshold of the related uncoupled ensemble. This phenomenon is often referred to as threshold saturation, and it has sometimes very important consequences. For example, in the case of regular LDPC code ensembles, spatial coupling enables to achieve asymptotically the capacity for any class of binary memoryless symmetric channels. Since then, this threshold saturation has been conjectured or proved for several other types of concatenations. In this work, we consider a serially concatenated scheme which is the serial concatenation of a simple outer convolutional code and a continuous phase modulator (CPM) separated by an interleaver. Then, we propose a method to do the spatial coupling of several replicas of this serially concatenated scheme, aiming to improve the asymptotic convergence threshold. First, exploiting the specific structure of the proposed system, an original procedure is proposed in order to terminate the spatially coupled turbo-coded CPM scheme. In particular, the proposed procedure aims to ensure the continuity of the transmitted signal among spatially coupled replicas, enabling to keep one of the core characteristics and advantages of coded CPM schemes. Then, based on an asymptotic analysis, we show that the proposed scheme has very competitive thresholds when compared to carefully designed spatially coupled LDPC codes. Furthermore, it is shown how we can accelerate the convergence rate of the designed systems by optimizing the connection distributions in the coupling matrices. Finally, by investigating on different continuous phase modulation schemes, we corroborate the conjecture stating that spatially coupled turbo-coded CPM schemes saturate to a lower bound very close to the threshold given by the extrinsic information transfer (EXIT) area theorem.

Performance‐traffic‐trade‐off of two novel hard decision and two soft decision fusion periodogram‐based algorithms for cooperative spectrum sensing under unreliable reporting channel

Spectrum sensing is one of the most important tasks of cognitive radios (CRs) to detect idle bands for opportunistic access. In centralised cooperative sensing, CRs sense a target band and share sensing information via reporting channel with a fusion centre to decide whether the sensed band is vacant or not. This information can be raw data in soft decision fusion scheme or local decisions on the channel occupation state in hard decision fusion one. A soft decision typically achieves better detection performances, but the hard one intrinsically offers smaller data traffic. This study proposes two periodogram-based hard decision algorithms and makes a detailed performance-traffic-trade-off with two periodogram-based soft-decision algorithms. All the approaches are based on the circular folding cooperative power spectral density split cancellation algorithm. Performance analyses are made under unavoidable errors in the reporting channel. In the first assumption, the simple memoryless binary symmetric channel is adopted. In the second one, the more realist memory Gilbert–Elliott channel is assumed. Results show that the proposed schemes are sensitive to reporting errors, but performance loss can be supplanted with channel encoding, yet keeping the traffic smaller than in the soft decision without channel encoding.

Velocity Analysis of BP Decoding Waves for SC-LDPC Ensembles on BMS Channels: An Interpolation-Based Approach

This paper is concerned with the dynamics of spatially-coupled low-density parity-check (SC-LDPC) ensembles for transmission on general binary memoryless symmetric (BMS) channels under belief-propagation (BP) decoding. The decoding waves of such ensembles are found to exhibit solitonic behavior, propagating along the Tanner graphs at asymptotically constant velocities. A low-complexity approach termed interpolated density evolution (IDE) is proposed to predict the decoding wave velocities. In this approach, the densities of a decoding wave are approximated by interpolating between some fixed points of an uncoupled DE recursion with one-dimensional functions. Two transfer functions for updating these interpolation functions in the IDE recursion are established and a simple strategy is introduced to deal with the coexistence of multiple transition regions. In addition, a threshold analysis is developed based on two ansatzes, explaining why our approach can achieve a good trade-off between computational cost and accuracy, as illustrated with some numerical examples at the end of this paper.

Performance Bounds of Concatenated Polar Coding Schemes

A concatenated coding scheme over binary memoryless symmetric (BMS) channels using a polarization transformation followed by outer sub-codes is analyzed. Achievable error exponents and upper bounds on the error rate are derived. The first bound is obtained using outer codes which are typical random linear codes. As a byproduct of this bound, it determines the rates of the outer codes. A lower bound on the error exponent that holds for all BMS channels with a given capacity is then derived. Improved bounds and approximations for finite blocklength codes using channel dispersions (normal approximation), as well as converse and approximate converse results, are also obtained. The bounds are compared with actual simulation results from the literature. For the cases considered, when transmitting over the binary input additive white Gaussian noise channel, there was only a small gap between the channel dispersion-based approximation and the actual error rate of concatenated BCH-polar codes.

Generalized Partial Orders for Polar Code Bit-Channels

We study partial orders (POs) for the synthesized bit-channels of polar codes. First, we give an alternative proof of an existing PO for bit-channels with the same Hamming weight and use the underlying idea to extend the bit-channel ordering to some additional cases. In particular, the bit-channel ordering for a given code block length is used to generate additional bit-channel ordering relationships for larger block lengths, generalizing previously known POs. Next, we consider POs especially for the binary erasure channel (BEC). We identify a symmetry property of the Bhattacharyya parameters of complementary bit-channel pairs on the BEC and provide a condition for the alignment of polarized sets of bit-channels for the BEC and general binary-input memoryless symmetric (BMS) channels. Numerical examples and further properties about the POs for the bit-channels with different Hamming weights are provided to illustrate the new POs. The bit-channels with universal ordering positions, which are independent of the channel erasure probability, are verified for all of the code block lengths. Finally, we show the threshold behavior of the Bhattacharyya parameters of some bit-channels by approximating the threshold values. The corresponding value for a bit-channel can be used to determine whether it is good or bad when the underlying channel is known.

A High-Speed Successive-Cancellation Decoder for Polar Codes Using Approximate Computing

Polar codes are a new class of forward-error-correction codes, which have been proved asymptotically capacity-achieving for symmetric memoryless channels. Successive-cancellation (SC) decoders are low complexity while suffering from low speed due to their serial processing nature. Based on a newly published fast simplified SC algorithm, we propose a more efficient decoder than the prior art in this brief. Well-optimized components in the decoder core are developed. Additionally, several efficient approximate computing units (Apx-CUs) are introduced to substitute their accurate counterparts in the decoder to achieve even higher operating frequency. Field-programmable gate-array implementation results show that our design is about 1.5 times faster than the prior art, and the proposed Apx-CUs can bring an additional 19% processing speed.

Displacement Convexity in Spatially Coupled Scalar Recursions

We introduce a technique for the analysis of general spatially coupled systems that are governed by scalar recursions. Such systems can be expressed in variational form in terms of a potential function. We show, under mild conditions, that the potential function is displacement convex and that the minimizers are given by the fixed points (FPs) of the recursions. Furthermore, we give the conditions on the system such that the minimizing FP is unique up to translation along the spatial direction. The condition matches with that of Kudekar et al. [20] for the existence of spatial FPs. Displacement convexity applies to a wide range of spatially coupled recursions appearing in coding theory, compressive sensing, random constraint satisfaction problems, as well as statistical-mechanics models. We illustrate it with applications to low-density parity-check (LDPC) and generalized LDPC codes used for the transmission on the binary erasure channel or general binary memoryless symmetric channels within the Gaussian reciprocal channel approximation as well as compressive sensing.

The Velocity of the Propagating Wave for Spatially Coupled Systems With Applications to LDPC Codes

We consider the dynamics of message passing for spatially coupled codes and, in particular, the set of density evolution equations that tracks the profile of decoding errors along the spatial direction of coupling. It is known that, for suitable boundary conditions and after a transient phase, the error profile exhibits a “solitonic behavior.” Namely, a uniquely shaped wavelike solution develops, which propagates with a constant velocity. Under this assumption, we derive an analytical formula for the velocity in the framework of a continuum limit of the spatially coupled system. The general formalism is developed for spatially coupled low-density parity-check codes on general binary memoryless symmetric channels, which form the main systems of interest in this paper. We apply the formula for special channels and illustrate that it matches the direct numerical evaluation of the velocity for a wide range of noise values. A possible application of the velocity formula to the evaluation of finite size scaling law parameters is also discussed. We conduct a similar analysis for general scalar systems and illustrate the findings with applications to compressive sensing and generalized low-density parity-check codes on the binary erasure or binary symmetric channels.