Binary Memoryless Symmetric Channels Research Articles

Reed–Muller Codes on BMS Channels Achieve Vanishing Bit-Error Probability for all Rates Below Capacity

This paper considers the performance of Reed-Muller (RM) codes transmitted over binary memoryless symmetric (BMS) channels under bitwise maximum-a-posteriori (bit-MAP) decoding. Its main result is that, for a fixed BMS channel, the family of binary RM codes can achieve a vanishing bit-error probability at rates approaching the channel capacity. This partially resolves a long-standing open problem that connects information theory and error-correcting codes. In contrast with the earlier result for the binary erasure channel, the new proof does not rely on hypercontractivity. Instead, it combines a nesting property of RM codes with new information inequalities relating the generalized extrinsic information transfer function and the extrinsic minimum mean-squared error.

Sub-4.7 Scaling Exponent of Polar Codes

Polar codes approach channel capacity provably and empirically and are thereby a constituent code of the 5G standard. Compared to low-density parity-check codes, however, the performance of short-length polar codes have rooms for improvement that could hinder its adoption by a wider class of applications. As part of the program that addresses the performance issue at short length, it is crucial to understand how fast binary memoryless symmetric channels polarize. A number, called scaling exponent, was defined to measure the speed of polarization and several estimates of the scaling exponent were given in literature. As of 2022, the tightest overestimate is 4.714 made by Mondelli, Hassani, and Urbanke in 2015. We lower the overestimate to 4.63. The idea behind this improvement is that, instead of describing the relation between a channel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> and its children <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ◻ and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ⊛, we describe the relation between <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> and its grandchildren <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ◻◻, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ◻⊛, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ⊛◻, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> ⊛⊛. By doing so, the evolution of channels becomes “less Markovian” and hence more tighter inequalities can be obtained.

Successive Cancellation Decoding of Single Parity-Check Product Codes: Analysis and Improved Decoding

A product code with single parity-check component codes can be described via the tools of a multi-kernel polar code, where the rows of the generator matrix are chosen according to the constraints imposed by the product code construction. Following this observation, successive cancellation decoding of such codes is introduced. In particular, the error probability of single parity-check product codes over binary memoryless symmetric channels under successive cancellation decoding is characterized. A bridge with the analysis of product codes introduced by Elias is also established for the binary erasure channel. Successive cancellation list decoding of single parity-check product codes is then described. For the provided example, simulations over the binary input additive white Gaussian channel show that successive cancellation list decoding outperforms belief propagation decoding applied to the code graph. Finally, the performance of the concatenation of a product code with a high-rate outer code is investigated via distance spectrum analysis. Examples of concatenations performing within 0.7 dB from the random coding union bound are provided.

An Information-Theoretic Perspective on Successive Cancellation List Decoding and Polar Code Design

This work identifies information-theoretic quantities that are closely related to the required list size on average for successive cancellation list (SCL) decoding to implement maximum-likelihood decoding over general binary memoryless symmetric (BMS) channels. It also provides upper and lower bounds for these quantities that can be computed efficiently for very long codes. For the binary erasure channel (BEC), we provide a simple method to estimate the mean accurately via density evolution. The analysis shows how to modify, e.g., Reed-Muller codes, to improve the performance when practical list sizes, e.g., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L\in {[{8, 1024}]}$ </tex-math></inline-formula> , are adopted. Exemplary constructions with block lengths <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N\in \{128,512\}$ </tex-math></inline-formula> outperform polar codes of 5G over the binary-input additive white Gaussian noise channel. It is further shown that there is a concentration around the mean of the logarithm of the required list size for sufficiently large block lengths, over discrete-output BMS channels. We provide the probability mass functions (p.m.f.s) of this logarithm, over the BEC, for a sequence of the modified RM codes with an increasing block length via simulations, which illustrate that the p.m.f.s concentrate around the estimated mean.

Decoder Ties Do Not Affect the Error Exponent of the Memoryless Binary Symmetric Channel

The generalized Poor-Verdú error lower bound established by Chang <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> (2020) for multihypothesis testing is studied in the classical channel coding context. It is proved that for any sequence of block codes sent over the memoryless binary symmetric channel (BSC), the minimum probability of error (under maximum likelihood decoding) has a relative deviation from the generalized bound that grows at most linearly in blocklength. This result directly implies that for arbitrary codes used over the BSC, decoder ties can only affect the subexponential behavior of the minimum probability of error.

Encoding–decoding strategy based resilient state estimation for bias-corrupted stochastic nonlinear systems

This paper is concerned with the resilient state estimation problem for a type of stochastic nonlinear systems, in which the possible dynamical bias is considered that is depicted by a dynamical equation. In pursuit of enhancing the robustness of the propagated data, a binary encoding strategy (BES) is exploited in the binary symmetric channel (BSC). While the random bit errors caused by the channel noise may take place during the propagation of the binary bit string via the memoryless BSC. To characterize the occurrence of the bit errors, a series of Bernoulli distributed random variables is adopted. More specifically, in order to deal with the possible gain fluctuation of the estimator in the execution process, a resilient state estimator is employed. This paper intends to put forward a novel resilient estimation scheme under the BES, which can assure that the estimation error dynamics is exponentially ultimately bounded in mean square. A sufficient criterion is first acquired for the existence of the expected resilient estimator and the estimator parameter is achieved by solving a convex optimization problem. Finally, an illustrative simulation example is provided to verify the validity of the theoretical results.

A Lower Bound on the Essential Interactive Capacity of Binary Memoryless Symmetric Channels

The essential interactive capacity of a discrete memoryless channel is defined in this paper as the maximal rate at which the transcript of any interactive protocol can be reliably simulated over the channel, using a deterministic coding scheme. In contrast to other interactive capacity definitions in the literature, this definition makes no assumptions on the order of speakers (which can be adaptive) and does not allow any use of private/public randomness; hence, the essential interactive capacity is a function of the channel model only. It is shown that the essential interactive capacity of any binary memoryless symmetric (BMS) channel is at least 0.0302 its Shannon capacity. To that end, we present a simple coding scheme, based on extended-Hamming codes combined with error detection, that achieves the lower bound in the special case of the binary symmetric channel (BSC). We then adapt the scheme to the entire family of BMS channels, and show that it achieves the same lower bound using extremes of the Bhattacharyya parameter.

Error Exponents of LDPC Codes under Low-Complexity Decoding.

This paper deals with the specific construction of binary low-density parity-check (LDPC) codes. We derive lower bounds on the error exponents for these codes transmitted over the memoryless binary symmetric channel (BSC) for both the well-known maximum-likelihood (ML) and proposed low-complexity decoding algorithms. We prove the existence of such LDPC codes that the probability of erroneous decoding decreases exponentially with the growth of the code length while keeping coding rates below the corresponding channel capacity. We also show that an obtained error exponent lower bound under ML decoding almost coincide with the error exponents of good linear codes.

Spatially coupled turbo-coded continuous phase modulation: asymptotic analysis and optimization

For serially or parallel concatenated communication systems, spatial coupling techniques enable to improve the threshold of these systems under iterative decoding using belief propagation (BP). For the case of low-density parity-check (LDPC) codes, it has been shown that, under some asymptotic assumptions, spatially coupled ensembles have BP thresholds that approach the bitwise maximum a posteriori (MAP) threshold of the related uncoupled ensemble. This phenomenon is often referred to as threshold saturation, and it has sometimes very important consequences. For example, in the case of regular LDPC code ensembles, spatial coupling enables to achieve asymptotically the capacity for any class of binary memoryless symmetric channels. Since then, this threshold saturation has been conjectured or proved for several other types of concatenations. In this work, we consider a serially concatenated scheme which is the serial concatenation of a simple outer convolutional code and a continuous phase modulator (CPM) separated by an interleaver. Then, we propose a method to do the spatial coupling of several replicas of this serially concatenated scheme, aiming to improve the asymptotic convergence threshold. First, exploiting the specific structure of the proposed system, an original procedure is proposed in order to terminate the spatially coupled turbo-coded CPM scheme. In particular, the proposed procedure aims to ensure the continuity of the transmitted signal among spatially coupled replicas, enabling to keep one of the core characteristics and advantages of coded CPM schemes. Then, based on an asymptotic analysis, we show that the proposed scheme has very competitive thresholds when compared to carefully designed spatially coupled LDPC codes. Furthermore, it is shown how we can accelerate the convergence rate of the designed systems by optimizing the connection distributions in the coupling matrices. Finally, by investigating on different continuous phase modulation schemes, we corroborate the conjecture stating that spatially coupled turbo-coded CPM schemes saturate to a lower bound very close to the threshold given by the extrinsic information transfer (EXIT) area theorem.

Performance‐traffic‐trade‐off of two novel hard decision and two soft decision fusion periodogram‐based algorithms for cooperative spectrum sensing under unreliable reporting channel

Spectrum sensing is one of the most important tasks of cognitive radios (CRs) to detect idle bands for opportunistic access. In centralised cooperative sensing, CRs sense a target band and share sensing information via reporting channel with a fusion centre to decide whether the sensed band is vacant or not. This information can be raw data in soft decision fusion scheme or local decisions on the channel occupation state in hard decision fusion one. A soft decision typically achieves better detection performances, but the hard one intrinsically offers smaller data traffic. This study proposes two periodogram-based hard decision algorithms and makes a detailed performance-traffic-trade-off with two periodogram-based soft-decision algorithms. All the approaches are based on the circular folding cooperative power spectral density split cancellation algorithm. Performance analyses are made under unavoidable errors in the reporting channel. In the first assumption, the simple memoryless binary symmetric channel is adopted. In the second one, the more realist memory Gilbert–Elliott channel is assumed. Results show that the proposed schemes are sensitive to reporting errors, but performance loss can be supplanted with channel encoding, yet keeping the traffic smaller than in the soft decision without channel encoding.

On the Commitment Capacity of Unfair Noisy Channels

Noisy channels are a valuable resource from a cryptographic point of view. They can be used for exchanging secret-keys as well as realizing other cryptographic primitives such as commitment and oblivious transfer. To be really useful, noisy channels have to be considered in the scenario where a cheating party has some degree of control over the channel characteristics. Damgard et al. (EUROCRYPT 1999) proposed a more realistic model where such level of control is permitted to an adversary, the so called unfair noisy channels, and proved that they can be used to obtain commitment and oblivious transfer protocols. Given that noisy channels are a precious resource for cryptographic purposes, one important question is determining the optimal rate in which they can be used. The commitment capacity has already been determined for the cases of discrete memoryless channels and Gaussian channels. In this work we address the problem of determining the commitment capacity of unfair noisy channels. We compute a single-letter characterization of the commitment capacity of unfair noisy channels. In the case where an adversary has no control over the channel (the fair case) our capacity reduces to the well-known capacity of a discrete memoryless binary symmetric channel.

Velocity Analysis of BP Decoding Waves for SC-LDPC Ensembles on BMS Channels: An Interpolation-Based Approach

This paper is concerned with the dynamics of spatially-coupled low-density parity-check (SC-LDPC) ensembles for transmission on general binary memoryless symmetric (BMS) channels under belief-propagation (BP) decoding. The decoding waves of such ensembles are found to exhibit solitonic behavior, propagating along the Tanner graphs at asymptotically constant velocities. A low-complexity approach termed interpolated density evolution (IDE) is proposed to predict the decoding wave velocities. In this approach, the densities of a decoding wave are approximated by interpolating between some fixed points of an uncoupled DE recursion with one-dimensional functions. Two transfer functions for updating these interpolation functions in the IDE recursion are established and a simple strategy is introduced to deal with the coexistence of multiple transition regions. In addition, a threshold analysis is developed based on two ansatzes, explaining why our approach can achieve a good trade-off between computational cost and accuracy, as illustrated with some numerical examples at the end of this paper.

A Deliberate Bit Flipping Coding Scheme for Data-Dependent Two-Dimensional Channels

In this paper, we present a deliberate bit flipping (DBF) coding scheme for binary two-dimensional (2-D) channels, where specific patterns in channel inputs are the significant cause of errors. The idea is to eliminate a constrained encoder and, instead, embed a constraint into an error correction codeword that is arranged into a 2-D array by deliberately flipping the bits that violate the constraint. The DBF method relies on the error correction capability of the code being used so that it should be able to correct both deliberate errors and channel errors. Therefore, it is crucial to flip minimum number of bits in order not to overburden the error correction decoder. We devise a constrained combinatorial formulation for minimizing the number of flipped bits for a given set of harmful patterns. The generalized belief propagation algorithm is used to find an approximate solution for the problem. We evaluate the performance gain of our proposed approach on a data-dependent 2-D channel, where 2-D isolated-bits patterns are the harmful patterns for the channel. Furthermore, the performance of the DBF method is compared with classical 2-D constrained coding schemes for the 2-D no isolated-bits constraint on a memoryless binary symmetric channel.

Performance Bounds of Concatenated Polar Coding Schemes

A concatenated coding scheme over binary memoryless symmetric (BMS) channels using a polarization transformation followed by outer sub-codes is analyzed. Achievable error exponents and upper bounds on the error rate are derived. The first bound is obtained using outer codes which are typical random linear codes. As a byproduct of this bound, it determines the rates of the outer codes. A lower bound on the error exponent that holds for all BMS channels with a given capacity is then derived. Improved bounds and approximations for finite blocklength codes using channel dispersions (normal approximation), as well as converse and approximate converse results, are also obtained. The bounds are compared with actual simulation results from the literature. For the cases considered, when transmitting over the binary input additive white Gaussian noise channel, there was only a small gap between the channel dispersion-based approximation and the actual error rate of concatenated BCH-polar codes.

Displacement Convexity in Spatially Coupled Scalar Recursions

We introduce a technique for the analysis of general spatially coupled systems that are governed by scalar recursions. Such systems can be expressed in variational form in terms of a potential function. We show, under mild conditions, that the potential function is displacement convex and that the minimizers are given by the fixed points (FPs) of the recursions. Furthermore, we give the conditions on the system such that the minimizing FP is unique up to translation along the spatial direction. The condition matches with that of Kudekar et al. [20] for the existence of spatial FPs. Displacement convexity applies to a wide range of spatially coupled recursions appearing in coding theory, compressive sensing, random constraint satisfaction problems, as well as statistical-mechanics models. We illustrate it with applications to low-density parity-check (LDPC) and generalized LDPC codes used for the transmission on the binary erasure channel or general binary memoryless symmetric channels within the Gaussian reciprocal channel approximation as well as compressive sensing.

The Velocity of the Propagating Wave for Spatially Coupled Systems With Applications to LDPC Codes

We consider the dynamics of message passing for spatially coupled codes and, in particular, the set of density evolution equations that tracks the profile of decoding errors along the spatial direction of coupling. It is known that, for suitable boundary conditions and after a transient phase, the error profile exhibits a “solitonic behavior.” Namely, a uniquely shaped wavelike solution develops, which propagates with a constant velocity. Under this assumption, we derive an analytical formula for the velocity in the framework of a continuum limit of the spatially coupled system. The general formalism is developed for spatially coupled low-density parity-check codes on general binary memoryless symmetric channels, which form the main systems of interest in this paper. We apply the formula for special channels and illustrate that it matches the direct numerical evaluation of the velocity for a wide range of noise values. A possible application of the velocity formula to the evaluation of finite size scaling law parameters is also discussed. We conduct a similar analysis for general scalar systems and illustrate the findings with applications to compressive sensing and generalized low-density parity-check codes on the binary erasure or binary symmetric channels.

FER Estimation in a Memoryless BSC With Variable Frame Length and Unreliable ACK/NAK Feedback

We consider the problem of estimating the frame error rate (FER) of a given memoryless binary symmetric channel by observing the success or failure of transmitted packets. Whereas FER estimation is relatively straightforward if all observations correspond to packets with equal length, the problem becomes considerably more complex when this is not the case. We develop FER estimators when transmissions of different lengths are observed, together with the Cramer–Rao lower bound (CRLB). Although the main focus is on maximum likelihood (ML) estimation, we also obtain low-complexity schemes performing close to optimal in some scenarios. In a second stage, we consider the case in which FER estimation is performed at a node different from the receiver, and incorporate the impairment of unreliable observations by considering noisy ACK/NAK feedback links. The impact of unreliable feedback is analyzed by means of the corresponding CRLB. In this setting, the ML estimator is obtained by applying the expectation–maximization algorithm to jointly estimate the error probabilities of the data and feedback links. Simulation results illustrate the benefits of the proposed estimators.

Unified Scaling of Polar Codes: Error Exponent, Scaling Exponent, Moderate Deviations, and Error Floors

Consider the transmission of a polar code of block length $N$ and rate $R$ over a binary memoryless symmetric channel $W$ and let $P_{\mathrm{ e}}$ be the block error probability under successive cancellation decoding. In this paper, we develop new bounds that characterize the relationship of the parameters $R$ , $N$ , $P_{\mathrm{ e}}$ , and the quality of the channel $W$ quantified by its capacity $I(W)$ and its Bhattacharyya parameter $Z(W)$ . In previous work, two main regimes were studied. In the error exponent regime, the channel $W$ and the rate $R are fixed, and it was proved that the error probability $P_{\mathrm{ e}}$ scales roughly as $2^{-\sqrt {N}}$ . In the scaling exponent approach, the channel $W$ and the error probability $P_{\mathrm{ e}}$ are fixed and it was proved that the gap to capacity $I(W)-R$ scales as $N^{-1/\mu }$ . Here, $\mu $ is called scaling exponent and this scaling exponent depends on the channel $W$ . A heuristic computation for the binary erasure channel (BEC) gives $\mu =3.627$ and it was shown that, for any channel $W$ , $3.579 \le \mu \le 5.702$ . Our contributions are as follows. First, we provide the tighter upper bound $\mu \le 4.714$ valid for any $W$ . With the same technique, we obtain the upper bound $\mu \le 3.639$ for the case of the BEC; this upper bound approaches very closely the heuristically derived value for the scaling exponent of the erasure channel. Second, we develop a trade-off between the gap to capacity $I(W)-R$ and the error probability $P_{\mathrm{ e}}$ as the functions of the block length $N$ . In other words, we neither fix the gap to capacity (error exponent regime) nor the error probability (scaling exponent regime), but we do consider a moderate deviations regime in which we study how fast both quantities, as the functions of the block length $N$ , simultaneously go to 0. Third, we prove that polar codes are not affected by error floors . To do so, we fix a polar code of block length $N$ and rate $R$ . Then, we vary the channel $W$ and study the impact of this variation on the error probability. We show that the error probability $P_{\mathrm{ e}}$ scales as the Bhattacharyya parameter $Z(W)$ raised to a power that scales roughly like ${\sqrt {N}}$ . This agrees with the scaling in the error exponent regime.

Thermodynamics of the binary symmetric channel

We study a hidden Markov process which is the result of a transmission of the binary symmetric Markov source over the memoryless binary symmetric channel. This process has been studied extensively in Information Theory and is often used as a benchmark case for the so-called denoising algorithms. Exploiting the link between this process and the 1D Random Field Ising Model (RFIM), we are able to identify the Gibbs potential of the resulting Hidden Markov process. Moreover, we obtain a stronger bound on the memory decay rate. We conclude with a discussion on implications of our results for the development of denoising algorithms.

From Polar to Reed-Muller Codes

The year 2016, in which I am writing these words, marks the centenary of Claude Shannon, the father of information theory. In his landmark 1948 paper Mathematical Theory of Communication, Shannon established the largest rate at which reliable communication is possible, and he referred to it as the channel capacity. Since then, researchers have focused on the design of practical coding schemes that could approach such a limit. The road to channel capacity has been almost 70 years long and, after many ideas, occasional detours, and some rediscoveries, it has culminated in the description of low-complexity and provably capacity-achieving coding schemes, namely, polar codes and iterative codes based on sparse graphs. However, next-generation communication systems require an unprecedented performance improvement and the number of transmission settings relevant in applications is rapidly increasing. Hence, although Shannon's limit seems finally close at hand, new challenges are just around the corner. In this thesis, we trace a road that goes from polar to Reed-Muller codes and, by doing so, we investigate three main topics: unified scaling, non-standard channels, and capacity via symmetry. First, we consider unified scaling. A coding scheme is capacity-achieving when, for any rate smaller than capacity, the error probability tends to 0 as the block length becomes increasingly larger. However, the practitioner is often interested in more specific questions such as, How much do we need to increase the block length in order to halve the gap between rate and capacity?. We focus our analysis on polar codes and develop a unified framework to rigorously analyze the scaling of the main parameters, i.e., block length, rate, error probability, and channel quality. Furthermore, in light of the recent success of a list decoding algorithm for polar codes, we provide scaling results on the performance of list decoders. Next, we deal with non-standard channels. When we say that a coding scheme achieves capacity, we typically consider binary memoryless symmetric channels. However, practical transmission scenarios often involve more complicated settings. For example, the downlink of a cellular system is modeled as a broadcast channel, and the communication on fiber links is inherently asymmetric. We propose provably optimal low-complexity solutions for these settings. In particular, we present a polar coding scheme that achieves the best known rate region for the broadcast channel, and we describe three paradigms to achieve the capacity of asymmetric channels. To do so, we develop general coding primitives, such as the chaining construction that has already proved to be useful in a variety of communication problems. Finally, we show how to achieve capacity via symmetry. In the early days of coding theory, a popular paradigm consisted in exploiting the structure of algebraic codes to devise practical decoding algorithms. However, proving the optimality of such coding schemes remained an elusive goal. In particular, the conjecture that Reed-Muller codes achieve capacity dates back to the 1960s. We solve this open problem by showing that Reed-Muller codes and, in general, codes with sufficient symmetry are capacity-achieving over erasure channels under optimal MAP decoding. As the proof does not rely on the precise structure of the codes, we are able to show that symmetry alone guarantees optimal performance.