Low-density Generator Matrix Codes Research Articles

Systematic Convolutional Low Density Generator Matrix Code

In this paper, we propose a systematic low density generator matrix (LDGM) code ensemble, which is defined by the Bernoulli process. We prove that, under maximum likelihood (ML) decoding, the proposed ensemble can achieve the capacity of binary-input output symmetric (BIOS) memoryless channels in terms of bit error rate (BER). The proof technique reveals a new mechanism, different from lowering down frame error rate (FER), that the BER can be lowered down by assigning light codeword vectors to light information vectors. The finite length performance is analyzed by deriving an upper bound and a lower bound, both of which are shown to be tight in the high signal-to-noise ratio (SNR) region. To improve the waterfall performance, we construct the systematic convolutional LDGM (SysConv-LDGM) codes by a random splitting process. The SysConv-LDGM codes are easily configurable in the sense that any rational code rate can be realized without complex optimization. As a universal construction, the main advantage of the SysConv-LDGM codes is their near-capacity performance in the waterfall region and predictable performance in the error-floor region that can be lowered down to any target as required by increasing the density of the uncoupled LDGM codes. Numerical results are also provided to verify our analysis.

3D EXIT Charts for Analyzing the 5G 3GPP New Radio LDPC Decoder

Low Density Parity Check (LDPC) codes have been selected to provide channel coding for data in the next generation of wireless communication standard referred to in practice as the 3GPP New Radio (NR). In contrast to the LDPC codes adopted in previous standards, the NR LDPC code can be considered to be a concatenation of a core LDPC code and a Low Density Generator Matrix (LDGM) code. This particular feature has the advantage of offering flexible coding rate, but it prevents the application of conventional extrinsic information transfer (EXIT) chart analysis. This article characterizes this problem and addresses it using a novel reinterpretation of the NR LDPC factor graph. Based on this factor graph, a novel 3D EXIT chart technique is conceived for our three-stage scheme, which facilitates the visual characterisation of the NR LDPC decoder's iterative decoding convergence process for the first time. The proposed EXIT chart analysis accurately visualizes the mutual information exchange amongst the components of the NR LDPC decoder, which was not facilitated by the conventional 2D EXIT chart. We demonstrate the power of this technique by using it to design a novel iterative decoding activation order for the NR LDPC decoder, which reduces the decoding complexity by approximately 17% compared to a conventional flooding decoder. This is achieved without degrading its error correction capability. We conclude by discussing several other opportunities for exploiting the proposed 3D EXIT chart technique to improve the design of concatenated LDPC and LDGM codes.

Successive Wyner-Ziv Coding for the Binary CEO Problem Under Logarithmic Loss

The $L$-link binary Chief Executive Officer (CEO) problem under logarithmic loss is investigated in this paper. A quantization splitting technique is applied to convert the problem under consideration to a $(2L-1)$-step successive Wyner-Ziv (WZ) problem, for which a practical coding scheme is proposed. In the proposed scheme, low-density generator-matrix (LDGM) codes are used for binary quantization while low-density parity-check (LDPC) codes are used for syndrome generation; the decoder performs successive decoding based on the received syndromes and produces a soft reconstruction of the remote source. The simulation results indicate that the rate-distortion performance of the proposed scheme can approach the theoretical inner bound based on binary-symmetric test-channel models.

Combining the Burrows-Wheeler Transform and RCM-LDGM Codes for the Transmission of Sources with Memory at High Spectral Efficiencies.

In this paper, we look at the problem of implementing high-throughput Joint Source- Channel (JSC) coding schemes for the transmission of binary sources with memory over AWGN channels. The sources are modeled either by a Markov chain (MC) or a hidden Markov model (HMM). We propose a coding scheme based on the Burrows-Wheeler Transform (BWT) and the parallel concatenation of Rate-Compatible Modulation and Low-Density Generator Matrix (RCM-LDGM) codes. The proposed scheme uses the BWT to convert the original source with memory into a set of independent non-uniform Discrete Memoryless (DMS) binary sources, which are then separately encoded, with optimal rates, using RCM-LDGM codes.

Asymptotic BER EXIT chart analysis for high rate codes based on the parallel concatenation of analog RCM and digital LDGM codes

This paper proposes an extrinsic information transfer (EXIT) chart analysis and an asymptotic bit error rate (BER) prediction method to speed up the design of high rate RCM-LDGM hybrid codes over AWGN and fast Rayleigh channels. These codes are based on a parallel concatenation of a rate compatible modulation (RCM) code with a low-density generator matrix (LDGM) code. The decoder uses the iterative sum-product algorithm to exchange information between the variable nodes (VNs) and the two types of constituent check nodes: RCM-CN and LDGM-CN. The novelty of the proposed EXIT chart procedure lies on the fact that it mixes together the analog RCM check nodes with the digital LDGM check nodes, something not possible in previous multi-edge EXIT charts proposed in the literature.

On Privacy Amplification, Lossy Compression, and Their Duality to Channel Coding

We examine the task of privacy amplification from information-theoretic and coding-theoretic points of view. In the former, we give a one-shot characterization of the optimal rate of privacy amplification against classical adversaries in terms of the optimal type-II error in asymmetric hypothesis testing. This formulation can be easily computed to give finite-blocklength bounds and turns out to be equivalent to smooth min-entropy bounds by Renner and Wolf [Asiacrypt 2005] and Watanabe and Hayashi [ISIT 2013], as well as a bound in terms of the $E_\gamma$ divergence by Yang, Schaefer, and Poor [arXiv:1706.03866 [cs.IT]]. In the latter, we show that protocols for privacy amplification based on linear codes can be easily repurposed for channel simulation. Combined with known relations between channel simulation and lossy source coding, this implies that privacy amplification can be understood as a basic primitive for both channel simulation and lossy compression. Applied to symmetric channels or lossy compression settings, our construction leads to proto- cols of optimal rate in the asymptotic i.i.d. limit. Finally, appealing to the notion of channel duality recently detailed by us in [IEEE Trans. Info. Theory 64, 577 (2018)], we show that linear error-correcting codes for symmetric channels with quantum output can be transformed into linear lossy source coding schemes for classical variables arising from the dual channel. This explains a "curious duality" in these problems for the (self-dual) erasure channel observed by Martinian and Yedidia [Allerton 2003; arXiv:cs/0408008] and partly anticipates recent results on optimal lossy compression by polar and low-density generator matrix codes.

Encoding of Spatially Coupled LDGM Codes for Lossy Source Compression

It has been shown that a class of spatially coupled low-density generator-matrix (SC-LDGM) code ensembles displays distortion saturation for the lossy binary symmetric source coding problem with the belief propagation guided decimation (BPGD) algorithm, i.e., the BPGD distortion approaches the optimal expected distortion of the underlying ensemble asymptotically in code length. We investigate the distortion performance of a practical class of protograph-based SC-LDGM code ensembles and demonstrate distortion saturation numerically. Moreover, taking advantage of the convolutional structure of the SC-LDGM codes, we propose an efficient windowed encoding (WE) algorithm with two decimation techniques for lowering the WE complexity that maintain distortion performance close to the rate-distortion bound.

Information-theoretic thresholds from the cavity method

Vindicating a sophisticated but non-rigorous physics approach called the cavity method, we establish a formula for the mutual information in statistical inference problems induced by random graphs and we show that the mutual information holds the key to understanding certain important phase transitions in random graph models. We work out several concrete applications of these general results. For instance, we pinpoint the exact condensation phase transition in the Potts antiferromagnet on the random graph, thereby improving prior approximate results (Contucci et al., 2013) [34]. Further, we prove the conjecture from Krzakala et al. (2007) [55] about the condensation phase transition in the random graph coloring problem for any number q≥3 of colors. Moreover, we prove the conjecture on the information-theoretic threshold in the disassortative stochastic block model (Decelle et al., 2011) [35]. Additionally, our general result implies the conjectured formula for the mutual information in Low-Density Generator Matrix codes (Montanari, 2005) [73].

Flexible WOM codes for NAND flash memory based on raptor-like codes

Write-once memory (WOM) codes aim to extend the lifetime and improve the writing efficiency of storage devices such as NAND flash memory by reducing the number of erase operations. In this paper, a new rewriting scheme for NAND flash memory is proposed, which supports two writes (or only one rewrite) and allows the second write incrementally done multiple times by using raptor-like codes as rate-compatible (RC) low-density generator matrix (LDGM) codes. The proposed scheme improves writing efficiency of the NAND flash memory when combined with a proper page selection method. It is verified via numerical analysis that the proposed WOM codes outperform the conventional WOM codes in terms of writing efficiency.

Randomized Serially Concatenated LDGM Codes for the Gaussian Wiretap Channel

We study the application of a special class of low-density parity-check codes to the wiretap channel. We construct a randomized coding scheme based on serially concatenated low-density generator matrix codes and their duals extending the approach used for convolutional and turbo codes. Furthermore, we propose an efficient iterative decoder for this scheme utilizing a joint iterative message passing algorithm. We demonstrate via numerical examples that this approach outperforms other available practical coding alternatives for the Gaussian wiretap channel in terms of the resulting security gap.

Binary Wyner–Ziv code design based on compound LDGM–LDPC structures

In this study, a practical coding scheme is designed for the binary Wyner–Ziv (WZ) problem by using nested low-density generator-matrix (LDGM) and low-density parity-check (LDPC) codes. This scheme contains two steps in the encoding procedure. The first step involves applying the binary quantisation by employing LDGM codes and the second one is using the syndrome-coding technique by utilising LDPC codes. The decoding algorithm of the proposed scheme is based on the sum-product algorithm with the help of a side information available at the decoder side. It is theoretically shown that the compound structure has the capability of achieving the WZ bound. The proposed method approaches this bound by utilising the iterative message-passing algorithms in both encoding and decoding, although theoretical results show that it is asymptotically achievable.

Low-Complexity Soft-Decision Concatenated LDGM-Staircase FEC for High-Bit-Rate Fiber-Optic Communication

A concatenated soft-decision forward error correction (FEC) scheme consisting of an inner low-density generator-matrix (LDGM) code and an outer staircase code is proposed. The soft-decision LDGM code is used for error reduction, while the majority of bit errors are corrected by the low-complexity hard-decision staircase code. Decoding complexity of the concatenated code is quantified by a score based on the number of edges in the LDGM code Tanner graph, the number of decoding iterations, and the number of staircase code decoding operations. The inner LDGM ensemble is designed by solving an optimization problem, which minimizes the product of the average node degree and an estimate of the required number of decoding iterations. A search procedure is used to find the inner and outer code pair with lowest complexity. The design procedure results in a Pareto-frontier characterization of the tradeoff between net coding gain and complexity for the concatenated code. Simulations of code designs at $\text{20}\%$ overhead showed that the proposed scheme achieves net coding gains equivalent to existing soft-decision FEC solutions, with up to $\text{57}\%$ reduction in complexity.

Bounds on the Reliability of RaptorQ Codes in the Finite-Length Regime

In this paper we analyze the bounds on the reliability of RaptorQ codes under maximum likelihood decoding, especially in the finite-length regime. RaptorQ code ensembles by a high-order low density generator-matrix (LDGM) code as pre-code and an inner Luby transform (LT) code. By investigating the rank of the product of two random coefficient matrices of the high-order LDGM code and the inner LT code, we derive the expressions for the upper and lower bounds of decoding failure probability (DFP) on the RaptorQ code. Then, the accuracy of our derived theoretical bounds are verified through the Monte Carlo simulations with different degree distributions and short packets. The high accuracy bounds are then exploited to design near-optimum finite-length RaptorQ codes, enabling a tight control on the tradeoff between decoding complexity and DFP.

Video over DSL with LDGM Codes for Interactive Applications

Digital Subscriber Line (DSL) network access is subject to error bursts, which, for interactive video, can introduce unacceptable latencies if video packets need to be re-sent. If the video packets are protected against errors with Forward Error Correction (FEC), calculation of the application-layer channel codes themselves may also introduce additional latency. This paper proposes Low-Density Generator Matrix (LDGM) codes rather than other popular codes because they are more suitable for interactive video streaming, not only for their computational simplicity but also for their licensing advantage. The paper demonstrates that a reduction of up to 4 dB in video distortion is achievable with LDGM Application Layer (AL) FEC. In addition, an extension to the LDGM scheme is demonstrated, which works by rearranging the columns of the parity check matrix so as to make it even more resilient to burst errors. Telemedicine and video conferencing are typical target applications.

Performance Analysis of Raptor Codes Under Maximum Likelihood Decoding

In this paper, we analyze the maximum likelihood decoding performance of Raptor codes with a systematic low-density generator-matrix code as the pre-code. By investigating the rank of the product of two random coefficient matrices, we derive upper and lower bounds on the decoding failure probability. The accuracy of our analysis is validated through simulations. Results of extensive Monte Carlo simulations demonstrate that for Raptor codes with different degree distributions and pre-codes, the bounds obtained in this paper are of high accuracy. The derived bounds can be used to design near-optimum Raptor codes with short and moderate lengths.

Approaching the Rate-Distortion Limit With Spatial Coupling, Belief Propagation, and Decimation

We investigate an encoding scheme for lossy compression of a binary symmetric source based on simple spatially coupled low-density generator-matrix codes. The degree of the check nodes is regular and the one of code-bits is Poisson distributed with an average depending on the compression rate. The performance of a low complexity belief propagation guided decimation algorithm is excellent. The algorithmic rate-distortion curve approaches the optimal curve of the ensemble as the width of the coupling window grows. Moreover, as the check degree grows both curves approach the ultimate Shannon rate-distortion limit. The belief propagation guided decimation encoder is based on the posterior measure of a binary symmetric test-channel. This measure can be interpreted as a random Gibbs measure at a temperature directly related to the noise level of the test-channel. We investigate the links between the algorithmic performance of the belief propagation guided decimation encoder and the phase diagram of this Gibbs measure. The phase diagram is investigated thanks to the cavity method of spin glass theory which predicts a number of phase transition thresholds. In particular, the dynamical and condensation phase transition temperatures (equivalently test-channel noise thresholds) are computed. We observe that: 1) the dynamical temperature of the spatially coupled construction saturates toward the condensation temperature and 2) for large degrees the condensation temperature approaches the temperature (i.e., noise level) related to the information theoretic Shannon test-channel noise parameter of rate-distortion theory. This provides heuristic insight into the excellent performance of the belief propagation guided decimation algorithm. This paper contains an introduction to the cavity method.

High-performance forward error correction: Enabling multi-gigabit flows and beyond on commodity GPU and CPU hardware in presence of packet loss

In demanding real-time multimedia transmissions, even a small packet loss might significantly degrade the visual quality. As retransmission is not an option in real-time transfers especially when transmitting the data over long distances, it is necessary to employ mechanisms of Forward Error Correction (FEC). Low-Density Generator Matrix (LDGM) codes are known to be suitable for coding on large block sizes, however, high bitrates of currently used video formats (FullHD, 4K) also require high throughput of FEC coding and decoding. We propose a parallel design of LDGM encoding and decoding algorithms suitable for off-the-shelf, (massively) parallel platforms, such CPUs with vector units or GPUs, and evaluate our approach in real-world scenarios of high-definition and 4K video transmissions. Our results show that offloading FEC computation to such platform is beneficial for low-latency, high-quality multimedia transmissions and may even enable transmissions beyond 10Gbps once the commodity network interfaces reach this speed.

Network Code Division Multiplexing for Wireless Relay Networks

In this paper, we investigate the performance of a wireless relay network with multiple transmission sessions, in which multiple groups of source nodes communicate with their respective destination nodes via a shared wireless relay network. A multiple transmission session model with network code division multiplexing (NCDM) scheme is proposed to remove the inter-session interference at each destination. The fundamental idea of the NCDM scheme takes advantage of the property of $\mathbf{G}\odot\mathbf{H}^T=\mathbf{0}$ of the low-density generator matrix (LDGM) codes. Based on the analysis of the NCDM scheme, we investigate the relationship among the equivalent received signal vector, the number of sessions and the column weight of the generator matrix. New code design criteria for the construction of the generator matrix is proposed. We further evaluate the multiple transmission session model with the proposed NCDM scheme in terms of throughput and complexity. Our evaluation demonstrates that the proposed scheme not only has a linear computational complexity, but also shows a similar error performance in the AWGN case and a considerable throughput improvement compared with its counterpart, which is referred to as a serial session scheme, where groups of source nodes communicate with their respective destinations in a time division manner.

Threshold Saturation for Spatially Coupled LDPC and LDGM Codes on BMS Channels

Spatially-coupled low-density parity-check (LDPC) codes, which were first introduced as LDPC convolutional codes, have been shown to exhibit excellent performance under low-complexity belief-propagation decoding. This phenomenon is now termed threshold saturation via spatial coupling. Spatially-coupled codes have been successfully applied in numerous areas. In particular, it was proven that spatially-coupled regular LDPC codes universally achieve capacity over the class of binary memoryless symmetric (BMS) channels under belief-propagation decoding. Recently, potential functions have been used to simplify threshold saturation proofs for scalar and vector recursions. In this paper, potential functions are used to prove threshold saturation for irregular LDPC and low-density generator-matrix (LDGM) codes on BMS channels, extending the simplified proof technique to BMS channels. The corresponding potential functions are closely related to the average Bethe free entropy of the ensembles in the large-system limit. These functions also appear in statistical physics when the replica method is used to analyze optimal decoding.

A Simple Proof of Maxwell Saturation for Coupled Scalar Recursions

Low-density parity-check (LDPC) convolutional codes (or spatially coupled codes) were recently shown to approach capacity on the binary erasure channel (BEC) and binary-input memoryless symmetric channels. The mechanism behind this spectacular performance is now called threshold saturation via spatial coupling. This new phenomenon is characterized by the belief-propagation threshold of the spatially coupled ensemble increasing to an intrinsic noise threshold defined by the uncoupled system. In this paper, we present a simple proof of threshold saturation that applies to a wide class of coupled scalar recursions. Our approach is based on constructing potential functions for both the coupled and uncoupled recursions. Our results actually show that the fixed point of the coupled recursion is essentially determined by the minimum of the uncoupled potential function and we refer to this phenomenon as Maxwell saturation. A variety of examples are considered including the density-evolution equations for: irregular LDPC codes on the BEC, irregular low-density generator-matrix codes on the BEC, a class of generalized LDPC codes with BCH component codes, the joint iterative decoding of LDPC codes on intersymbol-interference channels with erasure noise, and the compressed sensing of random vectors with independent identically distributed components.