Ensemble Of Random Codes Research Articles

Probabilistic Analysis of Linear Programming Decoding

We initiate the probabilistic analysis of linear programming (LP) decoding of low-density parity-check (LDPC) codes. Specifically, we show that for a random LDPC code ensemble, the linear programming decoder of Feldman et al. succeeds in correcting a constant fraction of errors with high probability. The fraction of correctable errors guaranteed by our analysis surpasses previous non-asymptotic results for LDPC codes, and in particular exceeds the best previous finite-length result on LP decoding by a factor greater than ten. This improvement stems in part from our analysis of probabilistic bit-flipping channels, as opposed to adversarial channels. At the core of our analysis is a novel combinatorial characterization of LP decoding success, based on the notion of a generalized matching. An interesting by-product of our analysis is to establish the existence of ``probabilistic expansion'' in random bipartite graphs, in which one requires only that almost every (as opposed to every) set of a certain size expands, for sets much larger than in the classical worst-case setting.

Fingerprinting With Minimum Distance Decoding

This paper adopts an information-theoretic framework for the design of collusion-resistant coding/decoding schemes for digital fingerprinting. More specifically, the minimum distance decision rule is used to identify 1 out of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> pirates. Achievable rates, under this detection rule, are characterized in two scenarios. First, we consider the averaging attack where a random coding argument is used to show that the rate 1/2 is achievable with <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> =2 pirates. Our study is then extended to the general case of arbitrary <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> highlighting the underlying complexity-performance tradeoff. Overall, these results establish the significant performance gains offered by minimum distance decoding compared to other approaches based on orthogonal codes and correlation detectors which can support only a subexponential number of users (i.e., a zero rate). In the second scenario, we characterize the achievable rates, with minimum distance decoding, under any collusion attack that satisfies the marking assumption. For <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> =2 pirates, we show that the rate 1- <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> (0.25) ap 0.188 is achievable using an ensemble of random linear codes. For <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> ges 3, the existence of a nonresolvable collusion attack, with minimum distance decoding, for any nonzero rate is established. Inspired by our theoretical analysis, we then construct coding/decoding schemes for fingerprinting based on the celebrated belief-propagation framework. Using an explicit repeat-accumulate code, we obtain a vanishingly small probability of misidentification at rate 1/3 under averaging attack with <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> =2. For collusion attacks, which satisfy the marking assumption, we use a more sophisticated accumulate repeat accumulate code to obtain a vanishingly small misidentification probability at rate 1/9 with <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> =2. These results represent a marked improvement over the best available designs in the literature.

Reliable channel regions for good binary codes transmitted over parallel channels

We study the average error probability performance of binary linear code ensembles when each codeword is divided into J subcodewords with each being transmitted over one of J parallel channels. This model is widely accepted for a number of important practical channels and signaling schemes including block-fading channels, incremental redundancy retransmission schemes, and multicarrier communication techniques for frequency-selective channels. Our focus is on ensembles of good codes whose performance in a single channel model is characterized by a threshold behavior, e.g., turbo and low-density parity-check (LDPC) codes. For a given good code ensemble, we investigate reliable channel regions which ensure reliable communications over parallel channels under maximum-likelihood (ML) decoding. To construct reliable regions, we study a modifed 1961 Gallager bound for parallel channels. By allowing codeword bits to be randomly assigned to each component channel, the average parallel-channel Gallager bound is simplified to be a function of code weight enumerators and channel assignment rates. Special cases of this bound, average union-Bhattacharyya (UB), Shulman-Feder (SF), simplified-sphere (SS), and modified Shulman-Feder (MSF) parallel-channel bounds, allow for describing reliable channel regions using simple functions of channel and code spectrum parameters. Parameters describing the channel are the average parallel-channel Bhattacharyya noise parameter, the average channel mutual information, and parallel Gaussian channel signal-to-noise ratios (SNRs). Code parameters include the union-Bhattacharyya noise threshold and the weight spectrum distance to the random binary code ensemble. Reliable channel regions of repeat-accumulate (RA) codes for parallel binary erasure channels (BECs) and of turbo codes for parallel additive white Gaussian noise (AWGN) channels are numerically computed and compared with simulation results based on iterative decoding. In addition, an examp

Statistical Mechanical Approach to Error Exponents of Lossy Data Compression

We present a scheme to accurately evaluate the error exponents of a lossy data compression problem, which characterize average probabilities over a code ensemble of compression failure and success above or below a critical compression rate, respectively, utilizing the replica method (RM). Although the existing method used in information theory (IT) is, in practice, limited to ensembles of randomly constructed codes, the proposed RM-based approach can be applied to a wider class of ensembles. This approach reproduces the optimal expressions of the error exponents achieved by the random code ensembles, which are known in IT. In addition, the proposed framework is used to show that codes composed of non-monotonic perceptrons of a specific type can provide the optimal exponents in most cases, which is supported by numerical experiments.

Weighted Nonbinary Repeat–Accumulate Codes

Repeat-accumulate (RA) codes are random-like codes having remarkably good performance over an additive white Gaussian noise (AWGN) channel, like turbo and low-density parity-check (LDPC) codes. In this correspondence, we introduce an ensemble of random codes called "weighted nonbinary repeat-accumulate (WNRA) codes" whose encoder consists of a nonbinary repeater, a weighter, a pseudorandom symbol interleaver, and an accumulator over a finite field GF(q). They can be decoded in a simple way by applying the sum-product algorithm to their factor graphs over GF(q). Simulation results show that WNRA codes with proper weighting values over GF(4) or GF(8) are superior to binary RA codes on AWGN channels.

Random codes: minimum distances and error exponents

Minimum distances, distance distributions, and error exponents on a binary-symmetric channel (BSC) are given for typical codes from Shannon's random code ensemble and for typical codes from a random linear code ensemble. A typical random code of length N and rate R is shown to have minimum distance N/spl delta//sub GV/(2R), where /spl delta//sub GV/(R) is the Gilbert-Varshamov (GV) relative distance at rate R, whereas a typical linear code (TLC) has minimum distance N/spl delta//sub GV/(R). Consequently, a TLC has a better error exponent on a BSC at low rates, namely, the expurgated error exponent.

Bounds for low-density parity-check codes over partial response channels

A bounding technique is used to evaluate the performance of high-rate low-density parity-check codes over partial response channels. Performance bounds for an ensemble of random codes, as well as for particular finite-geometry codes, are computed. Simulations are used to evaluate the bounds. The results show that the bounds appear to be valid for signal-to-noise ratios and code rates of interest in magnetic recording.

Efficient encoding of low-density parity-check codes

Low-density parity-check (LDPC) codes can be considered serious competitors to turbo codes in terms of performance and complexity and they are based on a similar philosophy: constrained random code ensembles and iterative decoding algorithms. We consider the encoding problem for LDPC codes. More generally we consider the encoding problem for codes specified by sparse parity-check matrices. We show how to exploit the sparseness of the parity-check matrix to obtain efficient encoders. For the (3,6)-regular LDPC code, for example, the complexity of encoding is essentially quadratic in the block length. However, we show that the associated coefficient can be made quite small, so that encoding codes even of length n/spl sime/100000 is still quite practical. More importantly, we show that optimized codes actually admit linear time encoding.

Minimax robust coding for channels with uncertainty statistics

The problem of minimax robust coding for classes of channels with uncertainty in their statistical description is addressed. Specific consideration is given to: 1) discrete memoryless channels with uncertainty in the probability transition matrices; 2) discrete-time stationary Gaussian channels with spectral uncertainty; and to uncertainty with classes determined by 2-alternating Choquet capacities. Both block codes and convolutional codes are considered. A robust maximum-likelihood decoding rule is derived; the rule guarantees that, for all channels in the uncertainty class and all rates smaller than a critical rate, the average probability of decoding error for the ensemble of random block codes and the ensemble of random time-varying convolutional codes converges to zero exponentially with increasing block length or constraint length, respectively. The channel capacity and cut-off rate of the class are then evaluated.