Stationary Memoryless Channels Research Articles

The “Nontriviality” of the Union Bound for Decoding of Convolutional Codes

Transmission of an infinite binary information sequence over a stationary memoryless channel is considered using a time-invariant non-catastrophic convolutional encoder and the Viterbi decoder. The conventional proofs of the “union upper bound” (and other upper bounds) for decoding performance of convolutional codes involve some mathematical inaccuracies. These proofs implicitly assume that all information symbols are necessarily decoded at some time and, moreover, that the decoded path on the trellis diagram “meets” the true path sooner or later. Although these facts look natural (at least, for a binary symmetric channel), they must be proved. The goal of this note is to substantiate these facts for a binary symmetric channel and a general stationary memoryless channel.

The Sphere Packing Bound for Memoryless Channels

Sphere packing bounds (SPBs)—with prefactors that are polynomial in the block length—are derived for codes on two families of memoryless channels using Augustin’s method: (possibly nonstationary) memoryless channels with (possibly multiple) additive cost constraints and stationary memoryless channels with convex constraints on the composition (i.e., empirical distribution, type) of the input codewords. A variant of Gallager’s bound is derived in order to show that these sphere packing bounds are tight in terms of the exponential decay rate of the error probability with the block length under mild hypotheses.

A simple derivation of the refined sphere packing bound under certain symmetry hypotheses

A judicious application of the Berry-Esseen theorem via suitable Augustin information measures is demonstrated to be sufficient for deriving the sphere packing bound with a prefactor that is $\mathit{\Omega}\left(n^{-0.5(1-E_{sp}'(R))}\right)$ for all codes on certain families of channels (including the Gaussian channels and the nonstationary Renyi symmetric channels) and for the constant composition codes on stationary memoryless channels. The resulting nonasymptotic bounds have definite approximation error terms. As a preliminary result that might be of interest on its own, the trade-off between type I and type II error probabilities in the hypothesis testing problem with (possibly non-stationary) independent samples is determined up to some multiplicative constants, assuming that the probabilities of both types of error are decaying exponentially with the number of samples,using the Berry-Esseen theorem.

Граница сферической упаковки для каналов без памяти

Sphere packing bounds (SPBs) ---with prefactors that are polynomial in the block length--- are derived for codes on two families of memoryless channels using Augustin's method: (possibly non-stationary) memoryless channels with (possibly multiple) additive cost constraints and stationary memoryless channels with convex constraints on the composition (i.e. empirical distribution, type) of the input codewords. A variant of Gallager's bound is derived in order to show that these sphere packing bounds are tight in terms of the exponential decay rate of the error probability with the block length under mild hypotheses.

A Simple Proof of Polarization and Polarization for Non-Stationary Memoryless Channels

We give a simple proof of Arikan’s polarization phenomenon that uses only elementary methods. Using the same method, we show that Arikan’s construction also polarizes non-stationary memoryless channels in the same way it polarizes the stationary memoryless channels.

Reliability and Secrecy Functions of the Wiretap Channel Under Cost Constraint

The wiretap channel has been devised and studied first by Wyner, and subsequently extended to the case with nondegraded general wiretap channels by Csiszar and Korner. Focusing mainly on the stationary memoryless channel with cost constraint, we newly introduce the notion of reliability and secrecy functions as a fundamental tool to analyze and/or design the performance of an efficient wiretap channel system, including binary symmetric wiretap channels, Poisson wiretap channels, and Gaussian wiretap channels. Compact formulas for those functions are explicitly given for stationary memoryless wiretap channels. It is also demonstrated that, based on such a pair of reliability and secrecy functions, we can control the tradeoff between reliability and secrecy (usually conflicting), both with exponentially decreasing rates as block length \(n\) becomes large. Four ways to do so are given on the basis of rate shifting, rate exchange, concatenation, and change of cost constraint. In addition, the notion of the \(\delta \) secrecy capacity is defined and shown to attain the strongest secrecy standard among others. The maximized versus averaged secrecy measures is also discussed.

A Construction of Channel Code, Joint Source-Channel Code, and Universal Code for Arbitrary Stationary Memoryless Channels Using Sparse Matrices

A channel code is constructed using sparse matrices for stationary memoryless channels that do not necessarily have a symmetric property like a binary symmetric channel. It is also shown that the constructed code has the following remarkable properties. 1. Joint source-channel coding: Combining channel code with lossy source code, which is also constructed by sparse matrices, a simpler joint source-channel code can be constructed than that constructed by the ordinary block code. 2. Universal coding: The constructed channel code has a universal property under a specified condition.

Fountain Capacity

Fountain codes are currently employed for reliable and efficient transmission of information via erasure channels with unknown erasure rates. This correspondence introduces the notion of fountain capacity for arbitrary channels. In contrast to the conventional definition of rate, in the fountain setup the definition of rate penalizes the reception of symbols by the receiver rather than their transmission. Fountain capacity measures the maximum rate compatible with reliable reception regardless of the erasure pattern. We show that fountain capacity and Shannon capacity are equal for stationary memoryless channels. In contrast, Shannon capacity may exceed fountain capacity if the channel has memory or is not stationary.