Binary Symmetric Source Research Articles

The rate-distortion function of a binary symmetric source when side information may be absent (Corresp.)

A binary symmetric source with binary side information is given. An encoder codes the source for data compression with no knowledge of the side information. It is then decoded, perhaps with and perhaps without the presence of side information. The rate-distortion function of this scheme is a function of two variables: D_{1} is the distortion when side information is present at the decoder, and D2 is the distortion when side information is absent at the decoder. The rate-distortion function is shown to reduce to previously solved problems in much of the (D_{1}, D_{2}) -plane. Tight upper and lower bounds are found for the rate-distortion function in the rest of the (D_{1}, D_{2}) -plane.

Tree encoding of stationary ergodic sources with a fidelity criterion

Tree source coding theorems with a single letter fidelity criterion are proved for stationary ergodic sources using tree codes with a fixed branch length. The higher-than-exponential convergence of distortions is shown for binary symmetric sources and Hamming distortion measure when an excess in rate is allowed.

Explicit bounds to R(D) for a binary symmetric Markov source

A new upper hound <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R_{u}(D)</tex> and lower hound <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R_{\ell}(D)</tex> are developed for the rate-distortion function of a binary symmetric Markov source with respect to the frequency of error criterion. Both hounds are explicit in the sense that they do not depend on a blocklength parameter. In the interval <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{c} &lt; D &lt; 1/2 = D_{max}</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{c}</tex> is Gray's critical value of distortion, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R_{u}(D)</tex> is convex downward and possesses the correct value and the correct slope at both endpoints. The new lower bound <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R_{ell}(D)</tex> diverges from the Shannon lower bound at the same value of distortion as does the second-order Wyner-Ziv lower bound. However, it remains strictly positive for all <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D \leq 1/2</tex> and therefore eventually rises above all the Wyner-Ziv lower bounds as <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</tex> approaches <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/2</tex> . Some generalizations suggested by the analytical and geometrical techniques employed to derive <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R_{u}(D)</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R_{ell}(D)</tex> are discussed.

Computation of channel capacity and rate-distortion functions

By defining mutual information as a maximum over an appropriate space, channel capacities can be defined as double maxima and rate-distortion functions as double minima. This approach yields valuable new insights regarding the computation of channel capacities and rate-distortion functions. In particular, it suggests a simple algorithm for computing channel capacity that consists of a mapping from the set of channel input probability vectors into itself such that the sequence of probability vectors generated by successive applications of the mapping converges to the vector that achieves the capacity of the given channel. Analogous algorithms then are provided for computing rate-distortion functions and constrained channel capacities. The algorithms apply both to discrete and to continuous alphabet channels or sources. In addition, a formalization of the theory of channel capacity in the presence of constraints is included. Among the examples is the calculation of close upper and lower bounds to the rate-distortion function of a binary symmetric Markov source.

On the minimum distortion of block codes for a binary symmetric source (Corresp.)

It is shown that the mixture of two codes of block length n can have a distortion strictly less than the distortion of the best possible code of block length n having the same rate. The seeming contradiction is explained by offering two interpretations for the mixture of two codes and then explaining why the bound for the performance of the best code cannot be applied directly to the mixture.

Coding the binary symmetric source with a fidelity criterion (Corresp.)

A procedure is defined for coding the binary symmetric first-order Markov source with the probability of error fidelity criterion. Since the rate-distortion function <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R(d)</tex> for this source has not yet been formulated, the coding performance provides an apparently tight upper bound for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R(d)</tex> .