Hamming Distortion Measure Research Articles

A Lower Bound on the Sum Rate of Multiple Description Coding With Symmetric Distortion Constraints

We derive a single-letter lower bound on the minimum sum rate of multiple description coding with symmetric distortion constraints. For the binary uniform source with the erasure distortion measure or Hamming distortion measure, this lower bound can be evaluated with the aid of certain minimax theorems. A similar minimax theorem is established in the quadratic Gaussian setting, which is further leveraged to analyze the special case where the minimum sum rate subject to two levels of distortion constraints (with the second level imposed on the complete set of descriptions) is attained; in particular, we determine the minimum achievable distortions at the intermediate levels.

LDGM-Based Multiple Description Coding for Finite Alphabet Sources

This work presents an LDGM-based practical successive coding scheme for the multiple description (MD) problem for finite alphabet sources. The scheme, which targets the Zhang-Berger (ZB) rate-distortion region, is shown to be asymptotically optimal with joint typicality encoding, while as a practical encoding solution a message passing algorithm is adopted. We further discuss in more detail the application of the coding scheme in three cases of the MD problem with the Hamming distortion measure: 1) no excess sum-rate for binary sources, 2) successive refinement, and 3) no excess marginal rate for the uniform binary source. In the no excess sum-rate case some progress is made in the characterization of fundamental limits by deriving the analytical expression of the distortion region for general binary sources, and of the auxiliary variables needed to achieve its boundary. The exact expression of the Zhang-Berger upper bound to the central distortion is also provided for the case of no excess marginal rate for the uniform binary source. The proposed LDGM-based coding scheme is tested in practice for all three aforementioned cases. The experimental results show very good performance, demonstrating its ability to approach the theoretical rate-distortion limits or the available upper bounds.

Robust Multiresolution Coding

In multiresolution coding a source sequence is encoded into a base layer and a refinement layer. The refinement layer, constructed using a conditional codebook, is in general not decodable without the correct reception of the base layer. By relating multiresolution coding with multiple description coding, we show that it is in fact possible to construct multiresolution codes in certain ways so that the refinement layer alone can be used to reconstruct the source to achieve a nontrivial distortion. As a consequence, one can improve the robustness of the existing multiresolution coding schemes without sacrificing the efficiency. Specifically, we obtain an explicit expression of the minimum distortion achievable by the refinement layer for arbitrary finite alphabet sources with Hamming distortion measure. Experimental results show that the information-theoretic limits can be approached using a practical robust multiresolution coding scheme based on low-density generator matrix codes.

Side-Information Scalable Source Coding

We consider the problem of side-information scalable (SI-scalable) source coding, where the encoder constructs a two-layer description, such that the receiver with high quality side information will be able to use only the first layer to reconstruct the source in a lossy manner, while the receiver with low quality side information will have to receive both layers in order to decode. We provide inner and outer bounds to the rate-distortion (R-D) region for general discrete memoryless sources. The achievable region is tight when either one of the decoders requires a lossless reconstruction, and when the distortion measures are degraded and deterministic. Furthermore, the gap between the inner and the outer bounds can be bounded by certain constants when the squared error distortion measure is used. The notion of perfect scalability is introduced, for which necessary and sufficient conditions are given for sources satisfying a mild support condition. Using SI-scalable coding and successive refinement Wyner-Ziv coding as basic building blocks, we provide a complete characterization of the rate-distortion region for the important quadratic Gaussian source with multiple jointly Gaussian side informations, where the side information quality is not necessarily monotonic along the scalable coding order. A partial result is provided for the doubly symmetric binary source under the Hamming distortion measure when the worse side information is a constant, for which one of the outer bounds is strictly tighter than the other.

On the Suboptimality of Linear Lossy Codes

This letter reveals that linear lossy codes cannot attain the rate-distortion function in general, even if the source is binary i.i.d. and the distortion is measured by the Hamming distortion measure.

On Successive Refinement of the Binary Symmetric Markov Source

We show that for every <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n ≫ 2$</tex> the standard <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$ n$</tex> th-order approximation <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$R_n(D)$</tex> , to the rate-distortion function of the binary-symmetric Markov source (BSMS) is not successively refineable under the Hamming distortion measure in an open interval of the form <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$D_n !≪ ! D !≪ !1/2 ! = !D_max $</tex> .

On the joint source-channel coding error exponent for discrete memoryless systems

We investigate the computation of Csisza/spl acute/r's bounds for the joint source-channel coding (JSCC) error exponent E/sub J/ of a communication system consisting of a discrete memoryless source and a discrete memoryless channel. We provide equivalent expressions for these bounds and derive explicit formulas for the rates where the bounds are attained. These equivalent representations can be readily computed for arbitrary source-channel pairs via Arimoto's algorithm. When the channel's distribution satisfies a symmetry property, the bounds admit closed-form parametric expressions. We then use our results to provide a systematic comparison between the JSCC error exponent E/sub J/ and the tandem coding error exponent E/sub T/, which applies if the source and channel are separately coded. It is shown that E/sub T//spl les/E/sub J//spl les/2E/sub T/. We establish conditions for which E/sub J/>E/sub T/ and for which E/sub J/=2E/sub T/. Numerical examples indicate that E/sub J/ is close to 2E/sub T/ for many source-channel pairs. This gain translates into a power saving larger than 2 dB for a binary source transmitted over additive white Gaussian noise (AWGN) channels and Rayleigh-fading channels with finite output quantization. Finally, we study the computation of the lossy JSCC error exponent under the Hamming distortion measure.

On the Error Exponent of Trellis Source Coding

In this paper, we develop a single-letter lower bound on the error exponent for the problem of trellis source coding. We demonstrate that for the case of a binary source with the Hamming distortion measure, and for rates close to the rate-distortion curve, this bound is superior to Marton's block-coding exponent, for the same computational complexity.

On Properties of Rate-Reliability-Distortion Functions

Some important properties of the rate-reliability-distortion function of discrete memoryless source (DMS) are established. For the binary source and Hamming distortion measure this function is derived and analyzed. Even that elementary case suffices to show the nonconvexity of the rate-reliability-distortion function in the reliability (error exponent) argument.

On Successive Refinement for the Wyner–Ziv Problem

Achievable rates are characterized for successive refinement (SR) in the Wyner-Ziv scenario, namely, in the presence of correlated side information (SI) at the receivers. In this setting, the encoder is assumed to operate in two stages, where the first corresponds to relatively low rate and high distortion, and the second, comprising a refinement code on top of the first code, is aimed at reproduction at reduced distortion. Both decoders (for low rate/high distortion and for high rate/low distortion) are equipped with SI streams, correlated to the source, but unavailable to the encoder. Furthermore, it is assumed that the decoder that receives the higher rate bitstream, i.e., the additional refinement bits, accesses also SI of better quality than that of the lower resolution decoder. By better quality, we mean that the source and the SI are modeled together as a stochastically degraded joint source, where the encoded symbols, the SI at the refinement stage, and the SI at the initial (coarse) stage, form a Markov chain in this order. For a memoryless joint process (that includes the source to be encoded and its instantaneously correlated SI streams), necessary and sufficient conditions are furnished, in terms of single-letter formulas, for the achievability of a pair of rates, corresponding to two given distortion levels. Special attention is devoted to the degenerate, but important, case where the two SI streams, at the two decoders, are identical. For this case, conditions are provided for successive refinability in the sense of the existence of codes that asymptotically achieve the Wyner-Ziv rate-distortion function, simultaneously at both distortion levels. In this context, the doubly symmetric binary source (with the Hamming distortion measure) and the jointly Gaussian source (with the squared error distortion measure) are successively refinable in the Wyner-Ziv setting. It is demonstrated that a source that is not successively refinable in the ordinary sense (i.e., without SI) may become successively refinable in the presence of SI at the decoders.

Scanning and prediction in multidimensional data arrays

The problem of sequentially scanning and predicting data arranged in a multidimensional array is considered. We introduce the notion of a scandictor, which is any scheme for the sequential scanning and prediction of such multidimensional data. The scandictability of any finite (probabilistic) data array is defined as the best achievable expected performance on that array. The scandictability of any (spatially) stationary random field on /spl Zopf//sup m/ is defined as the limit of its scandictability on finite boxes (subsets of /spl Zopf//sup m/), as their edges become large. The limit is shown to exist for any stationary field, and essentially be independent of the ratios between the box dimensions. Fundamental limitations on scandiction performance in both the probabilistic and the deterministic settings are characterized for the family of difference loss functions. We find that any stochastic process or random field that can be generated autoregressively with a maximum-entropy innovation process is optimally scandicted the way it was generated. These results are specialized for cases of particular interest. The scandictability of any stationary Gaussian field under the squared-error loss function is given a single-letter expression in terms of its spectral measure and is shown to be attained by the raster scan. For a family of binary Markov random fields (MRFs), the scandictability under the Hamming distortion measure is fully characterized.

A comment on "A rate of convergence result for a universal D-semifaithful code"

In the above paper (see ibid., vol.39, no.3, p.813-20, 1993) Yu and Speed propose a universal pointwise D-semifaithful code whose expected compression ratio, for discrete memoryless sources, approaches the rate-distortion function at a rate O(n/sup -1/ log n). They also conjecture that this is the fastest achievable convergence rate for pointwise D-semifaithful codes. In this correspondence, we use a simple extension of Kraft's inequality and prove that this conjecture is true, at least for the Hamming distortion measure. >

Multiple description source coding with no excess marginal rate

Multiple description source coding concerns situations in which the transmission of the source information is distributed over two data streams at rates R/sub 1/ and R/sub 2/, respectively. When both data streams are received, the decoder uses the combined data at R/sub 1/+R/sub 2/ to reconstruct the source information with average distortion d/sub 0/. If a communication breakdown prevents one of the data streams from reaching the receiver, the decoder has to base its reconstruction solely on the available data at either R/sub 1/ or R/sub 2/. This results in a higher distortion of either d/sub 1/ or d/sub 2/, respectively. The region /spl Rscr/ of all achievable quintuples (R/sub 1/, R/sub 2/, d/sub 0/, d/sub 1/, d/sub 2/) has been determined in the so-called no excess rate sum case defined by imposing the requirement R/sub 1/+R/sub 2/=R(d/sub 0/), where R(/spl middot/) is the rate-distortion function of the source. The case with excess sum, characterized by R/sub 1/+R/sub 2/>R(d/sub 0/), is challenging. We study in this paper a special case of it in which the requirements R/sub t/=R(d/sub t/), t=1, 2, are imposed; we refer to this as the no excess marginal rate case. The lower and upper bounds on d/sub 0/ we obtain are separated by only a tiny gap when evaluated for a binary equiprobable source and the Hamming distortion measure. >

Tree encoding of stationary ergodic sources with a fidelity criterion

Tree source coding theorems with a single letter fidelity criterion are proved for stationary ergodic sources using tree codes with a fixed branch length. The higher-than-exponential convergence of distortions is shown for binary symmetric sources and Hamming distortion measure when an excess in rate is allowed.

Speech Encoding by a Stack Algorithm

A metric-first tree coding algorithm, similar to the stack algorithm, is used to encode voiced speech. From experimental evidence, it is shown how to optimize the algorithm's performance with respect to the algorithm's storage, execution time, and number of tree branches searched per symbol released as output. For each of these, the optimal parameterizing of the algorithm differs markedly. Similarities are pointed out between our results for speech and earlier theoretical results for the binary i.i.d, source with Hamming distortion measure. Comparisons to the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</tex> algorithm are made.

A 2-cycle algorithm for source coding with a fidelity criterion

Since the birth of rate-distortion theory with the landmark paper of Shannon, research has centered on the extension of his theory to new situations and on the calculation of rate-distortion functions. Comparatively little has been done toward developing efficient algorithms for encoding information sources at rates near Shannon's rate-distortion limit. This paper reports one such algorithm, the 2-cycle algorithm, used with randomly chosen tree codes. In analyzing the 2-cycle algorithm, we present the first theoretical analysis of a realizable algorithm for tree coding of sources with a fidelity criterion. The algorithm may prove to be the first practical method of coding whose design actually derives from rate-distortion theory. Analysis proceeds by bounding methods related to difference equations and branching processes. Upper bounds on average distortion and encoding work are obtained and the latter are shown bounded as long as the coding rate exceeds Shannon's rate-distortion function. Of particular interest is the stack-searched branching process bound of Section IV. The two cycles of the algorithm are described fully in terms of expectations and probabilities, and recursions are listed to compute these quantities. Concluding the paper is numerical analysis, by theory and by simulation, with respect to the binary i.i.d, source and Hamming distortion measure. These numerical results are sufficient to optimize the algorithm over its free parameters.