Wyner-Ziv Problem Research Articles

Low-Delay Digital and Hybrid Digital/Analog Schemes for Wyner-Ziv Problem Over Gaussian Broadcast Channel

Low-Delay Digital and Hybrid Digital/Analog Schemes for Wyner-Ziv Problem Over Gaussian Broadcast Channel

Nested Lattice Coding With Algebraic Encoding and Geometric Decoding

Nested Lattice Coding (NLC) provides a generalized solution to Wyner-Ziv problems. Codebook of NLC is termed as the label-set. We find a significant difference between the label-set strictly satisfying coset's property and that predominantly used in existing NLC-related works. We name the former as Algebraic Label-Set (ALS), the latter as Geometric Label-Set (GLS), and present their enumeration strategies for practical implementation. Apart from random and coset binning, we name the enumeration of GLS as “geometric binning”. By applying ALS for encoding and GLS for decoding, we propose a refined NLC scheme called Nested Lattice Coding with Algebraic Encoding and Geometric Decoding (NLC-AC-GD). NLC-AC-GD has the same decoding reliability and higher compression rate compared with the traditional NLC scheme.

Strong Converse Using Change of Measure Arguments

The strong converse for a coding theorem shows that the optimal asymptotic rate possible with vanishing error cannot be improved by allowing a fixed error. Building on a method introduced by Gu and Effros for centralized coding problems, we develop a general and simple recipe for proving strong converse that is applicable for distributed problems as well. Heuristically, our proof of strong converse mimics the standard steps for proving a weak converse, except that we apply those steps to a modified distribution obtained by conditioning the original distribution on the event that no error occurs. A key component of our recipe is the replacement of the hard Markov constraints implied by the distributed nature of the problem with a soft information cost using a variational formula introduced by Oohama. We illustrate our method by providing a short proof of the strong converse for the Wyner-Ziv problem and strong converse theorems for interactive function computation, common randomness and secret key agreement, and the wiretap channel; the latter three strong converse problems were open prior to this work.

Successive Wyner-Ziv Coding for the Binary CEO Problem Under Logarithmic Loss

The $L$-link binary Chief Executive Officer (CEO) problem under logarithmic loss is investigated in this paper. A quantization splitting technique is applied to convert the problem under consideration to a $(2L-1)$-step successive Wyner-Ziv (WZ) problem, for which a practical coding scheme is proposed. In the proposed scheme, low-density generator-matrix (LDGM) codes are used for binary quantization while low-density parity-check (LDPC) codes are used for syndrome generation; the decoder performs successive decoding based on the received syndromes and produces a soft reconstruction of the remote source. The simulation results indicate that the rate-distortion performance of the proposed scheme can approach the theoretical inner bound based on binary-symmetric test-channel models.

Source Coding in Networks With Covariance Distortion Constraints

We consider a source coding problem with a network scenario in mind, and formulate it as a remote vector Gaussian Wyner-Ziv problem under covariance matrix distortions. We define a notion of minimum for two positive-definite matrices based on which we derive an explicit formula for the rate-distortion function (RDF). We then study the special cases and applications of this result. We show that two well-studied source coding problems, i.e. remote vector Gaussian Wyner-Ziv problems with mean-squared error and mutual information constraints are in fact special cases of our results. Finally, we apply our results to a joint source coding and denoising problem. We consider a network with a centralized topology and a given weighted sum-rate constraint, where the received signals at the center are to be fused to maximize the output SNR while enforcing no linear distortion. We show that one can design the distortion matrices at the nodes in order to maximize the output SNR at the fusion center. We thereby bridge between denoising and source coding within this setup.

The Likelihood Encoder for Lossy Compression

A likelihood encoder is studied in the context of lossy source compression. The analysis of the likelihood encoder is based on the soft-covering lemma. It is demonstrated that the use of a likelihood encoder together with the soft-covering lemma yields simple achievability proofs for classical source coding problems. The cases of the point-to-point rate-distortion function, the rate-distortion function with side information at the decoder (i.e. the Wyner-Ziv problem), and the multi-terminal source coding inner bound (i.e. the Berger-Tung problem) are examined in this paper. Furthermore, a non-asymptotic analysis is used for the point-to-point case to examine the upper bound on the excess distortion provided by this method. The likelihood encoder is also related to a recent alternative technique using properties of random binning.

Zero-Delay Joint Source-Channel Coding Using Hybrid Digital-Analog Schemes in the Wyner-Ziv Setting

This paper studies zero-delay joint source-channel coding (JSCC) of Gaussian sources over additive white Gaussian noise (AWGN) channels in the Wyner-Ziv scenario, that is, when there is side information about the source available only to the receiver. The proposed encoding scheme is based on hybrid digital-analog (HDA) transmission in a zero-delay fashion. Specifically, to achieve zero-delay, after applying scalar quantization to the source, the properly scaled analog information (quantization error) is superimposed on the scaled digital information (quantized source), and then transmitted. At the receiver side, several reconstruction schemes are analyzed. It is shown that all the schemes, when optimized, are superior to existing uncoded, digital, or hybrid transmission methods. It is also shown that the proposed HDA transmission may result in increased robustness to channel and/or side information mismatch compared to uncoded transmission. Observing that the bandwidth-matched Wyner-Ziv problem is intimately related to the coding scenario without side information but with bandwidth expansion factor 2, the proposed scheme is applied to the latter problem. It is shown that this application results in a better performance than the well-known inverse spiral mapping.

Distributed and Cascade Lossy Source Coding With a Side Information “Vending Machine”

Source coding with a side information “vending machine” is a recently proposed framework in which the statistical relationship between the side information and the source, instead of being given and fixed as in the classical Wyner-Ziv problem, can be controlled by the decoder. This control action is selected by the decoder based on the message encoded by the source node. Unlike conventional settings, the message can thus carry not only information about the source to be reproduced at the decoder, but also control information aimed at improving the quality of the side information. In this paper, the analysis of the tradeoffs between rate, distortion, and cost associated with the control actions is extended from the previously studied point-to-point setup to two basic multiterminal models. First, a distributed source coding model is studied, in which two encoders communicate over rate-limited links to a decoder, whose side information can be controlled. The control actions are selected by the decoder based on the messages encoded by both source nodes. For this setup, inner bounds are derived on the rate-distortion-cost region for both cases in which the side information is available causally and noncausally at the decoder. These bounds are shown to be tight under specific assumptions, including the scenario in which the sequence observed by one of the nodes is a function of the source observed by the other and the side information is available causally at the decoder. Then, a cascade scenario in which three nodes are connected in a cascade and the last node has controllable side information is also investigated. For this model, the rate-distortion-cost region is derived for general distortion requirements and under the assumption of causal availability of side information at the last node.

Perfectly Secure Encryption of Individual Sequences

In analogy to the well-known notion of finite-state compressibility of individual sequences, due to Lempel and Ziv, we define a similar notion of “finite-state encryptability” of an individual plain-text sequence, as the minimum asymptotic key rate that must be consumed by finite-state encrypters so as to guarantee perfect secrecy in a well-defined sense. Our main basic result is that the finite-state encryptability is equal to the finite-state compressibility for every individual sequence. This is in parallelism to Shannon's classical probabilistic counterpart result, asserting that the minimum required key rate is equal to the entropy rate of the source. However, the redundancy, defined as the gap between the upper bound (direct part) and the lower bound (converse part) in the encryption problem, turns out to decay at a different rate (in fact, much slower) than the analogous redundancy associated with the compression problem. We also extend our main theorem in several directions, allowing: 1) availability of side information (SI) at the encrypter/decrypter/eavesdropper, 2) lossy reconstruction at the decrypter, and 3) the combination of both lossy reconstruction and SI, in the spirit of the Wyner-Ziv problem.

Multi-Mode Nested Quantization in Presence of Uncertain Side Information and Feedback

In this paper, the problem of uniform quantization for the Wyner-Ziv (WZ) problem in presence of uncertain side information at the encoder is addressed. The uncertain side information problem arises in data compression with side information (SI) when the joint probability distribution of source and SI is not (fully) known at the encoder. The uncertainty is due to the time varying nature of the dependency of source and SI in settings such as wireless sensor networks and distributed video coding. In this paper, a solution is proposed which consists of a multi-mode WZ encoder in conjunction with a carefully designed feedback scheme from the decoder. Each encoder mode is associated with a uniform nested quantizer. In this framework, the quantization parameters, the feedback scheme, and the source coding rate are jointly optimized to minimize the average rate or distortion. Two distinct schemes with adaptive or fixed transmission rate are proposed. Simulation results are provided which demonstrate the effectiveness of the presented solutions and assess the effect of different design parameters including the number of modes (feedback rate) on the performance.

Reliability in Source Coding With Side Information

We study error exponents for source coding with side information. Both achievable exponents and converse bounds are obtained for the following two cases: lossless source coding with coded information and lossy source coding with full side information (Wyner-Ziv). These results recover and extend several existing results on source-coding error exponents and are tight in some circumstances. Our bounds have a natural interpretation as a two-player game between nature and the code designer, with nature seeking to minimize the exponent and the code designer seeking to maximize it. In the Wyner-Ziv problem, our analysis exposes a tension in the choice of test channel with the optimal test channel balancing two competing error events. The Gaussian and binary-erasure cases are examined in detail.

Wyner-Ziv Coding Based on Multidimensional Nested Lattices

Distributed source coding addresses the compression of correlated sources without communication links among them. This paper is concerned with the Wyner-Ziv problem: coding of an information source with side information available only at the decoder in the form of a noisy version of the source. Both the problems of theoretical analysis and code design are addressed in the framework of multi-dimensional nested lattice coding. For theoretical analysis, accurate computation of the rate-distortion function is given under the high-resolution assumption, and a new upper bound using the derivative of the theta series is derived. For practical code design, several low-complexity techniques are proposed. Compared to the existing Slepian-Wolf coded nested quantization for Wyner-Ziv coding based on one or two-dimensional lattices, our proposed multi-dimensional lattice coding can offer better performance at arguably lower complexity, since it does not require the second stage of Slepian-Wolf coding.

Efficient On-Line Schemes for Encoding Individual Sequences With Side Information at the Decoder

We present adaptive on-line schemes for lossy encoding of individual sequences, under the conditions of the Wyner-Ziv (WZ) problem. In the first part of this paper, a set of fixed-rate scalar source codes with zero delay is presented. We propose a randomized on-line coding scheme, which achieves asymptotically (and with high probability), the performance of the best source code in the set, uniformly over all source sequences. Efficient algorithms for implementing this scheme for small and large sets of encoders are presented. In the second part of this work, we generalize our results to the case of variable-rate coding. A set of variable-rate scalar source codes is presented. This time, the performance is measured by the Lagrangian Cost (LC), which is defined as a weighted sum of the distortion and the length of the encoded sequence. Efficient algorithms for implementing the generalized on-line coding scheme are presented. We then consider the special case of lossless variable-rate coding. An on-line scheme which uses Huffman codes is presented. We show that this scheme can be implemented efficiently using the same graphic methods from the first part. Finally, combining the results from former sections, we build a generalized efficient algorithm for a structured set of variable-rate encoders.

Source Coding With a Side Information “Vending Machine”

We study source coding in the presence of side information, when the system can take actions that affect the availability, quality, or nature of the side information. We begin by extending the Wyner-Ziv problem of source coding with decoder side information to the case where the decoder is allowed to choose actions affecting the side information. We then consider the setting where actions are taken by the encoder, based on its observation of the source. Actions may have costs that are commensurate with the quality of the side information they yield, and an overall per-symbol cost constraint may be imposed. We characterize the achievable tradeoffs between rate, distortion, and cost in some of these problem settings. Among our findings is the fact that even in the absence of a cost constraint, greedily choosing the action associated with the “best” side information is, in general, suboptimal. A few examples are worked out.

Distributed Rate-Distortion With Common Components

We describe a scheme for rate-distortion with distributed encoding in which the sources to be compressed contain a common component. We show that this scheme is optimal in some situations and that it strictly improves upon existing schemes, which do not make full use of common components. This establishes that independent quantization followed by independent binning is not optimal for the two-encoder problem with a distortion constraint on one source. We also show that independent quantization and binning is suboptimal for the three-encoder problem in which the goal is to reproduce one of the sources losslessly. This provides a counterexample that is fundamentally different from one provided earlier by Korner and Marton. The proofs rely on the binary analogue of the entropy power inequality and the existence of a rate loss for the binary symmetric Wyner-Ziv problem.

Hash property and coding theorems for sparse matrices and maximum-likelihood coding

The aim of this paper is to prove the achievability of rate regions for several coding problems by using sparse matrices (with logarithmic column degree) and maximum-likelihood (ML) coding. These problems are the Gel'fand-Pinsker problem, the Wyner-Ziv problem, and the one-helps-one problem (source coding with partial side information at the decoder). To this end, the notion of a hash property for an ensemble of functions is introduced and it is proved that an ensemble of q-ary sparse matrices satisfies the hash property. Based on this property, it is proved that the rate of codes using sparse matrices and ML coding can achieve the optimal rate.

A Universal Scheme for Wyner–Ziv Coding of Discrete Sources

We consider the Wyner-Ziv (WZ) problem of lossy compression where the decompressor observes a noisy version of the source, whose statistics are unknown. A new family of WZ coding algorithms is proposed and their universal optimality is proven. Compression consists of sliding-window processing followed by Lempel-Ziv (LZ) compression, while the decompressor is based on a modification of the discrete universal denoiser (DUDE) algorithm to take advantage of side information. The new algorithms not only universally attain the fundamental limits, but also suggest a paradigm for practical WZ coding. The effectiveness of our approach is illustrated with experiments on binary images, and English text using a low complexity algorithm motivated by our class of universally optimal WZ codes.

Coding and Common Reconstruction

This work studies problems of source and joint source-channel coding under the requirement that the encoder can produce an exact copy of the compressed source constructed by the decoder. This requirement, termed here as the common reconstruction constraint (CR), is satisfied automatically in rate-distortion theory for single sources. However, in the common formulation of problems of lossy source coding with side information at the decoder (the Wyner-Ziv problem), distributed source coding, and joint source-channel coding for networks, the destination can exploit the information it receives in a manner that cannot be exactly reproduced at the sender side. Some applications, like the transmission of sensitive medical information, may require that both sides-the sender and the receiver-will share a common version of the compressed data, for the purpose of future discussions or consulting. The purpose of this work is to study the implications of CR constraints on the achievable rates in scenarios of lossy source coding and lossy transmission of sources. Three problems are examined: source coding with side information at the decoder, simultaneous transmission of data and state over state-dependent channels, and joint source-channel coding for the degraded broadcast channel. Single-letter characterizations of the optimal performance are developed for these problems, under corresponding CR constraints. Implications of this constraint on problems of joint source-channel coding in networks are discussed.

Joint Wyner–Ziv/Dirty-Paper Coding by Modulo-Lattice Modulation

The combination of source coding with decoder side information (the Wyner-Ziv problem) and channel coding with encoder side information (the Gel'fand-Pinsker problem) can be optimally solved using the separation principle. In this work, we show an alternative scheme for the quadratic-Gaussian case, which merges source and channel coding. This scheme achieves the optimal performance by applying a modulo-lattice modulation to the analog source. Thus, it saves the complexity of quantization and channel decoding, and remains with the task of ldquoshapingrdquo only. Furthermore, for high signal-to-noise ratio (SNR), the scheme approaches the optimal performance using an SNR-independent encoder, thus it proves for this special case the feasibility of universal joint source-channel coding.

Channel Simulation With Quantum Side Information

We study and solve the problem of classical channel simulation with quantum side information at the receiver. This is a generalization of both the classical reverse Shannon theorem, and the classical-quantum Slepian-Wolf problem. The optimal noiseless communication rate is found to be reduced from the mutual information between the channel input and output by the Holevo information between the channel output and the quantum side information. Our main theorem has two important corollaries. The first is a quantum generalization of the Wyner-Ziv problem: rate-distortion theory with quantum side information. The second is an alternative proof of the tradeoff between classical communication and common randomness distilled from a quantum state. The fully quantum generalization of the problem considered is quantum state redistribution. Here the sender and receiver share a mixed quantum state and the sender wants to transfer part of her state to the receiver using entanglement and quantum communication. We present outer and inner bounds on the achievable rate pairs.