Decoder Side Information Research Articles

Secure and Private Distributed Source Coding With Private Keys and Decoder Side Information

The distributed source coding problem is extended by positing that noisy measurements of a remote source are the correlated random variables that should be reconstructed at another terminal. We consider a secure and private distributed lossy source coding problem with two encoders and one decoder such that (i) all terminals noncausally observe a noisy measurement of the remote source; (ii) a private key is available to each legitimate encoder and all private keys are available to the decoder; (iii) rate-limited noiseless communication links are available between each encoder and the decoder; (iv) the amount of information leakage to an eavesdropper about the correlated random variables is defined as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">secrecy</i> leakage, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">privacy</i> leakage is measured with respect to the remote source; and (v) two passive attack scenarios are considered, where a strong eavesdropper can access both communication links and a weak eavesdropper can choose only one of the links to access. Inner and outer bounds on the rate regions defined under secrecy, privacy, communication, and distortion constraints are derived for both passive attack scenarios. When one or both sources should be reconstructed reliably, the rate region bounds are simplified.

Private Key and Decoder Side Information for Secure and Private Source Coding.

We extend the problem of secure source coding by considering a remote source whose noisy measurements are correlated random variables used for secure source reconstruction. The main additions to the problem are as follows: (1) all terminals noncausally observe a noisy measurement of the remote source; (2) a private key is available to all legitimate terminals; (3) the public communication link between the encoder and decoder is rate-limited; and (4) the secrecy leakage to the eavesdropper is measured with respect to the encoder input, whereas the privacy leakage is measured with respect to the remote source. Exact rate regions are characterized for a lossy source coding problem with a private key, remote source, and decoder side information under security, privacy, communication, and distortion constraints. By replacing the distortion constraint with a reliability constraint, we obtain the exact rate region for the lossless case as well. Furthermore, the lossy rate region for scalar discrete-time Gaussian sources and measurement channels is established. An achievable lossy rate region that can be numerically computed is also provided for binary-input multiple additive discrete-time Gaussian noise measurement channels.

Exponential Strong Converse for Successive Refinement with Causal Decoder Side Information.

We consider the k-user successive refinement problem with causal decoder side information and derive an exponential strong converse theorem. The rate-distortion region for the problem can be derived as a straightforward extension of the two-user case by Maor and Merhav (2008). We show that for any rate-distortion tuple outside the rate-distortion region of the k-user successive refinement problem with causal decoder side information, the joint excess-distortion probability approaches one exponentially fast. Our proof follows by judiciously adapting the recently proposed strong converse technique by Oohama using the information spectrum method, the variational form of the rate-distortion region and Hölder’s inequality. The lossy source coding problem with causal decoder side information considered by El Gamal and Weissman is a special case () of the current problem. Therefore, the exponential strong converse theorem for the El Gamal and Weissman problem follows as a corollary of our result.

Channel Code Using Constrained-Random-Number Generator Revisited

A construction of a channel code by using a source code with decoder side information is introduced. The encoder and decoder pair of any source code can be used for the construction. Constrained-random-number generators, which generate random numbers satisfying a condition specified by a function and its value, are used to construct stochastic encoders and decoders. The result suggests that we can divide the channel coding problem into the problems of channel encoding and source decoding with side information.

A low-complexity and efficient encoder rate control solution for distributed residual video coding

Existing encoder rate control (ERC) solutions have two technical limitations that prevent them from being widely used in real-world applications. One is that encoder side information (ESI) is required to be generated which increases the complexity at the encoder. The other is that rate estimation is performed at bit plane level which incurs computation overheads and latency when many bit planes exist. To achieve a low-complexity encoder, we propose a new ERC solution that combines an efficient encoder block mode decision (EBMD) for the distributed residual video coding (DRVC). The main contributions of this paper are as follows: 1) ESI is not required as our ERC is based on the analysis of the statistical characteristics of the decoder side information (DSI); 2) a simple EBMD is introduced which only employs the values of residual pixels at the encoder to classify blocks into Intra mode, Skip mode, and WZ mode; 3) an ERC solution using pseudo-random sequence scrambling is proposed to estimate rates for all WZ blocks at frame level instead of at bit plane level, i.e., only one rate is estimated; and 4) a quantization-index estimation algorithm (QIEA) is proposed to solve the problem of rate underestimation. The simulation results show that the proposed solution is not only low complex but also efficient in both the block mode decision and the rate estimation. Also, as compared to DISCOVER system and the state-of-the-art ERC solution, our solution demonstrates a competitive rate-distortion(RD)performance. Due to maintain the low-complexity nature of the encoder and have good RD performance, we believe that our ERC solution is promising in practice.

Secure Cooperative Source-Coding With Side Information at the Eavesdropper

In the secure cooperative source-coding problem that we consider, Encoder 1 and Decoder 1 observe, respectively, a pair of independent identically distributed (i.i.d.) correlated sources and wish to communicate the first source to Decoder 2 subject to a distortion constraint. Encoder 1 sends a message to both the decoders over a public nonsecure channel, and then Decoder 1—the helper—sends a message just to Decoder 2 over a secure channel. An eavesdropper, which has access to memoryless side-information that is correlated with the first source, intercepts the message sent by Encoder 1 and based on both wishes to minimize its uncertainty about the first source. The secrecy measure of the system is the equivocation of the first source at the eavesdropper. We determine the rates equivocation region for this problem assuming that conditioned on the side-information of Decoder 1 the eavesdropper’s side-information is independent of the first source. To that end, we derive an inner and an outer bound that depend explicitly on the proportional rate that Encoder 1 spends in the cooperation process. Subsequently, we evaluate this region for the case of a doubly symmetric binary source and Hamming distortion.

Residual-Quantization Based Code Design for Compressing Noisy Sources With Arbitrary Decoder Side Information

This paper considers noisy Wyner–Ziv coding (WZC), in which a remote noisy source is compressed with side information at the decoder only. The decoder side information is not limited to being Gaussian, thus this noisy WZC problem cannot be transformed into the conventional one tailored for noiseless sources. A new coding structure, named the residual-quantization (RQ) based noisy WZC, is proposed to solve the problem. This scheme explicitly constructs the theoretical auxiliary random variable to facilitate optimal reconstruction of the noisy source. In this two-stage encoder, the noisy source is quantized twice and the quantization error (residue) of the first stage is the input of the second stage. By sending only the quantization index of the second stage to the decoder, the corresponding code rate can theoretically approach the noisy WZC bound. Moreover, the RQ-based noisy WZC is implemented using graph-based codes. The main challenge is that it is necessary to design a codebook that is simultaneously good for source and channel coding for the first stage quantization, since this quantization code also acts as a channel code at the decoder. This problem is solved by constructing a low-density parity check (LDPC) code with edge degrees optimized for channel coding, and enhancing its performance for source coding by using a modified reinforced belief-propagation quantization algorithm. Simulation results show that the noisy Wyner-Ziv bounds can be practically approached by our implementation. In addition, the proposed implementation offers more flexibility in the code rates compared with the existing practical designs, making it more suitable for emerging applications such as fronthaul compression.

Performance Analysis of Network Coding With Virtual Overhearing in Wireless Networks

Due to the broadcast nature of wireless channels, overhearing plays an important role in interflow network coding in that it serves as a way to acquire the side information for the decoding of network-coded packets. Since overhearing only takes effect in the one-hop neighborhood of nodes, it, in turn, restricts the identification of network coding opportunities. That is, if the side information for packet decoding cannot be overheard, no coding opportunity can be identified. Hence, in this paper, we aim to relax such a restriction to further explore network coding opportunities. Specifically, we propose a virtual overhearing (VOH) technique to enable network coding when the side information cannot be obtained from traditional overhearing previously mentioned. VOH allows a node to obtain a packet sent by another node that can be two or more hops away. As a result, VOH can serve as a supplementary way for side information acquirement, thus still enabling network coding between network flows when traditional overhearing is not possible to obtain the needed side information. The additional coding opportunities brought by VOH can be translated into additional performance gains. We first analyze network coding with VOH in a simple topology and show that with VOH, network capacity could be improved by over 12% (in some situations, network throughput could be improved by 50%). Then, we conduct simulations in NS-2 to show the advantage of VOH.

On Zero-Delay Source-Channel Coding

In this paper, we study the zero-delay source-channel coding problem, and specifically the problem of obtaining the vector transformations that optimally map between the m-dimensional source space and the k-dimensional channel space, under a given transmission power constraint and for the mean square error distortion. We first study the functional properties of this problem and show that the objective is concave in the source and noise densities and convex in the density of the input to the channel. We then derive the necessary conditions for optimality of the encoder and decoder mappings. A well known result in information theory pertains to the linearity of optimal encoding and decoding mappings in the scalar Gaussian source and channel setting, at all channel signal-to-noise ratios (CSNRs). In this paper, we study this result more generally, beyond the Gaussian source and channel, and derive the necessary and sufficient condition for linearity of optimal mappings, given a noise (or source) distribution, and a specified power constraint. We also prove that the Gaussian source-channel pair is unique in the sense that it is the only source-channel pair for which the optimal mappings are linear at more than one CSNR values. Moreover, we show the asymptotic linearity of optimal mappings for low CSNR if the channel is Gaussian regardless of the source and, at the other extreme, for high CSNR if the source is Gaussian, regardless of the channel. Our numerical results show strict improvement over prior methods. The numerical approach is extended to the scenario of source-channel coding with decoder side information. The resulting encoding mappings are shown to be continuous relatives of, and in fact subsume as special case, the Wyner-Ziv mappings encountered in digital distributed source coding systems.

Zero-Delay and Causal Single-User and Multi-User Lossy Source Coding with Decoder Side Information

We consider zero-delay, single-user, and multi-user source coding with an average distortion constraint and decoder side information. The zero-delay constraint translates into causal (sequential) encoder and decoder pairs as well as the use of instantaneous codes. For the single-user setting, we show that optimal performance is attained by time sharing at most two scalar encoder-decoder pairs, that use zero-error side information codes. Side information look-ahead is shown to be useless in this setting. Furthermore, we show that even without delay constraints, if either the encoder or decoder are restricted a priori to be scalar, the performance loss cannot be compensated by the other component, which can be scalar as well without further loss. Finally, we show that the multi-terminal source coding problem can be solved in the zero-delay regime and the rate-distortion region is provided.

List Decoding for Arbitrarily Varying Broadcast Channels With Receiver Side Information

In this paper, the discrete memoryless arbitrarily varying broadcast channel (AVBC) with receiver side information is studied and its random code and deterministic code capacity regions are derived for the average error criterion. In addition, it is analyzed for deterministic list codes and it is shown that the corresponding list capacity region displays a behavior, which is similar to Ahlswede's famous dichotomy result for the single-user arbitrarily varying channel: it either equals the random code capacity region or otherwise has an empty interior. This is characterized in terms of list sizes at the receivers and an appropriate concept of symmetrizability for the AVBC with receiver side information. The scenario studied here is motivated by the broadcast phase of bidirectional relaying, where a half-duplex relay node establishes a bidirectional communication between two other nodes using a decode-and-forward protocol. The relay decodes the messages both nodes have sent in the initial multiple access phase and broadcasts a re-encoded composition of them in the succeeding broadcast phase. Then, the broadcast phase corresponds to the AVBC with receiver side information, which differs from the classical broadcast channel, since both receivers can exploit their own messages from the previous phase as side information for decoding.

Error-resilient multi-view video coding using Wyner-Ziv techniques

In this paper, a Wyner-Ziv (WZ) coding based error-resilient scheme is proposed for multi-view video transmission over error-prone channels. At the encoder, the key frames of the odd views are protected by WZ encoding to generate the auxiliary bit-stream alongside the multi-view video coded bit-stream. At the decoder, error-concealed multi-view decoded frames are used as the side information (SI) for WZ decoding. Based on the study on the characteristics of multi-view video coding (MVC) and the propagating behavior of channel errors, a recursive model to estimate the transmission distortion is developed in the transform domain, in which the channel-induced distortion takes into consideration both motion and disparity compensation. With the proposed model, we propose a rate control strategy for WZ encoding to infer the minimum bit rate so as to correct the SI errors. The WZ bit rate estimation method exploits the correlation between the original bit-planes and the SI bit-planes as well as the bit-plane interdependency. Extensive experimental results show that the proposed error-resilient scheme outperforms Reed Solomon based forward error correction method by about 1.1 dB and outperforms the adaptive intra refresh algorithm by approximately 1.6 dB at the packet loss rate 10 %.

Distributed Compression of Linear Functions: Partial Sum-Rate Tightness and Gap to Optimal Sum-Rate

We consider the problem of distributed compression of the difference Z = Y1 - cY2 of two jointly Gaussian sources Y1 and Y2 under an MSE distortion constraint D on Z. The rate region for this problem is unknown if the correlation coefficient ρ and the weighting factor c satisfy cρ > 0. Inspired by Ahlswede and Han's scheme for the problem of distributed compression of the modulo-2 sum of two binary sources, we first propose a hybrid random-structured coding scheme that is capable of saving the sum-rate over both the random quantize-and-bin (QB) coding scheme and Krithivasan and Pradhan's structured lattice coding scheme. The main idea is to use a random coding component in the first layer to adjust the source correlation so that the structured coding component in the second layer can be more efficient with the outputs from the first layer as decoder side information. We then provide a new sum-rate lower bound for the problem in hand by connecting it to the Gaussian two-terminal source coding problem with covariance matrix distortion constraint. Our lower bound not only improves existing bounds in many cases, but also allows us to prove sum-rate tightness of the QB scheme when c is either relatively small or large and D is larger than some threshold. Furthermore, our lower bound enables us to show that our new hybrid scheme performs within two b/s from the optimal sum-rate for all values of ρ, c, and D.

Codeword averaged density evolution for distributed joint source and channel coding with decoder side information

The authors consider applying the systematic low-density parity-check codes with the parity based approach to the lossless (or near lossless) distributed joint source channel coding (DJSCC) with the decoder side information for the non-uniform sources over the asymmetric memoryless transmission channel. By using an equivalent channel coding model, which consists of two parallel subchannels: a correlation and a transmission sub-channel, respectively, they derive the codeword averaged density evolution (DE) for the DJSCC with the decoder side information for the asymmetrically correlated non-uniform sources over the asymmetric memoryless transmission channel. A new code ensemble definition of the irregular codes is introduced to distinguish between the source and the parity variable nodes, respectively. Extensive simulations demonstrate the effectiveness of the codeword averaged DE.

Adaptive reversible data hiding with pyramidal structure

In this paper, we propose an adaptive algorithm for reversible data hiding by employing the characteristics and pyramidal relationships of original images. The major goal of reversible data hiding is to keep the reversibility of algorithm. By use of the pyramidal structure to explore the inherent characteristics of original images, regions with different smoothness levels can be determined, and then data hiding can be performed adaptively with the pre-determined threshold for balancing the output image quality and embedding capacity. On the one hand, larger capacity can be hidden into smoother regions with limited degradation of output image quality. On the other hand, the size of location map, which serves as the side information for keeping reversibility, can be reduced for embedding into smoother or less smooth regions of original image. By carefully manipulating difference values between layers in pyramidal structure, secret information can effectively be embedded. With our method, we observe better performances over relating methods with enhanced image quality, the more embedding capacity, and comparable amount of side information for decoding. More importantly, the reversibility of our method is guaranteed, meaning that original image and secret information can both be perfectly recovered at the decoder. Simulation results demonstrate that proposed method in this paper outperforms those in conventional algorithms.

Lossless Coding for Distributed Streaming Sources

Distributed source coding is traditionally viewed in a block coding context wherein all source symbols are known in advance by the encoders. However, many modern applications to which distributed source coding ideas are applied, are better modeled as having streaming data. In a streaming setting, source symbol pairs are revealed to separate encoders in real time and need to be reconstructed at the decoder with subject to some tolerable end-to-end delay. In this paper, a causal sequential random binning encoder is introduced and paired with maximum likelihood (ML) and universal decoders. The latter uses a novel weighted empirical suffix entropy decoding rule. We derive a lower bounds on the error exponent with delay for each decoder. We also provide upper bounds for the special case of streaming with decoder side information and discuss when upper and lower bounds match. We show that both ML and universal decoders achieve the same (positive) error exponents for all rate pairs inside the Slepian-Wolf achievable rate region. The dominant error events in streaming are different from those in block-coding and result in different exponents. Because the sequential random binning scheme is also universal over delays, the resulting code eventually reconstructs every source symbol correctly with probability one.

Distributed Joint Source and Channel Coding with Low-Density Parity-Check Codes

Low-density parity-check (LDPC) codes with the parity-based approach for distributed joint source channel coding (DJSCC) with decoder side information is described in this paper. The parity-based approach is theoretical limit achievable. Different edge degree distributions are used for source variable nodes and parity variable nodes. Particularly, the codeword-averaged density evolution (CADE) is presented for asymmetrically correlated nonuniform sources over the asymmetric memoryless transmission channel. Extensive simulations show that the splitting of variable nodes can improve the coding efficiency of suboptimal codes and lower the error floor.

Decoder side information generation techniques in Wyner-Ziv video coding: a review

In Wyner-Ziv (WZ) video coding, decoder side information (SI) takes a key role in a WZ video codec among other building blocks. In this paper, we review the decoder SI generation techniques according to the information utilized in the SI generation process. Specifically, motion compensated extrapolation based approaches, motion compensated interpolation based approaches, and hash-based decoder motion estimation approaches in which auxiliary information is sent by the encoder side. We also review the approaches for decoder SI refinement. The review will help us get insight in the current development of the topic and understand how new and better decoder SI generation approaches can be designed.

Side Information Refinement for Transform domain Distributed Video Coding

A new side information refinement method for transform domain distributed video coding (DVC) is proposed. In DVC, side information is the frame constructed from key frames which are intra coded. It is utilized as the initial estimate of source symbols of Wyner-Ziv coded frame which is not transmitted to the decoder. Errors in the estimated frames are to be corrected by the decoder using the parity bits which are the only information received from the encoder. Quality of decoder generated side information is one of the most important factors deciding the overall performance of the decoder. A new refinement method of improving the quality of side information of the DVC decoder is proposed. The side information frame is refined at the end of each transform band decoding stage followed by a conventional bi-directional motion estimation and compensation technique. Proposed side information refinement technique consequently improves the quality of final reconstructed pictures.

Hierarchy-based reversible data hiding

In this paper, we propose a new method for reversible data hiding by employing the hierarchical relationships of original images. There are many parameters for accessing the performances of reversible data hiding algorithms, including the output image quality, the hiding capacity, and the overhead for decoding. Considering the ease of implementation and the little overhead needed for decoding, we employ modification of difference values between pixels by using histogram-based scheme with extensions to pyramidal structure by utilizing inherent characteristics of original images. By doing so, global and local characteristics of original images can be utilized for hiding more capacity with acceptable quality of output image. With our method, better performances can be obtained with enhanced image quality, the more embedding capacity, and comparable amount of side information for decoding. More importantly, the reversibility of our method is guaranteed, meaning that original image and hidden message can both be perfectly recovered at the decoder. Simulation results demonstrate that proposed method in this paper outperforms those in conventional algorithms.