Optimal Performance Theoretically Attainable Research Articles

Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources

We deal with zero-delay source coding of a vector-valued Gauss-Markov source subject to a mean-squared error (MSE) fidelity criterion characterized by the operational zero-delay vector-valued Gaussian rate distortion function (RDF). We address this problem by considering the nonanticipative RDF (NRDF) which is a lower bound to the causal optimal performance theoretically attainable (OPTA) function and operational zero-delay RDF. We recall the realization that corresponds to the optimal "test-channel" of the Gaussian NRDF, when considering a vector Gauss-Markov source subject to a MSE distortion in the finite time horizon. Then, we introduce sufficient conditions to show existence of solution for this problem in the infinite time horizon. For the asymptotic regime, we use the asymptotic characterization of the Gaussian NRDF to provide a new equivalent realization scheme with feedback which is characterized by a resource allocation (reverse-waterfilling) problem across the dimension of the vector source. We leverage the new realization to derive a predictive coding scheme via lattice quantization with subtractive dither and joint memoryless entropy coding. This coding scheme offers an upper bound to the operational zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then for "r" active dimensions of the vector Gauss-Markov source the gap between the obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1 bits/vector. We further show that it is possible when we use vector quantization, and assume infinite dimensional Gauss-Markov sources to make the previous gap to be negligible, i.e., Gaussian NRDF approximates the operational zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian sources of any finite memory under mild conditions. Our theoretical framework is demonstrated with illustrative numerical experiments.

Improved Upper Bounds to the Causal Quadratic Rate-Distortion Function for Gaussian Stationary Sources

We improve the existing achievable rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal rate-distortion function (RDF) under such distortion measure, denoted by R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it</sup> (D), for first-order Gauss-Markov processes. R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it</sup> (D) is a lower bound to the optimal performance theoretically attainable (OPTA) by any causal source code, namely R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">op</sup> (D). We show that, for Gaussian sources, the latter can also be upper bounded as R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">op</sup> (D) ≤ R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it</sup> (D) + 0.5 log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> (2πe) bits/sample. In order to analyze R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it</sup> (D) for arbitrary zero-mean Gaussian stationary sources, we introduce R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it̅</sup> (D), the information-theoretic causal RDF when the reconstruction error is jointly stationary with the source. Based upon R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it̅</sup> (D), we derive three closed-form upper bounds to the additive rate loss defined as R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it̅</sup> (D) - R(D), where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation. We then show that, for any source spectral density and any positive distortion D ≤ σ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , RU(D) can be realized by an additive white Gaussian noise channel surrounded by a unique set of causal pre-, post-, and feed- back niters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal niters and is guaranteed to converge to R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it̅</sup> (D). Finally, by establishing a connection to feedback quantization, we design a causal and a zero-delay coding scheme which, for Gaussian sources, achieves an operational rate lower than R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it̅</sup> (D) +0.254 and R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it̅</sup> (D) + 0.754 bits/sample, respectively. This implies that the OPTA among all zero-delay source codes, denoted by R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">zd</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">op</sup> (D), is upper bounded as R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">zd</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">op</sup> (D) <; R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">it̅</sup> (D) + 1-254 <; R(D) + 1.754 bits/sample.

On the Optimal Performance in Asymmetric Gaussian Wireless Sensor Networks With Fading

We study the estimation of a Gaussian source by a Gaussian wireless sensor network (WSN) where L distributed sensors transmit noisy observations of the source through a fading Gaussian multiple access channel to a fusion center. In a recent work Gastpar, [?Uncoded transmission is exactly optimal for a Simple Gaussian Sensor Network?, IEEE Trans. Inf. Theory, vol. 54, no. 11, pp. 5247-5251, Nov. 2008] showed that for a symmetric Gaussian WSN with no fading, uncoded (analog) transmission achieves the optimal performance theoretically attainable (OPTA). In this correspondence, we consider an asymmetric fading WSN in which the sensors have differing noise and transmission powers. We first present lower and upper bounds on the system's OPTA under random fading. We next focus on asymmetric networks with deterministic fading. By comparing the obtained lower and upper OPTA bounds under deterministic fading, we provide a sufficient condition for the optimality of the uncoded transmission scheme for a given power tuple \mbi P=(P 1,P 2,...,PL) . Then, allowing the sensor powers to vary under a weighted sum constraint (this includes the sum-power constraint as a special case), we obtain a sufficient condition for the optimality of uncoded transmission and provide the system's corresponding OPTA.

Analogue transmission over a two-hop gaussian cascade network

We investigate lossy transmission of two intercorrelated sources over a two-hop cascade network. A low delay analogue based joint source-channel coding scheme is proposed for the case of jointly Gaussian sources over channels with additive white Gaussian noise (AWGN), subject to a quadratic distortion constraint. For the first hop, uncoded transmission is used, with an optimal minimum mean-squared error (MMSE) decoder at the relay node that utilizes the other source as side information. For the second hop, we first perform decorrelation using Karhunen-Loeve transform, followed by dimension reduction mappings. Simulation results show that relative to the optimal performance theoretically attainable (OPTA), uncoded transmission with the optimal MMSE joint decoder for the first hop is able perform close to OPTA when the inter-correlation is relatively weak. For the second hop, result of the proposed scheme is able to come within 1.5 dB from OPTA.

Process definitions of distortion-rate functions and source coding theorems

The standard definition of the distortion-rate function involves a limit of information-tbeoretic minimizations over distributions of random vectors. Several alternative definitions, each involving a single minimization over random processes, are presented here and verified. These definitions parallel Khinchine's process definition of channel capacity, provide a new interpretation of block and nonblock source coding (with a fidelity criterion) theorems in terms of optimal stochastic codes, and provide a comparison between the optimal performance theoretically attainable (OPTA) using block and nonblock source codes. Coupling the process definitions with recently developed bounding techniques provides a new and simple proof of the block source coding theorem for ergodic sources.

Optimum pulse amplitude modulation--I: Transmitter-receiver design and bounds from information theory

Intersymbol interference and additive noise are two common sources of distortion in data transmission systems. For pulse amplitude modulation (PAM) communication links, the combination of transmitter waveform and linear receiver that minimizes the mean-squared error arising from these sources is determined. An extension to include the effects of timing jitter is performed in a companion paper. Performance characteristics of the optimal PAM systems, showing the mean-squared error versus the signal-to-noise ratio, are determined explicitly for several examples. These characteristics are compared both with those of certain suboptimal systems and with the optimal performance theoretically attainable (OPTA), derived by combining Shannon's concepts of the capacity of a channel and the rate distortion function of a source. The optimal PAM systems are seen to perform very close to the OPTA for low signal-to-noise ratios. For high signal-to-noise ratios, however, the mean-squared error of optimal PAM systems decreases as the reciprocal of the signal-to-noise ratio, but the OPTA decreases more rapidly, except for band-limited channels. The performance of PAM systems can be improved at high signal-to-noise ratios by coding techniques. One such technique, called Shannon-Cantor coding, is discussed briefly.