Universally Attainable Error Exponents for Rate-Distortion Coding of Noisy Sources

Abstract

Consider the problem of rate-constrained reconstruction of a finite-alphabet discrete memoryless signal X/sup n/=(X/sub 1/,...,X/sub n/), based on a noise-corrupted observation sequence Z/sup n/, which is the finite-alphabet output of a discrete memoryless channel (DMC) whose input is X/sup n/. Suppose that there is some uncertainty in the source distribution, in the channel characteristics, or in both. Equivalently, suppose that the distribution of the pairs (X/sub i/,Z/sub i/), rather than completely being known, is only known to belong to a set /spl Theta/. Suppose further that the relevant performance criterion is the probability of excess distortion, i.e., letting X/spl circ//sup n/(Z/sup n/) denote the reconstruction, we are interested in the behavior of P/sub /spl theta//(/spl rho/(X/sup n/,X/spl circ//sup n/(Z/sup n/))>d/sub /spl theta//), where /spl rho/ is a (normalized) block distortion induced by a distortion measure and P/sub /spl theta// denotes the probability measure corresponding to the case where (X/sub i/,Z/sub i/)/spl sim//spl theta/, /spl theta//spl isin//spl Theta/. Since typically this probability will either not decay at all or do so at an exponential rate, it is the rate of this decay which we focus on. More concretely, for a given rate R /spl ges/ 0 and a family of distortion levels {d/sub /spl theta//}/sub /spl theta//spl isin//spl Theta//, we are interested in families of exponential levels {I/sub /spl theta//}/sub /spl theta//spl isin//spl Theta// which are achievable in the sense that for large n there exist rate-R schemes satisfying -1/nlog P/sub /spl theta// (/spl rho/(X/sup n/, X/spl circ//sup n/(Z/sup n/)) > d/sub /spl theta//) /spl ges/ I/sub /spl theta//, for all /spl theta/ /spl isin/ /spl Theta/. Our main result is a complete single-letter characterization of achievable levels {I/sub /spl theta//}/sub /spl theta//spl isin//spl Theta// per any given triple (/spl Theta/,R,{d/sub /spl theta//}/sub /spl theta//spl isin//spl Theta//). Equipped with this result, we later turn to addressing the question of the right choice of {I/sub /spl theta//}/spl theta//spl isin//spl Theta/. Relying on methodology recently put forth by Feder and Merhav in the context of the composite hypothesis testing problem, we propose a competitive minimax approach for the choice of these levels and apply our main result for characterizing the associated key quantities. Subsequently, we apply the main result to characterize optimal performance in a Neyman-Pearson-like setting, where there are two possible noise-corrupted signals. In this problem, the goal of the observer of the noisy signal, rather than having to determine which of the two it is (as in the hypothesis testing problem), is to reproduce the underlying clean signal with as high a fidelity as possible (e.g., lowest number of symbol errors when distortion measure is Hamming), under the assumption that one source is active, while operating at a limited information rate R and subject to a constraint on the fidelity of reconstruction when the other source is active. Finally, we apply our result to characterize a sufficient condition for the source class /spl Theta/ to be universally encodable in the sense of the existence of schemes attaining the optimal distribution-dependent exponent, simultaneously for all sources in the class. This condition was shown in an earlier work to suffice for universality in expectation.