Distortion-rate theory for individual sequences

Abstract

For every individual infinite sequence <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">u</tex> we define a distortion-rate function <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d(R|u)</tex> which is shown to be an asymptotically attainable lower bound on the distortion that can be achieved for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">u</tex> by any finite-state encoder which operates at a fixed output information rate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</tex> . This is done by means of a coding theorem and its converse. No probabilistic characterization of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">u</tex> is assumed. The coding theorem demonstrates the existence of {\em universal} encoders which are asymptotically optimal for every infinite sequence over a given finite alphabet. The transmission of individual sequences via a noisy channel with a capacity <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</tex> is also investigated. It is shown that, for every given sequence <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">u</tex> and any finite-state encoder, the average distortion with respect to the channel statistics is lower bounded by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d(C|u)</tex> . Furthermore <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d(C|u)</tex> is asymptotically attainable.