A simple method for calculating the rate distortion function of a source with an unknown parameter

Abstract

The rate distortion function R( D) measures the minimum information rate of a source required to be transmitted at a fidelity level D. Although Blahut developed an elegant algorithm to calculate R( D) for discrete memoryless sources, computing R( D) for other types of sources is still very difficult. In this paper, we study the computation of R( D) for discrete sources with an unknown parameter which takes values in a continuous space. According to the well known ergodic decomposition theorem, a non-ergodic stationary source can be represented by a class of parameterized ergodic subsources with a known prior distribution. Based on this theory, a source matching approach and a simple algorithm is presented for computational purposes. The algorithm is shown to be convergent and efficient. In order to see the performance of this simple algorithm, we consider a special class of binary symmetric first-order Markov sources which has been previously studied. R( D) is computed over this class of sources and compared with the bound developed in previous work by Gray and Berger. The example shows that the algorithm is very efficient and produces results close to Gray and Berger's bound. Other examples further demonstrate the efficiency of the algorithm.