Adaptation for non-stationary binary sources for data compression

Abstract

In static entropy coding schemes such as static Huffman coding, the data is entropy coded under a fixed probability distribution. These methods are usually referred to as two pass, since the probability distribution is often, but not always, computed from the data in a first pass. In contrast, in the dynamic or adaptive, which is sometimes called one pass, the fixed probability distribution is generally too inefficient to be acceptable. The motivation for this work is to define a quantity corresponding to the intuitive concept of speed of adaptation. With the use of Bayesian framework and the inefficiency penalty function, adaptation risk is defined, and shown to have the desired properties. Since in an adaptive scheme the probability distribution is not fixed, the adaptation risk serves also the role of the second order quantity. Hence coding under fixed adaptation risk can be thought of as the simplest generalization of coding under fixed probability distribution. Of course variable adaptation risk is a valid and interesting possibility. As applications several adaptive coding methods are presented and their adaptation risks are analyzed.