Encoding finite sources with ranked probabilities

Abstract

The problem of designing a code for a source with at most N symbols and ranked probabilities is investigated. The alphabet size is considered as a random variable which takes values in { 1 , 2 , … , N } with probability 1 / N . The average distribution whose associated Huffman code is the best choice, according to the Minimum Average criterion, is derived. Some properties of the average distribution are studied and it is shown that a simple fixed-length code with codewords of length ⌈ log N ⌉ has at most 2.67 bits of additional redundancy with respect to the optimum code. A simple and efficient suboptimal code is proposed whose redundancy is less than 0.2 for N ⩾ 7 . It is shown that the fixed-length code is a suitable code even when small (or large) alphabet sizes are more probable.