Variable Rate Vector Quantization

Abstract

All of the lossy compression schemes considered thus far have been fixed rate schemes, systems that send a fixed number of bits for each input symbol or block or, equivalently, a fixed number of bits per unit time. The only variable rate schemes considered were the noiseless codes of Chapter 9 where channel codewords of differing sizes were assigned to different input vectors. In the noiseless code case the basic strategy is fairly obvious: by assigning short codewords to highly probable input vectors and long codewords to rare input vectors in an intelligent fashion, one can reduce the average number of bits sent. A similar strategy can be applied in vector quantization: instead of sending a fixed number, R, of bits for each input vector, one might have a code that assigns a variable number of bits to different codewords. For example, one could use several bits to describe codewords for active input vectors such as plosives in speech or edges in an image and use fewer bits for codewords describing low activity input vectors such as silence in speech or background in images. Unlike the lossless codes, however, it is not clear that here the best strategy is to assign shorter words to more probable vectors and longer words to less probable since the overall goal is now to minimize average distortion and hence both probability and distortion must be taken into account.