Deterministic Annealing for Entropy-Constrained Vector Quantizer Design

Abstract

Typical vector quantizer (VQ) design algorithms, such as generalized Lloyd methods, are susceptible to local minima. In addition, for entropy-constrained VQ, it is not obvious what the right number of codevectors is for any given source and target rate. One method that has shown promising results for solving both problems is deterministic annealing, where randomness is added to the encoding process, the uncertainty of which is gradually reduced until a deterministic coder is found. This method has been used for fixed-rate and entropy-constrained unstructured VQ, as well as for a variety of structurally constrained VQs. For the unstructured entropy-constrained case, we generalize the existing methods to include new cost functions, and we argue that, depending on the chosen cost function, some splitting criteria are more suitable than others. We give analytical advantages and disadvantages to several cost functions and several splitting criteria, then give performance evaluations by testing on synthetic and natural sources.