Why vector quantizers outperform scalar quantizers on stationary memoryless sources

Abstract

This question is frequently asked by newcomers to vector quantization (VQ), who recognize that, in this case, its ability to exploit correlation is of no use. An interesting approach is to a compare k-dimensional VQ with rate R to the k-dimensional product quantizer (PQ) induced by applying a scalar quantizer (SQ) with rate R to k successive source samples. It is then evident that one advantage of VQ is that its cells are more spherical than those of the PQ, which are rectangular. Another is that the points of the VQ are better distributed. Indeed, it is often thought that the PQ distributes points in a cubic fashion, whereas the VQ matches its distribution to the source; e.g. spherical for a Gaussian density. Using asymptotic quantization theory, we show that aside from the rectangularity of the induced PQ's cells, the shortcoming of SQ's is not that they are incapable of inducing a PQ with an optimal density. Rather, the structure of the PQ links the and cell shapes in a way that causes the best SQ to be a compromise between that which induces the best and that which induces the best cell shapes. Consequently, the optimum SQ suffers a loss and a cell shape loss. For large rates, we find formulas for these and evaluate them in the Gaussian and Laplacian cases. For example, in the Gaussian case, relative to high-dimensional VQ, an SQ has a 1.88 dB point density loss, a 1.53 dB cubic loss and a 0.94 dB oblongitis loss.