Gauss mixture vector quantization

Abstract

Gauss mixtures are a popular class of models in statistics and statistical signal processing because they can provide good fits to smooth densities, because they have a rich theory, and because they can be well estimated by existing algorithms such as the EM (expectation maximization) algorithm. We here extend an information theoretic extremal property for source coding from Gaussian sources to Gauss mixtures using high rate quantization theory and extend a method originally used for LPC (linear predictive coding) speech vector quantization to provide a Lloyd clustering approach to the design of Gauss mixture models. The theory provides formulas relating minimum discrimination information (MDI) for model selection and the mean squared error resulting when the MDI criterion is used in an optimized robust classified vector quantizer. It also provides motivation for the use of Gauss mixture models for robust compression systems for general random vectors.