Quantising for minimum information loss

Abstract

A random variable pair consisting of a continuous random vector (the observation or feature) vector and of a discrete random variable (the class) is considered. The authors report on the design of a machine able to accept as input the observation of, and present as output an approximation to, the conditional probability of the classes given the observation. More precisely, they deal with the design of a histogram-type approximation with variable cell size and shape. In this approach, the cells are the Voronoi regions of a nearest neighbour vector quantiser, and the position of code vectors (i.e. the size and the shape of the cells) is designed in such a way that the information loss caused by quantisation is minimised.