Competitive and selective learning method for vector quantizer design—Equidistortion principle and its algorithm

Abstract

AbstractA new competitive learning approach for optimal vector quantizer is presented. First, it is shown that the original Competitive Learning (CL) algorithm can be derived from the two necessary conditions forming the basis of the traditional nonconnectionist vector quantizer (VQ) design algorithm called the LBG algorithm. It is then shown that the conventional conscience principle or equiprobable principle is not optimal from the standpoint of the minimization of the expected dis‐tortion. Next, a basic principle is introduced. Called the equidistortion principle (a necessary condition for optimal vector quantizers) it is derived by using Gersho's asymptotic theory. It is shown to be applicable to distributions consisting of disjoint clusters. Then a new competitive learning algorithm with a selection mechanism, called the competitive and selective learning (CSL) algorithm and based on the equidistortion principle, is proposed here. Since the selection mechanism enables the system to escape from local minima, the proposed algorithm can obtain better performance without a particular initialization procedure even when the input data clusters in a number of regions in the input vector space. In a synthetic one‐dimensional quantizer problem in which the optimum quantizer can be numerically obtained, less than 2% of the minimum distortion has been found which had not been obtained previously by con‐ventional algorithms.