EBEM: An Entropy-based EM Algorithm for Gaussian Mixture Models

Abstract

In this paper we address the problem of estimating the parameters of a Gaussian mixture model. Although the EM algorithm yields the maximum-likelihood solution it requires a careful initialization of the parameters and the optimal number of kernels in the mixture may be unknown beforehand. We propose a criterion based on the entropy of the pdf (probability density function) associated to each kernel to measure the quality of a given mixture model. A novel method for estimating Shannon entropy based on Entropic Spanning Graphs is developed and a modification of the classical EM algorithm to find the optimal number of kernels in the mixture is presented. We test our algorithm in probability density estimation, pattern recognition and color image segmentation.