Asymptotic Convergence Properties of Entropy Regularized Likelihood Learning on Finite Mixtures with Automatic Model Selection

Abstract

In finite mixture modelling, it is crucial to select the number of components for a data set. We have proposed an entropy regularized likelihood (ERL) learning principle for the finite mixtures to solve this model selection problem under regularization theory. In this paper, we further give an asymptotic analysis of the ERL learning, and find that the global minimization of the ERL function in a simulated annealing way (i.e., the regularization factor is gradually reduced to zero) leads to automatic model selection on the finite mixtures with a good parameter estimation. As compared with the EM algorithm, the ERL learning can go across the local minima of the negative likelihood and keep robust with respect to initialization. The simulation experiments then prove our theoretic analysis.KeywordsMixture ModelGaussian Mixture ModelTrue ParameterFinite MixtureLearning PrincipleThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.