Fast Progressive Training of Mixture Models for Model Selection

Abstract

Finite Mixture Models are flexible models with varying uses such as density estimation, clustering, classification, modeling heterogeneity, model averaging, and handling missing data. One of the prerequisites of using mixture models is the a priori knowledge of the number of mixture components so that the Expectation Maximization (EM) algorithm can learn the maximum likelihood parameters of the mixture model. However, the number of mixing components is often unknown and determining the number of mixture components has been a central problem in mixture modelling. Thus, mixture modelling is often a two-stage process of determining the number of mixture components and then estimating the parameters of the mixture model. This paper proposes a fast, search-based model selection algorithm for mixture models using progressive merging of mixture components. The paper also proposes a data driven, fast approximation of the Kullback-Leibler (KL) divergence as a criterion to merge the mixture components. The proposed methodology is used in mixture modelling of two chromosomal aberration datasets showing that model selection is efficient and effective.