On the correct convergence of the EM algorithm for Gaussian mixtures

Abstract

It is well-known that the EM algorithm generally converges to a local maximum likelihood estimate. However, there have been many evidences to show that the EM algorithm can converge correctly to the true parameters as long as the overlap of Gaussians in the sample data is small enough. This paper studies this correct convergence problem asymptotically on the EM algorithm for Gaussian mixtures. It has been proved that the EM algorithm becomes a contraction mapping of the parameters within a neighborhood of the consistent solution of the maximum likelihood when the measure of average overlap among Gaussians in the original mixture is small enough and the number of samples is large enough. That is, if the initial parameters are set within the neighborhood, the EM algorithm will always converge to the consistent solution, i.e., the expected result. Moreover, the simulation results further demonstrate that this correct convergence neighborhood becomes larger as the average overlap becomes smaller.