Deterministic annealing EM algorithm

Abstract

This paper presents a deterministic annealing EM (DAEM) algorithm for maximum likelihood estimation problems to overcome a local maxima problem associated with the conventional EM algorithm. In our approach, a new posterior parameterized by `temperature' is derived by using the principle of maximum entropy and is used for controlling the annealing process. In the DAEM algorithm, the EM process is reformulated as the problem of minimizing the thermodynamic free energy by using a statistical mechanics analogy. Since this minimization is deterministically performed at each temperature, the total search is executed far more efficiently than in the simulated annealing. Moreover, the derived DAEM algorithm, unlike the conventional EM algorithm, can obtain better estimates free of the initial parameter values. We also apply the DAEM algorithm to the training of probabilistic neural networks using mixture models to estimate the probability density and demonstrate the performance of the DAEM algorithm.