Recent Extensions to the EM Algorithm

Abstract

Abstract Two recent extensions to the EM algorithm have made it even more powerful and applicable in practice. Both extensions follow the theme of EM by using repeated complete-data computations to handle incomplete data. The Supplemented EM or SEM algorithm (Meng and Rubin, 1991a) enables users of EM to calculate the incomplete-data asymptotic variance-covariance matrix associated with the maximum likelihood estimate obtained by EM, using only the computer code for EM and for the complete-data asymptotic variance-covariance matrix. The Expectation/Conditional Maximization or ECM algorithm (Meng and Rubin, 1991b) replaces the M-step of EM with a set of conditional maximization (CM) steps, and thus can eliminate undesirable nested iterations with EM when its M-step is not in closed form. SEM and ECM can also be combined into a Supplemented ECM (SECM) algorithm for computing the incomplete-data asymptotic variance-covariance matrix for maximum likelihood estimates found by ECM. In this paper, we provide a general description of these new techniques, as well as brief discussions of their relevance in Bayesian computations and their relationships with the popular Gibbs sampler (Geman and Geman, 1984), and its extension, GIBS.