Quadratic extrapolation for accelerating convergence of the EM fixed point problem

Abstract

Various extrapolation methods have been applied to accelerate convergence of the EM algorithm. These methods are easy to implement, since they work only with EM basic iterations. In other words, auxiliary quantities, such as gradient and hessian, are not needed. In this paper, we define a new family of iterative schemes based on nonlinear extrapolation methods. It is shown that these strategies can accelerate convergence of the EM algorithm much more stably than competing methods. They are extremely general in the sense that they can accelerate any linearly convergent fixed point iterative method, and hence, any EM-type algorithm. A randomly relaxed version is also deduced and numerically tested.