Accelerated Expectation Maximization Algorithm Research Articles

Convergence analysis of an accelerated expectation‐maximization algorithm for ill‐posed integral equations

The maximum‐likelihood expectation‐maximization (EM) algorithm has attracted considerable interest in single‐photon emission computed tomography, because it produces superior images in addition to be being flexible, simple, and allowing a physical interpretation. However, it often needs a large number of calculations because of the algorithm's slow rate of convergence. Therefore, there is a large body of literature concerning the EM algorithm's acceleration. One of the accelerated means is increasing an overrelaxation parameter, whereas we have not found any analysis in this method that would provide an immediate answer to the questions of the convergence. In this paper, our main focus is on the continuous version of an accelerated EM algorithm based on Lewitt and Muehllenner. We extend their conclusions to the infinite‐dimensional space and interpret and analyze the convergence of the accelerated EM algorithm. We also obtain some new properties of the modified algorithm. Copyright © 2015 John Wiley & Sons, Ltd.

Improving the vector $$\varepsilon $$ ε acceleration for the EM algorithm using a re-starting procedure

The expectation---maximization (EM) algorithm is a popular algorithm for finding maximum likelihood estimates from incomplete data. However, the EM algorithm converges slowly when the proportion of missing data is large. Although many acceleration algorithms have been proposed, they require complex calculations. Kuroda and Sakakihara (Comput Stat Data Anal 51:1549---1561, 2006) developed the $$\varepsilon $$?-accelerated EM algorithm which only uses the sequence of estimates obtained by the EM algorithm to get an accelerated sequence for the EM sequence but does not change the original EM sequence. We find that the accelerated sequence often has larger values of the likelihood than the current estimate obtained by the EM algorithm. Thus, in this paper, we try to re-start the EM iterations using the accelerated sequence and then generate a new EM sequence that increases its speed of convergence. This algorithm has another advantage of simple implementation since it only uses the EM iterations and re-starts the iterations by an estimate with a larger likelihood. The re-starting algorithm called the $$\varepsilon $$?R-accelerated EM algorithm can further improve the EM algorithm and the $$\varepsilon $$?-accelerated EM algorithm in the sense of that it can reduces the number of iterations and computation time.

Multi-user Detection Based on the Accelerated EM Algorithm

Multi-user detection (MUD) is one standard of 3G, which can effectively reduce the multiple access interfer- ence (MAI) and increase the system capacity. The Expectation-Maximization (EM) iterative algorithm is commonly used in recent years for missing data, which could be applied to MUD system. But the EM algorithm has a fatal weakness that its slow convergence speed. The new accelerated EM Algorithm is proposed in this paper, which is based on the EM Al- gorithm and Newton-Raphson algorithm, in order to improve the convergence speed. The proposed algorithm is applied to Multi-user Detection (MUD). The simulation results show that the performance of accelerated EM algorithm is almost as good as the standard EM algorithm in Gaussian noise. The proposed algorithm has better convergence speed than standard EM algorithm.

The Co-Integrated Vector Autoregression with Errors–in–Variables

The co-integrated vector autoregression is extended to allow variables to be observed with classical measurement errors (ME). For estimation, the model is parametrized as a time invariant state-space form, and an accelerated expectation-maximization algorithm is derived. A simulation study shows that (i) the finite-sample properties of the maximum likelihood (ML) estimates and reduced rank test statistics are excellent (ii) neglected measurement errors will generally distort unit root inference due to a moving average component in the residuals, and (iii) the moving average component may–in principle–be approximated by a long autoregression, but a pure autoregression cannot identify the autoregressive structure of the latent process, and the adjustment coefficients are estimated with a substantial asymptotic bias. An application to the zero-coupon yield-curve is given.

Acceleration of the EM algorithm using the vector epsilon algorithm

The expectation---maximization (EM) algorithm is a very general and popular iterative computational algorithm to find maximum likelihood estimates from incomplete data and broadly used to statistical analysis with missing data, because of its stability, flexibility and simplicity. However, it is often criticized that the convergence of the EM algorithm is slow. The various algorithms to accelerate the convergence of the EM algorithm have been proposed. The vector ? algorithm of Wynn (Math Comp 16:301---322, 1962) is used to accelerate the convergence of the EM algorithm in Kuroda and Sakakihara (Comput Stat Data Anal 51:1549---1561, 2006). In this paper, we provide the theoretical evaluation of the convergence of the ?-accelerated EM algorithm. The ?-accelerated EM algorithm does not use the information matrix but only uses the sequence of estimates obtained from iterations of the EM algorithm, and thus it keeps the flexibility and simplicity of the EM algorithm.

나이브베이즈 문서분류시스템을 위한 선택적샘플링 기반 EM 가속 알고리즘

본 논문은 온라인 전자문서환경에서 전통적 베이지안 통계기반 문서분류시스템의 분류성능을 개선하기 위해 EM(Expectation Maximization) 가속 알고리즘을 접목한 방법을 제안한다. 기계학습 기반의 문서분류시스템의 중요한 문제 중의 하나는 양질의 학습문서를 확보하는 것이다. EM 알고리즘은 소량의 학습문서집합으로 베이지안 문서분류 알고리즘의 성능을 높이는데 활용된다. 그러나 EM 알고리즘은 최적화 과정에서 느린 수렴성과 성능 저하 현상을 나타내는데, EM 알고리즘의 기본 가정을 따르지 않는 온라인 전자문서환경에서 특히 그러하다. 제안 기법의 주요 아이디어는 전통적 EM 알고리즘을 개선하기 위해 불확정성도 기반 선택적 샘플링 기법을 활용한 것이다. 성능평가를 위해 Reuter-21578 문서집합을 사용하여, 제안 알고리즘의 빠른 수렴성을 보이고 전통적 베이지안 알고리즘의 분류 정확성을 향상시켰음을 보인다. This paper presents a new method of significantly improving conventional Bayesian statistical text classifier by incorporating accelerated EM(Expectation Maximization) algorithm. EM algorithm experiences a slow convergence and performance degrade in its iterative process, especially when real online-textual documents do not follow EM's assumptions. In this study, we propose a new accelerated EM algorithm with uncertainty-based selective sampling, which is simple yet has a fast convergence speed and allow to estimate a more accurate classification model on Naive Bayesian text classifier. Experiments using the popular Reuters-21578 document collection showed that the proposed algorithm effectively improves classification accuracy.

A maximum-likelihood reconstruction algorithm for tomographic gamma-ray nondestructive assay

A new tomographic reconstruction algorithm for nondestructive assay with high-resolution gamma-ray spectroscopy (HRGS) is presented. The reconstruction problem is formulated using a maximum-likelihood approach in which the statistical structure of both the gross and continuum measurements used to determine the full-energy response in HRGS is modeled. An accelerated expectation-maximization algorithm is used to determine the optimal solution. The algorithm is applied to safeguards and environmental assays of large samples (for example, 55 gal drums) in which high continuum levels caused by Compton scattering are routinely encountered. Details of the implementation of the algorithm and a comparative study of the algorithm's performance are presented.

An evaluation of the accelerated expectation maximization algorithms for single-photon emission tomography image reconstruction

We previously reported that brain single-photon emission tomography (SPET) images could be improved by using an attenuation coefficient map constructed with transmission data and the iterative expectation maximization (EM) algorithm. However, the conventional EM algorithm (CEM) typically requires 30-80 iterations to provide acceptable results, limiting its clinical applicability. Several methods have been proposed to accelerate the EM algorithm. The purpose of this study was to search for a practical method for accelerating the EM algorithm. The methods investigated here include the accelerated EM algorithm (ACEM) using additive correction, ACEM using multiplicative correction, and Tanaka's filtered iterative reconstruction method (FIR). These methods were assessed by simulated SPET studies of a phantom incorporating nonuniform attenuation and by reference to clinical brain SPET data. In the simulation studies, the above methods were evaluated by using three parameters (root mean square error, log likelihood value, and contrast recovery coefficient); the results showed that FIR had an advantage over other methods in terms of all parameters. The results obtained using the clinical data demonstrated that FIR could reconstruct acceptable images in only five iterations. These results show that FIR offers significant advantages over CEM or other ACEMs, indicating that FIR can make the EM algorithm practical for clinical use in SPET.