EM Iteration Research Articles

Applying the Cox proportional hazards model for analysis of latency data with interval censoring

The latency time of an infectious disease is defined as the time from infection to disease onset. This paper applies the proportional hazards model to estimate the effect of covariates on latency when the time of disease onset is exact or right-censored but the time of infection is interval-censored. We use a Monte Carlo EM algorithm to estimate parameters of the joint distribution of infection times and latency times. At each EM iteration, exact infection times are multiply imputed from the density determined by the parameters of the infection and latency time distributions. The methodology is tested using a simulation study and is applied to data from a cohort of haemophiliacs with HIV disease.

On computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation

We address the problem of computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation. These are the largest eigenvalue and its associated eigenvector for the Jacobian of the EM operator at a maximum likelihood estimate, which are important for assessing convergence in iterative simulation. An estimate of the largest fraction of missing information is available from the EM iterates; this is often adequate since only a few figures of accuracy are needed. In some instances the EM iteration also gives an estimate of the worst linear function. We show that improved estimates can be essential for proper inference. In order to obtain improved estimates efficiently, we use the power method for eigencomputation. Unlike eigenvalue decomposition, the power method computes only the largest eigenvalue and eigenvector of a matrix, it can take advantage of a good eigenvector estimate as an initial value and it can be terminated after only a few figures of accuracy are achieved. Moreover, the matrix products needed in the power method can be computed by extrapolation, obviating the need to form the Jacobian of the EM operator. We give results of simulation studies on multivariate normal data showing that this approach becomes more efficient as the data dimension increases than methods that use a finite-difference approximation to the Jacobian, which is the only general-purpose alternative available.

Image oscillation reduction and convergence acceleration for OS-EM reconstruction [PET imaging

Investigates the use of two approaches to reduce the image oscillation of OS-EM (ordered subset expectation maximization) reconstruction that is due to the inconsistencies among different partial subsets of the projection measurements (the sinogram) when considering as a group. One approach pre-processes the sinogram to make it satisfy a sinogram consistency condition. The second approach takes the average of the intermediary images (i.e. it smoothes image values over sub-iterations). Both approaches were found to be capable of reducing the image oscillation, and a combination of both was most effective. With these approaches, the convergence of OS-EM reconstruction is further improved. For computer-simulated data and real PET data, a single iteration of this new OS-EM reconstruction was shown to yield images comparable to those with 80 EM iterations.

A hybrid EM/Gauss-Newton algorithm for maximum likelihood in mixture distributions

A faster alternative to the EM algorithm in finite mixture distributions is described, which alternates EM iterations with Gauss-Newton iterations using the observed information matrix. At the expense of modest additional analytical effort in obtaining the observed information, the hybrid algorithm reduces the computing time required and provides asymptotic standard errors at convergence. The algorithm is illustrated on the two-component normal mixture.

An EM algorithm for estimation in Markov-modulated Poisson processes

It has recently been shown that the maximum-likelihood estimate of the parameters of a Markov-modulated Poisson process is consistent. In this paper we present an EM algorithm for computing such estimates and discuss how it may be implemented. We also compare it to the Nelder-Mead downhill simplex algorithm for some numerical examples, and the results show that the number of iterations the EM algorithm requires to converge is in general smaller than the number of likelihood evaluations required by the downhill simplex algorithm. An EM iteration is more complicated than a likelihood evaluation, though, and thus also implementation aspects must be taken into account to determine the efficiencies of the algorithms.

Maximum Likelihood Location Estimation of Wideband Sources Using the EM Algorithm

Approximate maximum likelihood and related estimation techniques for source location estimation are investigated. An extended model of the spectral density matrix of the sensor array output for coherent sources (multipath propagation) is introduced. We show that the EM algorithm can be successfully appplied for this special case of practical importance. In contrast to [2], the EM iteration scheme is derived by exploiting that the distribution type of the finite Fourier- transformed sensor outputs belongs to the exponential family. This approach requires neither the additional noise parameters used in [2] nor an alternating step by step optimization of spectral and source location parameters. Finally, we investigate the co-called approximate dual maximum likelihood estimate.

A Smoothed Em Approach to Indirect Estimation Problems, with Particular Reference to Stereology and Emission Tomography

SUMMARY There are many practical problems where the observed data are not drawn directly from the density g of real interest, but rather from another distribution derived from g by the application of an integral operator. The estimation of g then entails both statistical and numerical difficulties. A natural statistical approach is by maximum likelihood, conveniently implemented using the EM algorithm, but this provides unsatisfactory reconstructions of g. In this paper, we modify the maximum likelihood–EM approach by introducing a simple smoothing step at each EM iteration. In our experience, this algorithm converges in relatively few iterations to good estimates of g that do not depend on the choice of starting configuration. Some theoretical background is given that relates this smoothed EM algorithm to a maximum penalized likelihood approach. Two applications are considered in detail. The first is the classical stereology problem of determining particle size distributions from data collected on a plane section through a composite medium. The second concerns the recovery of the structure of a section of the human body from external observations obtained by positron emission tomography; for this problem, we also suggest several technical improvements on existing methodology.

Maximum Likelihood Computations with Repeated Measures: Application of the EM Algorithm

Abstract The purpose of this article is to consider the use of the EM algorithm (Dempster, Laird, and Rubin 1977) for both maximum likelihood (ML) and restricted maximum likelihood (REML) estimation in a general repeated measures setting using a multivariate normal data model with linear mean and covariance structure (Anderson 1973). Several models and methods of analysis have been proposed in recent years for repeated measures data; Ware (1985) presented an overview. Because the EM algorithm is a general-purpose, iterative method for computing ML estimates with incomplete data, it has often been used in this particular setting (Dempster et al. 1977; Andrade and Helms 1984; Jennrich and Schluchter 1985). There are two apparently different approaches to using the EM algorithm in this setting. In one application, each experimental unit is observed under a standard protocol specifying measurements at each of n occasions (or under n conditions), and incompleteness implies that the number of measurements actually collected on each unit is less than the requisite n for at least some units. In this circumstance, incompleteness may be modeled if one regards the measurements actually collected as the observed data, the conceptual set of n measurements on each individual as the complete data, and the unobserved data as the missing measurements on those units with fewer than n observations. Application of the EM algorithm in this setting [referred to as “missing data” in Dempster et al. (1977) and “incomplete data” in Jennrich and Schluchter (1985)] was discussed by Orchard and Woodbury (1972), Beale and Little (1975), and Jennrich and Schluchter (1985). One drawback of this approach in the longitudinal data setting is that the multivariate model with linear mean and covariance structure does not, in general, possess closed-form solutions even with complete data (Anderson 1973; Szatrowski 1980). Thus implementing the EM algorithm requires either an iterative M step within each EM iteration or the use of a generalized EM (GEM) algorithm that requires only that the complete data likelihood be increased rather than maximized at each M step. A second drawback is that this approach requires specification of the covariates for both the observed and the missing observations. If the covariates are unknown for the missing observations, arbitrary values must be specified, which may affect the rate but not the final point of convergence (Jennrich and Schluchter 1985). The second application of the EM algorithm arises naturally when we use mixed models to analyze serial measurements. In this setting, the incomplete data are modeled quite differently. The observed data are as before, that is, the measurements actually collected on each unit. The complete data, however, consist of the observed data plus the unobservable random parameters and error terms specified in the mixed model. Thus the missing data (the random parameters and error terms) would not be viewed as data in the traditional statistical sense. Laird and Ware (1982) and Andrade and Helms (1984) took this approach. This article shows that the latter approach is more general and encompasses the missing-data approach as a special case. This result has several important applications. First, it means that EM algorithms encoded for models with random effects can also be used for multivariate normal models with arbitrary covariance structure and missing data. Second, this approach avoids specification of covariates for missing observations. Finally, use of the general formulation means that closed-form solutions for the complete data maximization will exist for a much broader class of models, enabling one to avoid use of GEM or iterations within each M step. For a certain class of multivariate growth curve models with random effects structure (Reinsel 1982), closed-form solutions exist for both ML and REML estimates of the mean and covariance parameters. Formulas for these closed-form solutions are given that are applicable whenever the solution is not on the boundary. The choice of starting values for the EM iterations is important, since the EM algorithm will not, in general, converge from arbitrary starting values to the closed-form solution (if it exists) in one iteration. Several possibilities for starting values are given. The rate of convergence of the EM algorithm is generally linear. The actual speed of convergence in two data examples is shown to depend heavily on both the actual data set and the assumed structure for the covariance matrix. We discuss two methods for accelerating convergence, which we find are most useful when the covariance matrix is assumed to have a random effects structure. When the covariance matrix is assumed to be arbitrary, the EM iterations reduce to familiar iteratively reweighted least squares (IRLS) computations. The EM algorithm has the unusual property in this setting that when all of the data are complete (no missing observations), the iterations are still IRLS, but the rate of convergence changes from linear to quadratic.

Alternative algorithms for the estimation of dynamic factor, mimic and varying coefficient regression models

This paper provides a general approach to the formulation and estimation of dynamic unobserved component models. After introducing the general model, two methods for estimating the unknown parameters are presented. Both are algorithms for maximizing the likelihood function. The first is based on the method of Scoring. The second is the EM algorithm, a derivative-free method. Each iteration of EM requires a Kalman filter and smoother followed by straightforward regression calculations. The paper suggests using the EM methods to quickly locate a neighborhood of the maximum. Scoring can then be used to pinpoint the maximum and calculate the information matrix.