Iterative Reconstruction Algorithms Based on Cross-Entropy Minimization

Abstract

The “expectation maximization maximum likelihood” algorithm (EMML) has received considerable attention in the literature since its introduction in 1982 by Shepp and Vardi. A less well known algorithm, discovered independently in 1972 by Schmidlin (“iterative separation of sections”) and by Darroch and Ratcliff (“general-ized iterative scaling”), and rediscovered and called the “simultaneous multiplicative algebraic reconstruction technique” (SMART) in 1992, is quite similar to the EMML. Both algorithms can be derived within a framework of alternating minimization of cross-entropy distances between convex sets. By considering such a parallel development of EMML and SMART we discover that certain questions answered for SMART remain open for EMML. We also demonstrate the importance of cross-entropy (or Kullback-Leibler) distances in understanding these algorithms, as well as the usefulness of Pythagorean-like orthogonality conditions in the proofs of the results. The SMART is closely related to the “multiplicative algebraic reconstruction technique” (MART) of Gordon, Bender and Herman; we include a derivation of MART within the same alternating minimization framework and provide an elementary proof of the convergence of MART in the consistent case, extending the theorem of Lent. Some partial results on the behavior of MART in the inconsistent case are also discussed.