Existing Expectation Maximization Research Articles

EM and SAGE Algorithms for DOA Estimation in the Presence of Unknown Uniform Noise.

The existing expectation maximization (EM) and space-alternating generalized EM (SAGE) algorithms are only applied to direction of arrival (DOA) estimation in known noise. In this paper, the two algorithms are designed for DOA estimation in unknown uniform noise. Both the deterministic and random signal models are considered. In addition, a new modified EM (MEM) algorithm applicable to the noise assumption is also proposed. Next, these EM-type algorithms are improved to ensure the stability when the powers of sources are not equal. After being improved, simulation results illustrate that the EM algorithm has similar convergence with the MEM algorithm, the SAGE algorithm outperforms the EM and MEM algorithms for the deterministic signal model, and the SAGE algorithm cannot always outperform the EM and MEM algorithms for the random signal model. Furthermore, simulation results show that processing the same snapshots from the random signal model, the SAGE algorithm for the deterministic signal model can require the fewest computations.

On the parameter estimation of Box-Cox transformation cure model.

We propose an improved estimation method for the Box-Cox transformation (BCT) cure rate model parameters. Specifically, we propose a generic maximum likelihood estimation algorithm through a non-linear conjugate gradient (NCG) method with an efficient line search technique. We then apply the proposed NCG algorithm to BCT cure model. Through a detailed simulation study, we compare the model fitting results of the NCG algorithm with those obtained by the existing expectation maximization (EM) algorithm. First, we show that our proposed NCG algorithm allows simultaneous maximization of all model parameters unlike the EM algorithm when the likelihood surface is flat with respect to the BCT index parameter. Then, we show that the NCG algorithm results in smaller bias and noticeably smaller root mean square error of the estimates of the model parameters that are associated with the cure rate. This results in more accurate and precise inference on the cure rate. In addition, we show that when the sample size is large the NCG algorithm, which only needs the computation of the gradient and not the Hessian, takes less CPU time to produce the estimates. These advantages of the NCG algorithm allows us to conclude that the NCG method should be the preferred estimation method over the already existing EM algorithm in the context of BCT cure model. Finally, we apply the NCG algorithm to analyze a well-known melanoma data and show that it results in a better fit when compared to the EM algorithm.

Improved Expectation-Maximization Algorithm for Unknown Reverberant Audio-Source Separation

The problem of undecided Separating reverberant audio sources is crucial for speech and audio processing. Numerous separation strategies have been developed to solve this problem; however, all of them estimate model parameters in the time–frequency domain, resulting in permutation ambiguity and poor separation performance. Additionally, one of the main challenges with existing expectation–maximization (EM) strategies is the time needed for each iterative step to update the model parameters. In this article, we offer an enhanced EM approach that combines nonnegative matrix factorization (NMF) with time differences of arrival (TDOA) estimations while eliminating time expenditure to the EM algorithm's starting values being appropriately selected. The suggested approach avoids permutation ambiguity by using the NMF source model, and acoustic localization is accomplished by converting the TDOA. Following that, model parameters are changed to improve separation outcomes. Finally, Wiener filters are used to separate the source signals. The experimental findings indicate that the suggested algorithm outperforms current blind separation approaches in terms of source separation.

Variational Bayesian inference for the identification of FIR systems via quantized output data

For identifying the parameters in finite impulse response (FIR) systems via the quantized output, existing expectation maximization (EM) methods are involved with the intractable integrals. To overcome the above shortcoming, this paper proposes an approach referred to as explicit EM (EEM). First, we fit the quantized output data by a linear Gaussian regression distribution with some compensation parameters. As a result of the linear Gaussianity, the posterior of the excited output can be explicitly computed, so that EEM has the less computation burden than the existing EMs. Second, through optimizing these compensation parameters by resorting to variational Bayesian inference, the intrinsic distribution of the quantized output can be well fitted with guaranteed identification accuracy. Third, the linear Gaussian regression distribution is independent of the specific quantization function form, which enables EEM applicable for different quantization forms. Finally, the simulation demonstrates the feasibility and effectiveness of the proposed approach.

Underdetermined Reverberant Audio-Source Separation Through Improved Expectation–Maximization Algorithm

Underdetermined reverberant audio-source separation is an important issue in speech and audio processing. To solve this problem, many separation algorithms have been proposed, in which model parameter estimation is performed in the time–frequency domain, leading to permutation ambiguity and poor separation performance. Additionally, in the existing expectation–maximization (EM) algorithms, one of the crucial problem is that updating the model parameters at each iterative step is time-consuming. In this paper, we present an improved EM algorithm that combines nonnegative matrix factorization (NMF) and time differences of arrival (TDOA) estimation, avoiding the time consumption by properly selecting initial values of the EM algorithm. In the proposed algorithm, NMF source model is used to avoid the permutation ambiguity problem, and acoustic localization can be achieved by transforming the TDOA. Then, model parameters are updated to obtain better separation results. Finally, the source signals are separated using Wiener filters. The experimental results show that compared with existing blind separation methods, the proposed algorithm achieves better performance on source separation.

A fast collapsed data method for estimating haplotype frequencies from pooled genotype data with applications to the study of rare variants

Haplotype information could lead to more powerful tests of genetic association than single-locus analyses but it is not easy to estimate haplotype frequencies from genotype data due to phase ambiguity. The challenge is compounded when individuals are pooled together to save costs or to increase sample size, which is crucial in the study of rare variants. Existing expectation-maximization type algorithms are slow and cannot cope with large pool size or long haplotypes. We show that by collapsing the total allele frequencies of each pool suitably, the maximum likelihood estimates of haplotype frequencies based on the collapsed data can be calculated very quickly regardless of pool size and haplotype length. We provide a running time analysis to demonstrate the considerable savings in time that the collapsed data method can bring. The method is particularly well suited to estimating certain union probabilities useful in the study of rare variants. We provide theoretical and empirical evidence to suggest that the proposed estimation method will not suffer much loss in efficiency if the variants are rare. We use the method to analyze re-sequencing data collected from a case control study involving 148 obese persons and 150 controls. Focusing on a region containing 25 rare variants around the MGLL gene, our method selects three rare variants as potentially causal. This is more parsimonious than the 12 variants selected by a recently proposed covering method. From another set of 32 rare variants around the FAAH gene, we discover an interesting potential interaction between two of them.

Swift block-updating EM and pseudo-EM procedures for Bayesian shrinkage analysis of quantitative trait loci

Virtually all existing expectation-maximization (EM) algorithms for quantitative trait locus (QTL) mapping overlook the covariance structure of genetic effects, even though this information can help enhance the robustness of model-based inferences. Here, we propose fast EM and pseudo-EM-based procedures for Bayesian shrinkage analysis of QTLs, designed to accommodate the posterior covariance structure of genetic effects through a block-updating scheme. That is, updating all genetic effects simultaneously through many cycles of iterations. Simulation results based on computer-generated and real-world marker data demonstrated the ability of our method to swiftly produce sensible results regarding the phenotype-to-genotype association. Our new method provides a robust and remarkably fast alternative to full Bayesian estimation in high-dimensional models where the computational burden associated with Markov chain Monte Carlo simulation is often unwieldy. The R code used to fit the model to the data is provided in the online supplementary material.

A new clustering method for detecting rare senses of abbreviations in clinical notes

Abbreviations are widely used in clinical documents and they are often ambiguous. Building a list of possible senses (also called sense inventory) for each ambiguous abbreviation is the first step to automatically identify correct meanings of abbreviations in given contexts. Clustering based methods have been used to detect senses of abbreviations from a clinical corpus [1]. However, rare senses remain challenging and existing algorithms are not good enough to detect them. In this study, we developed a new two-phase clustering algorithm called Tight Clustering for Rare Senses (TCRS) and applied it to sense generation of abbreviations in clinical text. Using manually annotated sense inventories from a set of 13 ambiguous clinical abbreviations, we evaluated and compared TCRS with the existing Expectation Maximization (EM) clustering algorithm for sense generation, at two different levels of annotation cost (10 vs. 20 instances for each abbreviation). Our results showed that the TCRS-based method could detect 85% senses on average; while the EM-based method found only 75% senses, when similar annotation effort (about 20 instances) was used. Further analysis demonstrated that the improvement by the TCRS method was mainly from additionally detected rare senses, thus indicating its usefulness for building more complete sense inventories of clinical abbreviations.

Fast ML Estimation for the Mixture of Factor Analyzers via an ECM Algorithm

In this brief, we propose a fast expectation conditional maximization (ECM) algorithm for maximum-likelihood (ML) estimation of mixtures of factor analyzers (MFA). Unlike the existing expectation-maximization (EM) algorithms such as the EM in Ghahramani and Hinton, 1996, and the alternating ECM (AECM) in McLachlan and Peel, 2003, where the missing data contains component-indicator vectors as well as latent factors, the missing data in our ECM consists of component-indicator vectors only. The novelty of our algorithm is that closed-form expressions in all conditional maximization (CM) steps are obtained explicitly, instead of resorting to numerical optimization methods. As revealed by experiments, the convergence of our ECM is substantially faster than EM and AECM regardless of whether assessed by central processing unit (CPU) time or number of iterations.

A New Method for Haplotype Inference Including Full-Sib Information

Recent literature has suggested that haplotype inference through close relatives, especially from nuclear families, can be an alternative strategy in determining linkage phase and estimating haplotype frequencies. In the case of no possibility to obtain genotypes for parents, and only full-sib information being used, a new approach is suggested to infer phase and to reconstruct haplotypes. We present a maximum-likelihood method via an expectation-maximization algorithm, called FSHAP, using only full-sib information when parent information is not available. FSHAP can deal with families with an arbitrary number of children, and missing parents or missing genotypes can be handled as well. In a simulation study we compare FSHAP with another existing expectation-maximization (EM)-based approach (FAMHAP), the conditioning approach implemented in FBAT and GENEHUNTER, which is only pedigree based and assumes linkage equilibrium. In most situations, FSHAP has the smallest discrepancy of haplotype frequency estimation and the lowest error rate in haplotype reconstruction, only in some cases FAMHAP yields comparable results. GENEHUNTER produces the largest discrepancy, and FBAT produces the highest error rate in offspring in most situations. Among the methods compared, FSHAP has the highest accuracy in reconstructing the diplotypes of the unavailable parents. Potential limitations of the method, e.g., in analyzing very large haplotypes, are indicated and possible solutions are discussed.

On improved EM algorithm and confidence interval construction for incomplete [formula omitted] tables

Constructing confidence interval (CI) for functions of cell probabilities (e.g., rate difference, rate ratio and odds ratio) is a standard procedure for categorical data analysis in clinical trials and medical studies. In the presence of incomplete data, existing methods could be problematic. For example, the inverse of the observed information matrix may not exist and the asymptotic CIs based on delta methods are hence not available. Even though the inverse of the observed information matrix exists, the large-sample delta methods are generally not reliable in small-sample studies. In addition, existing expectation-maximization (EM) algorithm via the conventional data augmentation (DA) may suffer from slow convergence due to the introduction of too many latent variables. In this article, for r × c tables with incomplete data, we propose a novel DA scheme that requires fewer latent variables and this will consequently lead to a more efficient EM algorithm. We present two bootstrap-type CIs for parameters of interest via the new EM algorithm with and without the normality assumption. For r × c tables with only one incomplete/supplementary margin, the improved EM algorithm converges in only one step and the associated maximum likelihood estimates can hence be obtained in closed form. Theoretical and simulation results showed that the proposed EM algorithm outperforms the existing EM algorithm. Three real data from a neurological study, a rheumatoid arthritis study and a wheeze study are used to illustrate the methodologies.

Hidden Gauss-Markov models for signal classification

Continuous-state hidden Markov models (CS-HMMs) are developed as a tool for signal classification. Analogs of the Baum (1972), Viterbi (1962), and Baum-Welch algorithms are formulated for this class of models. The CS-HMM algorithms are then specialized to hidden Gauss-Markov models (HGMMs) with linear Gaussian state-transition and output densities. A new Gaussian refactorization lemma is used to show that the Baum and Viterbi algorithms for HGMMs are implemented by two different formulations of the fixed-interval Kalman smoother. The measurement likelihoods obtained from the forward pass of the HGMM Baum algorithm and from the Kalman-filter innovation sequence are shown to be equal. A direct link between the Baum-Welch training algorithm and an existing expectation-maximization (EM) algorithm for Gaussian models is demonstrated. A new expression for the cross covariance between time-adjacent states in HGMMs is derived from the off-diagonal block of the conditional joint covariance matrix. A parameter invariance structure is noted for the HGMM likelihood function. CS-HMMs and HGMMs are extended to incorporate mixture densities for the a priori density of the initial state. Application of HGMMs to signal classification is demonstrated with a three-class test simulation.

Blind restoration of images degraded by space-variant blurs using iterative algorithms for both blur identification and image restoration

This paper addresses the problem of digital blind restoration of images degraded by space-variant blurs and noise. The existing Expectation-Maximization (EM) algorithm reported in the literature is extended in this paper and combined with the region adaptive technique to handle the problem of identifying spatially variant blurs. The proposed algorithm is a two-step interative process. The expectation step of the EM algorithm is modified by the use of iterative image restoration. The entire image is divided into disjointed regions and the blur is identified in these regions using the proposed modified form of the EM algorithm. The iterative Constrained Least Squares (CLS) algorithm used in space-invariant image restoration is extended to restore the space variant blur images. Spatially adaptive algorithms for restoration are also applied. Experiments have been carried out to evaluate the performances of the proposed algorithms.