Proposed Expectation-maximization Algorithm Research Articles

Influence of detection noise on the maximum likelihood estimation of atmospheric turbulence fading parameters.

Free space optical system design and performance analysis are highly related to channel statistics obtained by the channel parameter estimation. The accurate estimation of the channel parameters and scintillation index needs to consider the photoelectric detection noise at the receiving end. We propose a maximum likelihood (ML) method for estimating the parameters of a gamma-gamma fading channel affected by photoelectric detection noise. The Newton-Raphson method and expectation maximization (EM) algorithm are developed to compute the ML estimates of atmospheric turbulence fading parameters and variance of the detection noise. We also derive the Cramer-Rao bound for the unknown parameters. By way of the mean square estimation errors, our estimation technique performance is compared with existing methods of the estimation which ignore detection noise. Based on the measured channel and detection noise data under three weather conditions, it is verified that the proposed EM algorithm, considering the influence of detection noise, can significantly improve the estimation accuracy of atmospheric turbulence fading parameters.

Analysis of skewed data by using compound Poisson exponential distribution with applications to insurance claims

The main aim of this paper is to introduce a new family of distributions, namely compound zero-truncated Poisson exponential distribution of which exponential distribution is a special case. The proposed family of distributions represents the zero truncated-Poisson sum of independent and identically distributed exponential random variables. The proposed distribution has two parameters and its probability density function can be skewed and unimodal. It can be used quite effectively in analyzing skewed data. We suggest to use expectation–maximization (EM)-type algorithm to estimate the unknown parameters, and it is observed that it is easy to implement in practice. We further consider the bivariate version of the proposed model which has three parameters and provides different properties. We have performed extensive simulation studies to see the performances of the proposed EM algorithm, and a real data set has been analyzed to see the effectiveness of the proposed models.

Simultaneous variable selection in regression analysis of multivariate interval-censored data.

Multivariate interval-censored data arise when each subject under study can potentially experience multiple events and the onset time of each event is not observed exactly but is known to lie in a certain time interval formed by adjacent examination times with changed statuses of the event. This type of incomplete and complex data structure poses a substantial challenge in practical data analysis. In addition, many potential risk factors exist in numerous studies. Thus, conducting variable selection for event-specific covariates simultaneously becomes useful in identifying important variables and assessing their effects on the events of interest. In this paper, we develop a variable selection technique for multivariate interval-censored data under a general class of semiparametric transformation frailty models. The minimum information criterion (MIC) method is embedded in the optimization step of the proposed expectation-maximization (EM) algorithm to obtain the parameter estimator. The proposed EM algorithm greatly reduces the computational burden in maximizing the observed likelihood function, and the MIC naturally avoids selecting the optimal tuning parameter as needed in many other popular penalties, making the proposed algorithm promising and reliable. The proposed method is evaluated through extensive simulation studies and illustrated by an analysis of patient data from the Aerobics Center LongitudinalStudy.

Regression analysis of arbitrarily censored survival data under the proportional odds model.

Arbitrarily censored data are referred to as the survival data that contain a mixture of exactly observed, left-censored, interval-censored, and right-censored observations. Existing research work on regression analysis on arbitrarily censored data is relatively sparse and mainly focused on the proportional hazards model and the accelerated failure time model. This article studies the proportional odds (PO) model and proposes a novel estimation approach through an expectation-maximization (EM) algorithm for analyzing such data. The proposed EM algorithm has many appealing properties such as being robust to initial values, easy to implement, converging fast, and providing the variance estimate of the regression parameter estimate in closed form. An informal diagnosis plot is developed for checking the PO model assumption. Our method has shown excellent performance in estimating the regression parameters as well as the baseline survival function in a simulation study. A real-life dataset about metastatic colorectal cancer is analyzed for illustration. An R package regPO has been created for practitioners to implement our method.

An efficient EM algorithm for the mixture of negative binomial models

Overdispersion is a widespread phenomenon in most count data sets. The negative binomial distribution is commonly adopted to fit over-dispersed count data. On the other hand, the mixture model always plays an important role in unsupervised classification. However, when estimating the parameters in the mixture of negative binomial models, the typical generalized Expectation Maximization (EM) algorithm which involves additional iterative procedures in M-step increases computational time. Hence, there remains a need for an efficient algorithm that can speed up the procedure of parameter estimation. For this purpose, here we develop a novel EM algorithm that successfully avoids the typical numerical solution in M-step for the mixture of negative binomial models. We extend further this EM algorithm to the zero-inflated negative binomial model. In the simulation studies, we focus on the runtimes and the classification performance of our proposed algorithm implemented in the mixture of negative binomial model. We found that our proposed EM algorithm can reduce the runtime of maximum likelihood estimation effectively, while achieving the similar classification performance in comparison with the typical EM algorithm. The mixture of negative binomial model and the proposed EM algorithm finally illustrates their good performance of fitting the real earthquake count data.

Marginalized maximum a posteriori estimation for the four-parameter logistic model under a mixture modelling framework.

The four-parameter logistic model (4PLM) has recently attracted much interest in various applications. Motivated by recent studies that re-express the four-parameter model as a mixture model with two levels of latent variables, this paper develops a new expectation-maximization (EM) algorithm for marginalized maximum a posteriori estimation of the 4PLM parameters. The mixture modelling framework of the 4PLM not only makes the proposed EM algorithm easier to implement in practice, but also provides a natural connection with popular cognitive diagnosis models. Simulation studies were conducted to show the good performance of the proposed estimation method and to investigate the impact of the additional upper asymptote parameter on the estimation of other parameters. Moreover, a real data set was analysed using the 4PLM to show its improved performance over the three-parameter logistic model.

An expectation-maximization algorithm for Bayesian operational modal analysis with multiple (possibly close) modes

The Bayesian FFT method has gained attention in operational modal analysis of civil engineering structures. Not only the most probable value (MPV) of modal parameters can be computed efficiently, but also the identification uncertainty can be rigorously quantified in terms of posterior covariance matrix. A recently developed fast algorithm for general multiple (possibly close) modes was found to work well in most cases, but convergence could be slow or even fail in challenging situations. The algorithm is also tedious to computer-code. Aiming at resolving these issues, an expectation-maximization (EM) algorithm is developed by viewing the modal response as a latent variable. The parameter-expansion EM and the parabolic-extrapolation EM are further adopted, allowing mode shape norm constraints to be incorporated and accelerating convergence, respectively. A robust implementation is provided based on the QR and Cholesky decompositions, so that the computation can be done efficiently and reliably. Empirical studies verify the performance of the proposed EM algorithm. It offers a more efficient and robust (in terms of convergence) alternative that can be especially useful when the existing algorithm has difficulty to converge. In addition, it opens a way to compute the MPV in the Bayesian FFT method for other unexplored cases, e.g., multi-mode multi-setup problem.

Identifying genes with tri-modal association with survival and tumor grade in cancer patients

BackgroundPrevious cancer genomics studies focused on searching for novel oncogenes and tumor suppressor genes whose abundance is positively or negatively correlated with end-point observation, such as survival or tumor grade. This approach may potentially miss some truly functional genes if both its low and high modes have associations with end-point observation. Such genes act as both oncogenes and tumor suppressor genes, a scenario that is unlikely but theoretically possible.ResultsWe invented an Expectation-Maximization (EM) algorithm to divide patients into low-, middle- and high-expressing groups according to the expression level of a certain gene in both tumor and normal patients. We found one gene, ORMDL3, whose low and high modes were both associated with worse survival and higher tumor grade in breast cancer patients in multiple patient cohorts. We speculate that its tumor suppressor gene role may be real, while its high expression correlating with worse end-point outcome is probably due to the passenger event of the nearby ERBB2’s amplification.ConclusionsThe proposed EM algorithm can effectively detect genes having tri-modal distributed expression in patient groups compared to normal genes, thus rendering a new perspective on dissecting the association between genomic features and end-point observations. Our analysis of breast cancer datasets suggest that the gene ORMDL3 may have an unexploited tumor suppressive function.

Spectrum Efficient Joint Frequency Offset and Channel Estimation for Time-Asynchronous Amplify-and Forward Two-Way Relay Networks

This paper considers the problem of the joint channel and carrier frequency offset (CFO) estimation for asynchronous amplify-and-forward (AF) two-way relay networks (TWRNs). This problem has not been solved to the best of our knowledge, and the presence of timing offsets between the two terminals, in addition to the frequency offsets, makes it necessary to develop signal sampling and joint estimation schemes that explicitly take into account both offsets. As potential solutions, we investigate two estimation strategies. The first strategy is pilot-based estimation, and in this context, we develop the maximum-likelihood joint CFO and channel estimator. Then, we consider the semi-blind estimation strategy, proposing a novel estimator based on expectation maximization (EM). As an alternative to the EM, we also investigate decision-directed (DD) semi-blind estimation. The semi-blind Cramer-Rao bound (CRB) is also obtained as a benchmark. Extensive experiments are used to compare the different estimation methods, showing that the proposed EM algorithm yields significantly lower mean-squared error than both the pilot-based estimator and the DD estimator and converges to the obtained CRB. It also yields much better symbol error rates. This is achieved at a reasonable computational complexity, and only a limited number of EM iterations are sufficient for convergence. The main contribution of our work is the development of the EM estimator, the corresponding CRB, and the use of simulations to demonstrate the superior performance of the proposed EM algorithm compared to other potential solutions.

Reconstructing Video of Time-Varying Sources From Radio Interferometric Measurements

Very long baseline interferometry (VLBI) makes it possible to recover the images of astronomical sources with extremely high angular resolution. Most recently, the Event Horizon Telescope (EHT) has extended VLBI to short millimeter wavelengths with a goal of achieving angular resolution sufficient for imaging the event horizons of nearby supermassive black holes. Interferometry provides measurements related to the underlying source image through a sparse set spatial frequencies. An image can then be recovered from these measurements by making assumptions about the underlying image. One of the most important assumptions made by conventional imaging methods is that over the course of a night's observation the image is static. However, for quickly evolving sources, such as the galactic center's supermassive black hole (SgrA*) targeted by the EHT, this assumption is violated and these conventional imaging approaches fail. This paper presents a new way to model VLBI measurements that allows for the recovery of both the appearance and dynamics of an evolving source by reconstructing a video rather than a static image. By modeling VLBI measurements using a Gaussian Markov Model, information can be propagated across observations in time to reconstruct a video, while simultaneously learning about the dynamics of the source's emission region. This paper demonstrates our proposed expectation-maximization algorithm, StarWarps, on realistic synthetic observations of black holes, and shows how it substantially improves the results compared to conventional imaging methods. In addition to synthetic data, the technique is demonstrated on real VLBA data of the M87 jet.

Plane-extraction from depth-data using a Gaussian mixture regression model

We propose a novel algorithm for unsupervised extraction of piecewise planar models from depth-data. Among other applications, such models are a good way of enabling autonomous agents (robots, cars, drones, etc.) to effectively perceive their surroundings and to navigate in three dimensions. We propose to do this by fitting the data with a piecewise-linear Gaussian mixture regression model whose components are skewed over planes, making them flat in appearance rather than being ellipsoidal, by embedding an outlier-trimming process that is formally incorporated into the proposed expectation-maximization algorithm, and by selectively fusing contiguous, coplanar components. Part of our motivation is an attempt to estimate more accurate plane-extraction by allowing each model component to make use of all available data through probabilistic clustering. The algorithm is thoroughly evaluated against a standard benchmark and is shown to rank among the best of the existing state-of-the-art methods.

Univariate and bivariate geometric discrete generalized exponential distributions

In 1997, Marshall and Olkin introduced a very powerful method to introduce an additional parameter to a class of continuous distribution functions that brings more flexibility to the model. They demonstrated their method for the exponential and Weibull classes. In the same paper they briefly indicated its bivariate extension. The main aim of this article is to introduce the same method, for the first time, to the class of discrete generalized exponential distributions both for the univariate and bivariate cases. We investigate several properties of the proposed univariate and bivariate classes. The univariate class has three parameters, whereas the bivariate class has five parameters. It is observed that depending on the parameter values, the univariate class can be zero inflated as well as heavy tailed. We propose to use an expectation–maximization (EM) algorithm to estimate the unknown parameters. Small simulation experiments have been performed to see the effectiveness of the proposed EM algorithm, and a bivariate data set has been analyzed; it is observed that the proposed models and the EM algorithm work quite well in practice.

Spectrally Efficient Channel Estimation for Asynchronous Amplify-and-Forward Two-Way Relay Networks

In this paper, we consider the problem of channel estimation for asynchronous amplify-and-forward (AF) two-way relay networks. We propose a novel semi-blind channel estimation algorithm for this problem based on the expectation- maximization (EM) framework. The proposed EM algorithm has low complexity, and only a small number of EM iterations are needed to achieve convergence. The semi-blind Cramer-Rao bound (CRB) for channel estimation in asynchronous AF two-way relay networks is also obtained. Using simulations, we show that the proposed EM algorithm significantly outperforms pilot-based estimation by using only a limited number of received data samples in addition to the pilot samples. Furthermore, the achieved mean-squared error performance almost overlaps with the obtained CRB. Finally, a semi-blind generalized likelihood ratio testing (GLRT) method is proposed to tackle sequence arriving order (SAO) detection at the terminals and is shown to yield a higher probability of detection than the pilot-based GLRT for SAO detection.

Gamma lifetimes and associated inference for interval-censored cure rate model with COM–Poisson competing cause

ABSTRACTIn this article, we consider a competing cause scenario and assume the wider family of Conway–Maxwell–Poisson (COM–Poisson) distribution to model the number of competing causes. Assuming the type of the data to be interval censored, the main contribution is in developing the steps of the expectation maximization (EM) algorithm to determine the maximum likelihood estimates (MLEs) of the model parameters. A profile likelihood approach within the EM framework is proposed to estimate the COM–Poisson shape parameter. An extensive simulation study is conducted to evaluate the performance of the proposed EM algorithm. Model selection within the wider class of COM–Poisson distribution is carried out using likelihood ratio test and information-based criteria. A study to demonstrate the effect of model mis-specification is also carried out. Finally, the proposed estimation method is applied to a data on smoking cessation and a detailed analysis of the obtained results is presented.

Multivariate geometric skew-normal distribution

ABSTRACTAzzalini [A class of distributions which include the normal. Scand J Stat. 1985;12:171–178] introduced a skew-normal distribution of which normal distribution is a special case. Recently, Kundu [Geometric skew normal distribution. Sankhya Ser B. 2014;76:167–189] introduced a geometric skew-normal distribution and showed that it has certain advantages over Azzalini's skew-normal distribution. In this paper we discuss about the multivariate geometric skew-normal (MGSN) distribution. It can be used as an alternative to Azzalini's skew-normal distribution. We discuss different properties of the proposed distribution. It is observed that the joint probability density function of the MGSN distribution can take a variety of shapes. Several characterization results have been established. Generation from an MGSN distribution is quite simple, hence the simulation experiments can be performed quite easily. The maximum likelihood estimators of the unknown parameters can be obtained quite conveniently using the expectation–maximization (EM) algorithm. We perform some simulation experiments and it is observed that the performances of the proposed EM algorithm are quite satisfactory. Furthermore, the analyses of two data sets have been performed, and it is observed that the proposed methods and the model work very well.

Estimating bayesian networks parameters using EM and Gibbs sampling

A method based on Expectation Maximization (EM) algorithm and Gibbs sampling is proposed to estimate Bayesian networks (BNs) parameters. We employ the Gibbs sampling to approximate the E-step of EM algorithm. According to transition probability, Gibbs sampling is utilized in data completion of E-step, which can reduce the computational complexity of EM algorithm. The experiments for comparison between the proposed method and EM algorithm are made. For the proposed method, the consumed time and the number of iterations are all less than those of EM algorithm. However, the KL divergence is higher than that of EM algorithm, which is a limitation for the proposed method.

An Expectation Maximization algorithm for fitting the generalized odds-rate model to interval censored data.

The generalized odds-rate model is a class of semiparametric regression models, which includes the proportional hazards and proportional odds models as special cases. There are few works on estimation of the generalized odds-rate model with interval censored data because of the challenges in maximizing the complex likelihood function. In this paper, we propose a gamma-Poisson data augmentation approach to develop an Expectation Maximization algorithm, which can be used to fit the generalized odds-rate model to interval censored data. The proposed Expectation Maximization algorithm is easy to implement and is computationally efficient. The performance of the proposed method is evaluated by comprehensive simulation studies and illustrated through applications to datasets from breast cancer and hemophilia studies. In order to make the proposed method easy to use in practice, an R package 'ICGOR' was developed. Copyright © 2016 John Wiley & Sons, Ltd.

Simultaneous Bayesian Estimation of Excitatory and Inhibitory Synaptic Conductances by Exploiting Multiple Recorded Trials.

Advanced statistical methods have enabled trial-by-trial inference of the underlying excitatory and inhibitory synaptic conductances (SCs) of membrane-potential recordings. Simultaneous inference of both excitatory and inhibitory SCs sheds light on the neural circuits underlying the neural activity and advances our understanding of neural information processing. Conventional Bayesian methods can infer excitatory and inhibitory SCs based on a single trial of observed membrane potential. However, if multiple recorded trials are available, this typically leads to suboptimal estimation because they neglect common statistics (of synaptic inputs (SIs)) across trials. Here, we establish a new expectation maximization (EM) algorithm that improves such single-trial Bayesian methods by exploiting multiple recorded trials to extract common SI statistics across the trials. In this paper, the proposed EM algorithm is embedded in parallel Kalman filters or particle filters for multiple recorded trials to integrate their outputs to iteratively update the common SI statistics. These statistics are then used to infer the excitatory and inhibitory SCs of individual trials. We demonstrate the superior performance of multiple-trial Kalman filtering (MtKF) and particle filtering (MtPF) relative to that of the corresponding single-trial methods. While relative estimation error of excitatory and inhibitory SCs is known to depend on the level of current injection into a cell, our numerical simulations using MtKF show that both excitatory and inhibitory SCs are reliably inferred using an optimal level of current injection. Finally, we validate the robustness and applicability of our technique through simulation studies, and we apply MtKF to in vivo data recorded from the rat barrel cortex.

Anterior cruciate ligament reconstruction model based on anatomical position locating

In the reconstruction surgery of Anterior Cruciate Ligament (ACL), how to locate the anatomical position is a very hard point to clinician occupational therapists. In this paper, we propose an Anatomical Position Locating (APL) approach based on Expectation Maximization (EM) algorithm. Firstly, the proposed intersection set operation algorithm is proposed to compute the attachment region between the injured ACL and femur or tibia. Then, the anatomical position is located by the 3D points cloud with the Gaussian spatial distribution. The last, the attachment spatial distribution and the barycenter, which are also viewed as the candidates of the anatomical position by a proposed EM algorithm, is partition. Experimental results verify our assumption and demonstrate that the located anatomical position has great serviceability.

Joint Temperature-Lasing Mode Compensation for Time-of-Flight LiDAR Sensors

We propose an expectation maximization (EM) strategy for improving the precision of time of flight (ToF) light detection and ranging (LiDAR) scanners. The novel algorithm statistically accounts not only for the bias induced by temperature changes in the laser diode, but also for the multi-modality of the measurement noises that is induced by mode-hopping effects. Instrumental to the proposed EM algorithm, we also describe a general thermal dynamics model that can be learned either from just input-output data or from a combination of simple temperature experiments and information from the laser’s datasheet. We test the strategy on a SICK LMS 200 device and improve its average absolute error by a factor of three.