Expectation Maximization Step Research Articles

Energy efficient topology control in Underwater Wireless Sensor Networks

Topology control in Underwater Wireless Sensor Networks (UWSNs) has been an active area of research owing to unique challenges an underwater environment poses. Node mobility due to water currents creates problems in localization and data transfer. In this paper, we propose an energy efficient topology control for mobile UWSNs using Expectation Maximization (EM) and Vickrey–Clarke–Groves (VCG) Auction mechanism. In the first stage, followed by a water current induced topology disturbance, the designated cluster head runs the EM algorithm to find similar sensor nodes in its proximity. In the second stage, cluster head runs VCG auction mechanism to compensate sensor nodes for costs incurred during data transfer. This ensures seamless data transmission between sensor nodes of heterogeneous ownership leading to higher energy efficiency. Simulations present higher energy efficiency of the network post EM and VCG steps. Comparison with existing protocols demonstrates superior performance of the protocol proposed in this paper.

Latent Network Estimation and Variable Selection for Compositional Data via Variational EM*

Network estimation and variable selection have been extensively studied in the statistical literature, but only recently have those two challenges been addressed simultaneously. In this paper, we seek to develop a novel method to simultaneously estimate network interactions and associations to relevant covariates for count data, and specifically for compositional data, which have a fixed sum constraint. We use a hierarchical Bayesian model with latent layers and employ spike-and-slab priors for both edge and covariate selection. For posterior inference, we develop a novel variational inference scheme with an expectation maximization step, to enable efficient estimation. Through simulation studies, we demonstrate that the proposed model outperforms existing methods in its accuracy of network recovery. We show the practical utility of our model via an application to microbiome data. The human microbiome has been shown to contribute to many of the functions of the human body, and also to be linked with a number of diseases. In our application, we seek to better understand the interaction between microbes and relevant covariates, as well as the interaction of microbes with each other. We call our algorithm SINC (Simultaneous Inference for Networks and Covariates) and provide a Python implementation, which is available online.

Sparse Bayesian Inference-Based Direct Off-Grid Position Determination in Multipath Environments

The classical super-resolution Direct Position Determination (SR-DPD) algorithms fail to suppress the coherent Non-Line-of-Sight (NLOS) interference due to the lack of independent measurements. The existing Sparse Signal Reconstruction (SSR) based DPD approach suffers from the intractable complexity since it needs to solve a Second-Order Cone Programming (SOCP) problem. Besides, the Grid Quantization Error (GQE) exists in all above on-grid model based algorithms inherently. The proposed Sparse Bayesian Inference (SBI) based off-grid DPD algorithm is easy to implement as the Expectation-Maximization (EM) method is applied to decouple the multi-dimensional optimization problem. In addition, the GQE is also eliminated by introducing the Gradient Descent (GD) mechanism into the EM steps to update the grid point coordinates of interest iteratively.

A Mixed Filtering Approach for Real-Time Seizure State Tracking Using Multi-Channel Electroencephalography Data.

Real-time continuous tracking of seizure state is necessary to develop feedback neuromodulation therapy that can prevent or terminate a seizure early. Due to its high temporal resolution, high scalp coverage, and non-invasive applicability, electroencephalography (EEG) is a good candidate for seizure tracking. In this research, we make multiple seizure state estimations using a mixed-filter and multiple channels found over the entire sensor space; then by applying a Kalman filter, we produce a single seizure state estimation made up of these individual estimations. Using a modified wrapper feature selection, we determine two optimal features of mixed data type, one continuous and one binary analyzing all available channels. These features are used in a state-space framework to model the continuous hidden seizure state. Expectation maximization is performed offline on the training and validation data sets to estimate unknown parameters. The seizure state estimation process is performed for multiple channels, and the seizure state estimation is derived using a square-root Kalman filter. A second expectation maximization step is utilized to estimate the unknown square-root Kalman filter parameters. This method is tested in a real-time applicable way for seizure state estimation. Applying this approach, we obtain a single seizure state estimation with quantitative information about the likelihood of a seizure occurring, which we call seizure probability. Our results on the experimental data (CHB-MIT EEG database) validate the proposed estimation method and we achieve an average accuracy, sensitivity, and specificity of 92.7%, 92.8%, and 93.4%, respectively. The potential applications of this seizure estimation model are for closed-loop neuromodulation and long-term quantitative analysis of seizure treatment efficacy.

A probabilistic topic model for event-based image classification and multi-label annotation

We propose an enhanced latent topic model based on latent Dirichlet allocation and convolutional neural nets for event classification and annotation in images. Our model builds on the semantic structure relating events, objects and scenes in images. Based on initial labels extracted from convolution neural networks (CNNs), and possibly user-defined tags, we estimate the event category and final annotation of an image through a refinement process based on the expectation–maximization (EM) algorithm. The EM steps allow to progressively ascertain the class category and refine the final annotation of the image. Our model can be thought of as a two-level annotation system, where the first level derives the image event from CNN labels and image tags and the second level derives the final annotation consisting of event-related objects/scenes. Experimental results show that the proposed model yields better classification and annotation performance in the two standard datasets: UIUC-Sports and WIDER.

The spike-and-slab lasso Cox model for survival prediction and associated genes detection.

Large-scale molecular profiling data have offered extraordinary opportunities to improve survival prediction of cancers and other diseases and to detect disease associated genes. However, there are considerable challenges in analyzing large-scale molecular data. We propose new Bayesian hierarchical Cox proportional hazards models, called the spike-and-slab lasso Cox, for predicting survival outcomes and detecting associated genes. We also develop an efficient algorithm to fit the proposed models by incorporating Expectation-Maximization steps into the extremely fast cyclic coordinate descent algorithm. The performance of the proposed method is assessed via extensive simulations and compared with the lasso Cox regression. We demonstrate the proposed procedure on two cancer datasets with censored survival outcomes and thousands of molecular features. Our analyses suggest that the proposed procedure can generate powerful prognostic models for predicting cancer survival and can detect associated genes. The methods have been implemented in a freely available R package BhGLM ( http://www.ssg.uab.edu/bhglm/ ). nyi@uab.edu. Supplementary data are available at Bioinformatics online.

A New Self-Training-Based Unsupervised Satellite Image Classification Technique Using Cluster Ensemble Strategy

This letter addresses the problem of unsupervised land-cover classification of remotely sensed multispectral satellite images from the perspective of cluster ensembles and self-learning. The cluster ensembles combine multiple data partitions generated by different clustering algorithms into a single robust solution. A cluster-ensemble-based method is proposed here for the initialization of the unsupervised iterative expectation-maximization (EM) algorithm which eventually produces a better approximation of the cluster parameters considering a certain statistical model is followed to fit the data. The method assumes that the number of land-cover classes is known. A novel method for generating a consistent labeling scheme for each clustering of the consensus is introduced for cluster ensembles. A maximum likelihood classifier is henceforth trained on the updated parameter set obtained from the EM step and is further used to classify the rest of the image pixels. The self-learning classifier, although trained without any external supervision, reduces the effect of data overlapping from different clusters which otherwise a single clustering algorithm fails to identify. The clustering performance of the proposed method on a medium resolution and a very high spatial resolution image have effectively outperformed the results of the individual clustering of the ensemble.

On the Estimation of a Univariate Gaussian Distribution: A Comparative Approach

Estimation of the unknown mean, μ and variance, σ2 of a univariate Gaussian distribution given a single study variable x is considered. We propose an approach that does not require initialization of the sufficient unknown distribution parameters. The approach is motivated by linearizing the Gaussian distribution through differential techniques, and estimating, μ and σ2 as regression coefficients using the ordinary least squares method. Two simulated datasets on hereditary traits and morphometric analysis of housefly strains are used to evaluate the proposed method (PM), the maximum likelihood estimation (MLE), and the method of moments (MM). The methods are evaluated by re-estimating the required Gaussian parameters on both large and small samples. The root mean squared error (RMSE), mean error (ME), and the standard deviation (SD) are used to assess the accuracy of the PM and MLE; confidence intervals (CIs) are also constructed for the ME estimate. The PM compares well with both the MLE and MM approaches as they all produce estimates whose errors have good asymptotic properties, also small CIs are observed for the ME using the PM and MLE. The PM can be used symbiotically with the MLE to provide initial approximations at the expectation maximization step.

Mahalanobis Encodings for Visual Categorization

Nowadays, the design of the representation of images is one of the most crucial factors in the performance of visual categorization. A common pipeline employed in most of recent researches for obtaining an image representa- tion consists of two steps: the encoding step and the pooling step. In this paper, we introduce the Mahalanobis metric to the two popular image patch encoding modules, Histogram Encoding and Fisher Encoding, that are used for Bag- of-Visual-Word method and Fisher Vector method, respectively. Moreover, for the proposed Fisher Vector method, a close-form approximation of Fisher Vector can be derived with the same assumption used in the original Fisher Vector, and the codebook is built without resorting to time-consuming EM (Expectation-Maximization) steps. Experimental evaluation of multi-class classification demonstrates the effectiveness of the proposed encoding methods.

Joint Vehicle Trajectory and Model Parameter Estimation Using Road Side Sensors

This paper shows how a particle smoother-based system identification method can be applied for estimating the trajectory of road vehicles. As sensors, a combination of an accelerometer measuring the road surface vibrations and a magnetometer measuring magnetic disturbances mounted on the side of the road are considered. First, sensor models describing the measurements of the two sensors are introduced. It is shown that these depend on unknown, static parameters that have to be considered in the estimation. Second, the sensor models are combined with a two-dimensional constant velocity motion model. Third, the system identification algorithm is introduced which iteratively runs a Rao-Blackwellized particle smoother to estimate the vehicle trajectory followed by an expectation-maximization step to estimate the parameters. Finally, the method is applied to both simulation and measurement data. It is found that the method works well in general and some issues when real data is considered are identified as future work.

Hybrid Metaheuristic Approaches to the Expectation Maximization for Estimation of the Hidden Markov Model for Signal Modeling

The expectation maximization (EM) is the standard training algorithm for hidden Markov model (HMM). However, EM faces a local convergence problem in HMM estimation. This paper attempts to overcome this problem of EM and proposes hybrid metaheuristic approaches to EM for HMM. In our earlier research, a hybrid of a constraint-based evolutionary learning approach to EM (CEL-EM) improved HMM estimation. In this paper, we propose a hybrid simulated annealing stochastic version of EM (SASEM) that combines simulated annealing (SA) with EM. The novelty of our approach is that we develop a mathematical reformulation of HMM estimation by introducing a stochastic step between the EM steps and combine SA with EM to provide better control over the acceptance of stochastic and EM steps for better HMM estimation. We also extend our earlier work and propose a second hybrid which is a combination of an EA and the proposed SASEM, (EA-SASEM). The proposed EA-SASEM uses the best constraint-based EA strategies from CEL-EM and stochastic reformulation of HMM. The complementary properties of EA and SA and stochastic reformulation of HMM of SASEM provide EA-SASEM with sufficient potential to find better estimation for HMM. To the best of our knowledge, this type of hybridization and mathematical reformulation have not been explored in the context of EM and HMM training. The proposed approaches have been evaluated through comprehensive experiments to justify their effectiveness in signal modeling using the speech corpus: TIMIT. Experimental results show that proposed approaches obtain higher recognition accuracies than the EM algorithm and CEL-EM as well.

ADAPTIVE ANNEALED IMPORTANCE SAMPLING FOR MULTIMODAL POSTERIOR EXPLORATION AND MODEL SELECTION WITH APPLICATION TO EXTRASOLAR PLANET DETECTION

We describe an algorithm that can adaptively provide mixture summaries of multimodal posterior distributions. The parameter space of the involved posteriors ranges in size from a few dimensions to dozens of dimensions. This work was motivated by an astrophysical problem called extrasolar planet (exoplanet) detection, wherein the computation of stochastic integrals that are required for Bayesian model comparison is challenging. The difficulty comes from the highly nonlinear models that lead to multimodal posterior distributions. We resort to importance sampling (IS) to estimate the integrals, and thus translate the problem to be how to find a parametric approximation of the posterior. To capture the multimodal structure in the posterior, we initialize a mixture proposal distribution and then tailor its parameters elaborately to make it resemble the posterior to the greatest extent possible. We use the effective sample size (ESS) calculated based on the IS draws to measure the degree of approximation. The bigger the ESS is, the better the proposal resembles the posterior. A difficulty within this tailoring operation lies in the adjustment of the number of mixing components in the mixture proposal. Brute force methods just preset it as a large constant, which leads to an increase in the required computational resources. We provide an iterative delete/merge/add process, which works in tandem with an expectation-maximization step to tailor such a number online. The efficiency of our proposed method is tested via both simulation studies and real exoplanet data analysis.

Automated Algorithm for Quantitative Analysis of Fluorescence Nanoscopy Images

Fluorescence nanoscopy, both in the far-field (STED) and near-field version (NSOM) allows imaging of biological samples with exquisite resolution and high signal-to-noise ratio, representing a powerful tool to address biological questions at the nanometer scale. Yet, despite the highly improved resolution and quality in comparison with conventional confocal microscopy, Poisson noise, background noise and cell autofluorescence are limiting factors for the automated quantification of molecular spatial organization as well as for the recognition of distinct features such as clusters and aggregates. For these reasons, the analysis of nanoscopy images, if at all, is often performed in a semi-automatic fashion, requiring the manual identification of image features and the supply of user-defined parameters. As such, these approaches are time-consuming and might suffer from non-objective evaluation, which are critical parameters in the analysis of samples with high emitters density.Here we propose a method that allows fully automated reconstruction of fluorescence nanoscopy images and quantitative analysis of protein spatial organization. Our approach is based on the combination of a maximum likelihood algorithm and an expectation-maximization step, by means of which the image is de-noised and decomposed into point-spread functions. The further application of a local maxima identifying routine and Voronoi tessellation, allow us to extract the maximum information from the fluorescence images without impacting on the optical resolution. The performance of the method has been tested on simulated images at varying emitter density and signal-to-noise ratio, showing its applicability to standard STED and NSOM images. Furthermore, we have successfully applied this algorithm to quantitatively analyze fluorescence nanoscopy images of high densely packed membrane receptors on mammalian cells. Our results show faithful retrieval of receptor positions, stoichiometry and their lateral distribution of receptors on the cell membrane.

A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation

A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution–typically a posterior distribution, of which we only require a kernel–in the sense that the Kullback–Leibler divergence between target and mixture is minimized. We label this approach Mixture oftby Importance Sampling weighted Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis–Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously, while the quality of the approximation remains almost unchanged. This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach. This is useful for importance or Metropolis–Hastings sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model’s mixture regimes’ parameters. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.

A Class of Adaptive Importance Sampling Weighted EM Algorithms for Efficient and Robust Posterior and Predictive Simulation

A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution - typically a posterior distribution, of which we only require a kernel - in the sense that the Kullback-Leibler divergence between target and mixture is minimized. We label this approach Mixture of t by Importance Sampling and Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis-Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner. Second, we introduce a permutation-augmented MitISEM approach. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.

Performance of Different Population Pharmacokinetic Algorithms

There has been an increased focus on population pharmacokinetics (PK) to improve the drug development process since the "Critical Path paper" by the Food and Drug Administration. This increased interest has given rise to additional algorithms. The purpose of this exercise was to compare the new algorithms iterative-2-stage (ITS) and maximum likelihood expectation maximization (MLEM) available in ADAPT 5 with other methods. A total of 29 clinical trials with different study designs were simulated. Different algorithms were used to fit the simulated data, and the estimated parameters were compared with the true values. The algorithms ITS and MLEM were compared with the standard-2-stage, Iterative-2-Stage (IT2S) method in the IT2S package and the first-order conditional estimate (FOCE) method in NONMEM version VI. Imprecision and bias for the population PK parameters, variances, and individual PK parameters were used to compare the methods. Population PK parameters were well estimated and bias low for all nonlinear mixed effect modeling approaches. These approaches were superior to the standard-2-stage analyses. The algorithm MLEM was better than IT2S and ITS in predicting the PK and variability parameters. Residual variability was better estimated using MLEM and FOCE. A difference in the estimation of the variance exists between FOCE and the other methods. Variances estimated with FOCE often had shrinkage issues, whereas MLEM in ADAPT 5 had practically no shrinkage problems. Using MLEM, a reduction from 3000 to 1000 samples in the expectation maximization step had no impact on the results. The new algorithm MLEM in ADAPT 5 was consistently better than IT2S and ITS in its prediction of PK parameters, variances, and the residual variability. It was comparable with the FOCE method with significantly fewer shrinkage issues in the estimation of variance. The number of samples used in the expectation maximization step with MLEM did not influence the results.

Exploiting towed array maneuvers for online maximum likelihood field directionality estimation.

Passive estimation of the time‐varying field directionality using a towed acoustic array remains a central problem in passive sonar. Typically, array processing algorithms are designed around the assumption of a linear array and fail to take advantage of the dynamics associated with a towed array. Performance of such systems can often be complicated by left‐right ambiguities associated with the array as well as poor resolution near endfire directions. In this paper, the dynamics of the towed array are exploited allowing for left‐right disambiguation as well as improved endfire resolution. A new method for online spatial spectrum estimation is presented. The maximum likelihood (ML) of the time‐varying field is solved for using a single expectation maximization step after each received data snapshot. A multi‐source, dynamic simulation is used to illustrate the proposed algorithm’s ability to suppress ambiguous towed‐array backlobes and resolve closely spaced interferers near endfire which pose challenges for conventional beamforming approaches, especially during array maneuvers. Simulation results suggest the online ML method offers improved M‐of‐N detection performance over that of conventional beamforming. [This work was supported by ONR.]

Fast estimation of Gaussian mixture models for image segmentation

The expectation maximization algorithm has been classically used to find the maximum likelihood estimates of parameters in probabilistic models with unobserved data, for instance, mixture models. A key issue in such problems is the choice of the model complexity. The higher the number of components in the mixture, the higher will be the data likelihood, but also the higher will be the computational burden and data overfitting. In this work, we propose a clustering method based on the expectation maximization algorithm that adapts online the number of components of a finite Gaussian mixture model from multivariate data or method estimates the number of components and their means and covariances sequentially, without requiring any careful initialization. Our methodology starts from a single mixture component covering the whole data set and sequentially splits it incrementally during expectation maximization steps. The coarse to fine nature of the algorithm reduce the overall number of computations to achieve a solution, which makes the method particularly suited to image segmentation applications whenever computational time is an issue. We show the effectiveness of the method in a series of experiments and compare it with a state-of-the-art alternative technique both with synthetic data and real images, including experiments with images acquired from the iCub humanoid robot.

A Class of Adaptive EM-Based Importance Sampling Algorithms for Efficient and Robust Posterior and Predictive Simulation

textabstractA class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution -typically a posterior distribution, of which we only require a kernel - in the sense that the Kullback-Leibler divergence between target and mixture is minimized. We label this approach Mixture of t by Importance Sampling and Expectation Maximization (MitISEM). We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously. This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach, for importance sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model's mixture regimes' parameters. Third, we propose a partial MitISEM approach, which aims at approximating the marginal and conditional posterior distributions of subsets of model parameters, rather than the joint. This division can substantially reduce the dimension of the approximation problem.

A stochastic version of Expectation Maximization algorithm for better estimation of Hidden Markov Model

This paper attempts to overcome the local convergence problem of the Expectation Maximization (EM) based training of the Hidden Markov Model (HMM) in speech recognition. We propose a hybrid algorithm, Simulated Annealing Stochastic version of EM (SASEM), combining Simulated Annealing with EM that reformulates the HMM estimation process using a stochastic step between the EM steps and the SA. The stochastic processes of SASEM inside EM can prevent EM from converging to a local maximum and find improved estimation for HMM using the global convergence properties of SA. Experiments on the TIMIT speech corpus show that SASEM obtains higher recognition accuracies than the EM.