Online EM Algorithm Research Articles

Deep expectation-maximization network for unsupervised image segmentation and clustering

Unsupervised learning, such as unsupervised image segmentation and clustering, are fundamental tasks in image representation learning. In this paper, we design a deep expectation-maximization (DEM) network for unsupervised image segmentation and clustering. It is based on the statistical modeling of image in its latent feature space by Gaussian mixture model (GMM), implemented in a novel deep learning framework. Specifically, in the unsupervised setting, we design an auto-encoder network and an EM module over the image latent features, for jointly learning the image latent features and GMM model of the latent features in a single framework. To construct the EM-module, we unfold the iterative operations of EM algorithm and the online EM algorithm in fixed steps to be differentiable network blocks, plugged into the network to estimate the GMM parameters of the image latent features. The proposed network parameters can be end-to-end optimized using losses based on log-likelihood of GMM, entropy of Gaussian component assignment probabilities and image reconstruction error. Extensive experiments confirm that our proposed networks achieve favorable results compared with several state-of-the-art methods in unsupervised image segmentation and clustering.

A Markov Model-Based Fusion Algorithm for Distorted Electronic Technology Archives.

This paper presents an in-depth study and analysis of the restoration of distorted electronic technology archives using Markov models and proposes a corresponding fusion algorithm. Using the image gradient parametrization as a regular term, the filtering restoration process is constrained and the fuzzy kernel is estimated to solve the degradation problem existing in the Tibetan antiquarian literature. In the algorithmic framework of nonlocal mean filtering, the calculation of the weight function is improved to reduce the computational effort. In the simulation results, it is shown that the improved nonlocal mean filtering restoration algorithm in this paper has good overall quality evaluation performance in the restoration of text-based images. The structural similarity between the generated image and the real image is guaranteed, and the internal mixed-mode learning of a single SAR image is performed by combining the pyramidal hierarchical network structure to improve the effectiveness of the generated image in general. The method further improves the similarity between the generated and real images, while improving the accuracy of the classification based on the data expanded by the generation method. The feasibility and accuracy of the online algorithm for the parameter estimation problem of the model are illustrated through numerical experiments with two specific examples of hidden Markov models, namely, a double Gaussian mixture model and a finite-state Markov chain model with Gaussian noise. At the same time, the advantages of the algorithm proposed in this paper are demonstrated by comparing the experimental results of the online EM algorithm with those of the offline EM algorithm on a unified model. Finally, the empirical analysis is used to illustrate the application of the algorithm in practical scenarios.

Learning discrete Bayesian network parameters from continuous data streams: What is the best strategy?

We compare three approaches to learning numerical parameters of discrete Bayesian networks from continuous data streams: (1) the EM algorithm applied to all data, (2) the EM algorithm applied to data increments, and (3) the online EM algorithm. Our results show that learning from all data at each step, whenever feasible, leads to the highest parameter accuracy and model classification accuracy. When facing computational limitations, incremental learning approaches are a reasonable alternative. While the differences in speed between incremental algorithms are not large (online EM is slightly slower), for all but small data sets online EM tends to be more accurate than incremental EM.

Large-Scale Estimation in Cyberphysical Systems Using Streaming Data: A Case Study With Arterial Traffic Estimation

Controlling and analyzing cyberphysical and robotics systems is increasingly becoming a Big Data challenge. We study the case of predicting drivers' travel times in a large urban area from sparse GPS traces. We present a framework that can accommodate a wide variety of traffic distributions and spread all the computations on a cluster to achieve small latencies. Our framework is built on Discretized Streams, a recently proposed approach to stream processing at scale. We demonstrate the usefulness of Discretized Streams with a novel algorithm to estimate vehicular traffic in urban networks. Our online EM algorithm can estimate traffic on a very large city network (the San Francisco Bay Area) by processing tens of thousands of observations per second, with a latency of a few seconds.

A mixture model for expression deconvolution from RNA-seq in heterogeneous tissues.

BackgroundRNA-seq, a next-generation sequencing based method for transcriptome analysis, is rapidly emerging as the method of choice for comprehensive transcript abundance estimation. The accuracy of RNA-seq can be highly impacted by the purity of samples. A prominent, outstanding problem in RNA-seq is how to estimate transcript abundances in heterogeneous tissues, where a sample is composed of more than one cell type and the inhomogeneity can substantially confound the transcript abundance estimation of each individual cell type. Although experimental methods have been proposed to dissect multiple distinct cell types, computationally "deconvoluting" heterogeneous tissues provides an attractive alternative, since it keeps the tissue sample as well as the subsequent molecular content yield intact.ResultsHere we propose a probabilistic model-based approach, Transcript Estimation from Mixed Tissue samples (TEMT), to estimate the transcript abundances of each cell type of interest from RNA-seq data of heterogeneous tissue samples. TEMT incorporates positional and sequence-specific biases, and its online EM algorithm only requires a runtime proportional to the data size and a small constant memory. We test the proposed method on both simulation data and recently released ENCODE data, and show that TEMT significantly outperforms current state-of-the-art methods that do not take tissue heterogeneity into account. Currently, TEMT only resolves the tissue heterogeneity resulting from two cell types, but it can be extended to handle tissue heterogeneity resulting from multi cell types. TEMT is written in python, and is freely available at https://github.com/uci-cbcl/TEMT.ConclusionsThe probabilistic model-based approach proposed here provides a new method for analyzing RNA-seq data from heterogeneous tissue samples. By applying the method to both simulation data and ENCODE data, we show that explicitly accounting for tissue heterogeneity can significantly improve the accuracy of transcript abundance estimation.

Online EM Algorithm for Background Subtraction

Gaussian mixture model is a popular model in background subtraction and efficient equations have been derived to update GMM parameters previously. In order to compute parameters more accurately while maintain constant computing time per frame, we apply online EM algorithm to update the parameters of Gaussian mixture models. To avoid computing the inverse of covariance matrix, we use isotropic matrix and the corresponding incremental EM equations are derived. Experiments demonstrate that online EM algorithm can give more accurate segment result than previous update equations.

Parameter Estimation of a State-Space Model of Noise for Robust Speech Recognition

In this paper, parameter estimation of a state-space model of noise or noisy speech cepstra is investigated. A blockwise EM algorithm is derived for the estimation of the state and observation noise covariance from noise-only input data. It is supposed to be used during the offline training mode of a speech recognizer. Further a sequential online EM algorithm is developed to adapt the observation noise covariance on noisy speech cepstra at its input. The estimated parameters are then used in model-based speech feature enhancement for noise-robust automatic speech recognition. Experiments on the AURORA4 database lead to improved recognition results with a linear state model compared to the assumption of stationary noise.

Divide and Conquer: A Mixture-Based Approach to Regional Adaptation for MCMC

The efficiency of Markov chain Monte Carlo (MCMC) algorithms can vary dramatically with the choice of simulation parameters. Adaptive MCMC (AMCMC) algorithms allow the automatic tuning of the parameters while the simulation is in progress. A multimodal target distribution may call for regional adaptation of Metropolis–Hastings samplers so that the proposal distribution varies across regions in the sample space. Establishing such a partition is not straightforward and, in many instances, the learning required for its specification takes place gradually, as the simulation proceeds. In the case in which the target distribution is approximated by a mixture of Gaussians, we propose an adaptation process for the partition. It involves fitting the mixture using the available samples via an online EM algorithm and, based on the current mixture parameters, constructing the regional adaptive algorithm with online recursion (RAPTOR). The method is compared with other regional AMCMC samplers and is tested on simulated as well as real data examples.Relevant theoretical proofs, code and datasets are posted as an online supplement.

Robust Face Tracking via Collaboration of Generic and Specific Models

Significant appearance changes of objects under different orientations could cause loss of tracking, "drifting." In this paper, we present a collaborative tracking framework to robustly track faces under large pose and expression changes and to learn their appearance models online. The collaborative tracking framework probabilistically combines measurements from an offline-trained generic face model with measurements from online-learned specific face appearance models in a dynamic Bayesian network. In this framework, generic face models provide the knowledge of the whole face class, while specific face models provide information on individual faces being tracked. Their combination, therefore, provides robust measurements for multiview face tracking. We introduce a mixture of probabilistic principal component analysis (MPPCA) model to represent the appearance of a specific face under multiple views, and we also present an online EM algorithm to incrementally update the MPPCA model using tracking results. Experimental results demonstrate that the collaborative tracking and online learning methods can handle large pose changes and are robust to distractions from the background.

Online updating appearance generative mixture model for meanshift tracking

This paper proposes an appearance generative mixture model based on key frames for meanshift tracking. Meanshift tracking algorithm tracks an object by maximizing the similarity between the histogram in tracking window and a static histogram acquired at the beginning of tracking. The tracking therefore could fail if the appearance of the object varies substantially. In this paper, we assume the key appearances of the object can be acquired before tracking and the manifold of the object appearance can be approximated by piece-wise linear combination of these key appearances in histogram space. The generative process is described by a Bayesian graphical model. An Online EM algorithm is proposed to estimate the model parameters from the observed histogram in the tracking window and to update the appearance histogram. We applied this approach to track human head motion and to infer the head pose simultaneously in videos. Experiments verify that our online histogram generative model constrained by key appearance histograms alleviates the drifting problem often encountered in tracking with online updating, that the enhanced meanshift algorithm is capable of tracking object of varying appearances more robustly and accurately, and that our tracking algorithm can infer additional information such as the object poses.

An online classification EM algorithm based on the mixture model

Mixture model-based clustering is widely used in many applications. In certain real-time applications the rapid increase of data size with time makes classical clustering algorithms too slow. An online clustering algorithm based on mixture models is presented in the context of a real-time flaw-diagnosis application for pressurized containers which uses data from acoustic emission signals. The proposed algorithm is a stochastic gradient algorithm derived from the classification version of the EM algorithm (CEM). It provides a model-based generalization of the well-known online k-means algorithm, able to handle non-spherical clusters. Using synthetic and real data sets, the proposed algorithm is compared with the batch CEM algorithm and the online EM algorithm. The three approaches generate comparable solutions in terms of the resulting partition when clusters are relatively well separated, but online algorithms become faster as the size of the available observations increases.

Online EM algorithm for mixture with application to internet traffic modeling

Since histograms of many real network traces show strong evidence of mixture, this paper uses mixture distributions to model Internet traffic and applies the EM algorithm to fit the models. Making use of the fact that at each iteration of the EM algorithm the parameter increment has a positive projection on the gradient of the likelihood function, this paper proposes an online EM algorithm to fit the models and the Bayesian Information Criterion is applied to select the best model. Experimental results on real traces are provided to illustrate the efficiency of the proposed algorithm.

Hierarchical GMM to handle sharp changes in moving object detection

The Gaussian mixture model (GMM) is an important background model of background subtraction methods in moving object detection, and is fit to deal with gradual changes of illumination. To handle sharp changes, hierarchical GMM (HGMM) is proposed as a generic solution which uses state models without temporal correlation on different scales. A new on-line EM algorithm is devised to model new states quickly and accurately. Experiments show that the presented method brings fast adaptation to sharp changes of illumination.

Robust online appearance models for visual tracking

We propose a framework for learning robust, adaptive, appearance models to be used for motion-based tracking of natural objects. The model adapts to slowly changing appearance, and it maintains a natural measure of the stability of the observed image structure during tracking. By identifying stable properties of appearance, we can weight them more heavily for motion estimation, while less stable properties can be proportionately downweighted. The appearance model involves a mixture of stable image structure, learned over long time courses, along with two-frame motion information and an outlier process. An online EM-algorithm is used to adapt the appearance model parameters over time. An implementation of this approach is developed for an appearance model based on the filter responses from a steerable pyramid. This model is used in a motion-based tracking algorithm to provide robustness in the face of image outliers, such as those caused by occlusions, while adapting to natural changes in appearance such as those due to facial expressions or variations in 3D pose.

On-Line EM Reinforcement Learning for Automatic Control of Continuous Dynamical Systems

In this paper, we propose a new reinforcement learning (RL) method for dynamical systems that have continuous state and action spaces. Our RL method has an architecture like the actorcritic model. The critic tries to approximate the Q-function, and the actor tries to approximate a stochastic soft-max policy dependent on the Q-function. An on-line EM algorithm is used to train the critic and the actor. We apply this method to two control problems. Computer simulations in two tasks show that our method is able to acquire good control after a few learning trials.

Reconstruction of chaotic dynamics by on-line EM algorithm

In this paper, we discuss the reconstruction of chaotic dynamics by using a normalized Gaussian network (NGnet), which is a network of local linear regression units. The NGnet is trained by an on-line EM algorithm in order to learn the vector field of the chaotic dynamics. We investigate the robustness of our approach to two kinds of noise processes: system noise and observation noise. System noise disturbs the system dynamics itself. Observation noise is added when the state variables are observed. It is shown that the trained NGnet is able to reproduce a chaotic attractor, which approximates the complexity and instability of the original chaotic attractor well, even under various noise conditions. The trained NGnet also shows good prediction performance. When only part of the dynamical variables are observed, the delay embedding method is used. The NGnet is then trained to learn the vector field in the delay coordinate space. Experiments show that the chaotic dynamics embedded in the delay coordinate space is able to be learned under the two types of noise.

Application of reinforcement learning based on on‐line EM algorithm to balancing of acrobot

AbstractThe acrobot is a two‐link robot, actuated only at the joint between the two links. It is a difficult task in reinforcement learning (RL) to control the acrobot because it has nonlinear dynamics and continuous state and action spaces. In this article, we discuss applying the RL to the task of balancing control of the acrobot. Our RL method has an architecture similar to the actor‐critic model. The actor is a controller that yields a control signal for the current state. The critic predicts the expected return in the future. The actor and the critic are approximated by normalized Gaussian networks, which are trained by an on‐line EM algorithm. We also introduce a new method to promote the critic's learning. Our computer simulation shows that our method is able to acquire fairly good control through a small number of learning trials. © 2001 Scripta Technica, Syst Comp Jpn, 32(5): 12–20, 2001

Learning chaotic dynamics under noise with on-line EM algorithm

Learning chaotic dynamics under noise with on-line EM algorithm

On-line EM Algorithm for the Normalized Gaussian Network

A normalized gaussian network (NGnet) (Moody & Darken, 1989) is a network of local linear regression units. The model softly partitions the input space by normalized gaussian functions, and each local unit linearly approximates the output within the partition. In this article, we propose a new on-line EMalgorithm for the NGnet, which is derived from the batch EMalgorithm (Xu, Jordan, &Hinton 1995), by introducing a discount factor. We show that the on-line EM algorithm is equivalent to the batch EM algorithm if a specific scheduling of the discount factor is employed. In addition, we show that the on-line EM algorithm can be considered as a stochastic approximation method to find the maximum likelihood estimator. A new regularization method is proposed in order to deal with a singular input distribution. In order to manage dynamic environments, where the input-output distribution of data changes over time, unit manipulation mechanisms such as unit production, unit deletion, and unit division are also introduced based on probabilistic interpretation. Experimental results show that our approach is suitable for function approximation problems in dynamic environments. We also apply our on-line EM algorithm to robot dynamics problems and compare our algorithm with the mixtures-of-experts family.