General Hidden Markov Models Research Articles

Asynchronous Boundary Stabilization of Stochastic Markov Jump Reaction-Diffusion Systems

Dissipativity-based asynchronous boundary stabilization problem is addressed for stochastic Markov jump reaction-diffusion systems (SMJRDSs). In practical engineering, nonsynchronous behavior between system modes and controller modes is inevitable, and the incomplete matrix information makes the problem analysis difficult, so this work considers the asynchronous stabilization. Different from the distributed control, we apply a simple boundary control strategy, which greatly reduces the cost of the control design. Note that three issues need to be addressed: 1) how to model the asynchronous behavior? 2) how to design the asynchronous boundary controller? and 3) how to process the incomplete matrix information? We deal with these problems one by one. Based on a general hidden Markov model (HMM), an asynchronous boundary feedback controller is considered. Via the Wirtinger-type inequality, Schur complement technique, and transition matrix properties, sufficient conditions ensuring exponentially mean square stability and strictly <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(W, P, R)-\alpha $ </tex-math></inline-formula> dissipativity are established, which covers several special cases. Finally, a numerical example is presented to illustrate the proposed control strategies.

Apnea bradycardia detection based on new coupled hidden semi Markov model.

In this paper, a method for apnea bradycardia detection in preterm infants is presented based on coupled hidden semi Markov model (CHSMM). CHSMM is a generalization of hidden Markov models (HMM) used for modeling mutual interactions among different observations of a stochastic process through using finite number of hidden states with corresponding resting time. We introduce a new set of equations for CHSMM to be integrated in a detection algorithm. The detection algorithm was evaluated on a simulated data to detect a specific dynamic and on a clinical dataset of electrocardiogram signals collected from preterm infants for early detection of apnea bradycardia episodes. For simulated data, the proposed algorithm was able to detect the desired dynamic with sensitivity of 96.67% and specificity of 98.98%. Furthermore, the method detected the apnea bradycardia episodes with 94.87% sensitivity and 96.52% specificity with mean time delay of 0.73 s. The results show that the algorithm based on CHSMM is a robust tool for monitoring of preterm infants in detecting apnea bradycardia episodes. Graphical Abstract Apnea Bradycardia detection using Coupled hidden semi Markov Model from electrocardiography. In this model, a sequence of hidden states is assigned to each observation based on the effects of previous states of all observations.

Automated recognition and discrimination of human–animal interactions using Fisher vector and hidden Markov model

Human–animal interactions may affect the animal welfare and productivity in rearing environments. Previously proposed human–animal-related techniques focus on the manual discrimination of single animal behaviors or simple human–animal interactions. To address the automatic detection and classification of complex animal behaviors and the animals reactions to human, we propose an approach built upon both the visual representation with Fisher vectors and the end-to-end generative hidden Markov model to facilitate the discrimination of both coarse- and fine-grained animal–human interactions. To satisfy the requirement for abundant data samples of the generative approach, we recorded and annotated more than 480 hours of videos featuring eight persons and 210 laying hens during the process of feeding and cleaning. The experimental results show that the proposed method outperforms state-of-the-art approaches. According to the experimental performance of our method on practical videos, our approach can be used to monitor the human–animal interactions or animal behaviors in modern poultry farms.

A Markov-Switching model for building occupant activity estimation

Heating and ventilation strategies in buildings can be improved significantly if information about the current presence and activity level of the occupants is taken into account. Therefore, there is a high demand for inexpensive sensor-based methods to detect the occupancy or occupant activity level. It is shown that the carbon dioxide (CO2) level in a room is dependent on the activity level rather than only on just the number of people. Therefore, this study suggests a new model based on the use of CO2 trajectories to estimate the occupant activity level, trained on measurements both from a school classroom and from a Danish summerhouse. A hidden Markov-switching model was employed to identify the activity level. This modelling approach is a generalization of hidden Markov models, taking autocorrelation in the observations into account. This is done by an additional autoregressive part which models the persistence of the CO2 concentration by relating the current value to past lags. The analysis of one-step prediction residuals shows that this method inherits the dynamics of the CO2 curves much better than an ordinary hidden Markov model, and can therefore be considered a promising candidate for occupant activity estimation. Furthermore, it is shown that the presented model can be used for simulations of activity level and of the accompanying CO2 levels.

Particle-based online estimation of tangent filters with application to parameter estimation in nonlinear state-space models

This paper presents a novel algorithm for efficient online estimation of the filter derivatives in general hidden Markov models. The algorithm, which has a linear computational complexity and very limited memory requirements, is furnished with a number of convergence results, including a central limit theorem with an asymptotic variance that can be shown to be uniformly bounded in time. Using the proposed filter derivative estimator, we design a recursive maximum likelihood algorithm updating the parameters according the gradient of the one-step predictor log-likelihood. The efficiency of this online parameter estimation scheme is illustrated in a simulation study.

Variational Bayesian Learning of Generalized Dirichlet-Based Hidden Markov Models Applied to Unusual Events Detection.

Learning a hidden Markov model (HMM) is typically based on the computation of a likelihood which is intractable due to a summation over all possible combinations of states and mixture components. This estimation is often tackled by a maximization strategy, which is known as the Baum-Welch algorithm. However, some drawbacks of this approach have led to the consideration of Bayesian methods that add a prior over the parameters in order to work with the posterior probability and the marginal likelihood. These approaches can lead to good models but to the cost of extremely long computations (e.g., Markov Chain Monte Carlo). More recently, variational Bayesian frameworks have been proposed as a Bayesian alternative that keeps the computation tractable and the approximation tight. It relies on the introduction of a prior over the parameters to be learned and on an approximation of the true posterior distribution. After proving good standing in the case of finite mixture models and discrete and Gaussian HMMs, we propose here to derive the equations of the variational learning of the Dirichlet mixture-based HMM, and to extend it to the generalized Dirichlet. The latter case presents several properties that make the estimation more accurate. We prove the validity of this approach within the context of unusual event detection in public areas using the University of California San Diego data sets. HMMs are trained over normal video sequences using the typical Baum-Welch approach versus the variational one. The variational learning leads to more accurate models for the detection and localization of anomaly, and the general HMM approach is shown to be versatile enough to handle the detection of various synthetically generated tampering events.

EMG Channel Selection for Improved Hand Gesture Classification.

Electromyographic activities (EMG) from upper limb muscles exhibit distinct patterns associated with different hand gestures. To capture these EMG activities, an appropriate number of sensors (channels) affixed at right locations is needed. This paper presents a novel approach to automatically identify EMG channels most salient to classifying different hand gestures. The proposed approach is based on a regularized generative-discriminative encoding of time-series EMG data. The proposed approach 1) encodes each time-series channel of multi-channel EMG observations into a generative hidden Markov model (HMM), 2) constructs a shared probabilistic embedding space, where EMG observations are represented in terms of pair-wise distances between the channel-specific HMMs, 3) weighs different dimensions of the shared probabilistic space using multinomial logistic regression with group least absolute shrinkage and selection operator (group Lasso) penalty, with each group corresponding to channel-specific distances (i.e., distances between a channel and the rest of the channels), 4) determines salient channels based on the weighted parameters, and 5) classifies gestures based on the identified salient channels. The performance of the proposed approach was evaluated using the NinaPro2 dataset. The identified salient EMG channels improved classification accuracies as high as 11{% compared with the case where all the channels were used.

Detection of Irregular Behavior in Room Using Environmental Sensors and Power Consumption of Home Appliances Learning in HMMs

We propose a human behavior detect method based on our development system of multifunctional outlet. This is a low-power sensor network system that can recognize human behavior without any wearable devices. In order to detect human regular daily behaviors, we setup various sensors in rooms and use them to record daily lives. In this paper we present a monitoring method of unusual behaviors, and it also can be used for healthcare and so on. We use Hidden Markov Model(HMM), and set two series HMM input to recognize irregular movement from daily lives, One is time sequential sensor data blocks whose sensor values are binarized and splitted by its response. And the other is time sequential labels using Support Vector Machine (SVM). In experiments, our developed sensor network system logged 34days data. HMM learns data of the first 34days that include only usual daily behaviors as training data, and then evaluates the last 8 days that include unusual behaviors.&#x0D; Index Terms—multifunctional outlet system; behavior detection; hidden markov model; sensor network; support vector machine.&#x0D; REFERENCES&#x0D; [1] T.Sasama, S.Iwasaki, and T.Okamoto, “Sensor Data Classification for Indoor Situation Using the Multifunctional Outlet”, The Institute of Electrinical Engineers of Japan, vol.134(7),2014,pp.949-995&#x0D; [2] M.Anjali Manikannan, R.Jayarajan, “Wireless Sensor Netwrork For Lonely Elderly Perple Wellness”, International Journal of Advanced Computational Engineering and Networking, ISSN: 2320-2106, vol. 3, 2015, pp.41-45&#x0D; [3] Nagender Kumar Suryadevara, “Wireless Sensor Network Based Home Monitoring System for Wellness Determination of Elderly”, IEEE SENSORS JOURNAL, VOL. 12, NO. 6, JUNE 2012, pp. 1965-1972.&#x0D; [4] iTec Co., safety confirmation system: Mimamorou, http://www.minamoro.biz/.&#x0D; [6] Alexander Schliep's group for bioinformatics, The General Hidden Markov Model library(GHMM), http://ghmm.sourceforge.net/.&#x0D; [7] Jr Joe H.Ward, Joumal of the American Statistical Association, vol58(301), 1963, pp236-244 [5] SOLXYZ Co., status monitoring system:Ima-Irumo, http://www.imairumo.com/.

Bayesian switching factor analysis for estimating time-varying functional connectivity in fMRI

There is growing interest in understanding the dynamical properties of functional interactions between distributed brain regions. However, robust estimation of temporal dynamics from functional magnetic resonance imaging (fMRI) data remains challenging due to limitations in extant multivariate methods for modeling time-varying functional interactions between multiple brain areas. Here, we develop a Bayesian generative model for fMRI time-series within the framework of hidden Markov models (HMMs). The model is a dynamic variant of the static factor analysis model (Ghahramani and Beal, 2000). We refer to this model as Bayesian switching factor analysis (BSFA) as it integrates factor analysis into a generative HMM in a unified Bayesian framework. In BSFA, brain dynamic functional networks are represented by latent states which are learnt from the data. Crucially, BSFA is a generative model which estimates the temporal evolution of brain states and transition probabilities between states as a function of time. An attractive feature of BSFA is the automatic determination of the number of latent states via Bayesian model selection arising from penalization of excessively complex models. Key features of BSFA are validated using extensive simulations on carefully designed synthetic data. We further validate BSFA using fingerprint analysis of multisession resting-state fMRI data from the Human Connectome Project (HCP). Our results show that modeling temporal dependencies in the generative model of BSFA results in improved fingerprinting of individual participants. Finally, we apply BSFA to elucidate the dynamic functional organization of the salience, central-executive, and default mode networks—three core neurocognitive systems with central role in cognitive and affective information processing (Menon, 2011). Across two HCP sessions, we demonstrate a high level of dynamic interactions between these networks and determine that the salience network has the highest temporal flexibility among the three networks. Our proposed methods provide a novel and powerful generative model for investigating dynamic brain connectivity.

Observable Operator Models

This paper describes a new approach to model discrete stochastic processes, called observable operator models (OOMs). The OOMs were introduced by Jaeger as a generalization of hidden Markov models (HMMs). The theory of OOMs makes use of both probabilistic and linear algebraic tools, which has an important advantage: using the tools of linear algebra a very simple and efficient learning algorithm can be developed for OOMs. This seems to be better than the known algorithms for HMMs. This learningalgorithm is presented in detail in the second part of the article.

Beyond Bell’s Theorem II: Scenarios with Arbitrary Causal Structure

It has recently been found that Bell scenarios are only a small subclass of interesting setups for studying the non-classical features of quantum theory within spacetime. We find that it is possible to talk about classical correlations, quantum correlations and other kinds of correlations on any directed acyclic graph, and this captures various extensions of Bell scenarios which have been considered in the literature. From a conceptual point of view, the main feature of our approach is its high level of unification: while the notions of source, choice of setting and measurement play all seemingly different roles in a Bell scenario, our formalism shows that they are all instances of the same concept of "event". Our work can also be understood as a contribution to the subject of causal inference with latent variables. Among other things, we introduce hidden Bayesian networks as a generalization of hidden Markov models.

Efficient particle-based online smoothing in general hidden Markov models: The PaRIS algorithm

This thesis consists of two papers studying online inference in general hidden Markov models using sequential Monte Carlo methods.The first paper present an novel algorithm, the particle-based, rapid incremental smoother (PaRIS), aimed at efficiently perform online approximation of smoothed expectations of additive state functionals in general hidden Markov models. The algorithm has, under weak assumptions, linear computational complexity and very limited memory requirements. The algorithm is also furnished with a number of convergence results, including a central limit theorem.The second paper focuses on the problem of online estimation of parameters in a general hidden Markov model. The algorithm is based on a forward implementation of the classical expectation-maximization algorithm. The algorithm uses the PaRIS algorithm to achieve an efficient algorithm.

Recognition of facial actions and their temporal segments based on duration models

Being able to automatically analyze finegrained changes in facial expression into action units (AUs), of the Facial Action Coding System (FACS), and their temporal models (i.e., sequences of temporal phases, neutral, onset, apex, and offset), in face videos would greatly benefit for facial expression recognition systems. Previous works, considered combining, per AU, a discriminative frame-based Support Vector Machine (SVM) and a dynamic generative Hidden Markov Models (HMM), to detect the presence of the AU in question and its temporal segments in an input image sequence. The major drawback of HMMs, is that they do not model well time dependent dynamics as the ones of AUs, especially when dealing with spontaneous expressions. To alleviate this problem, in this paper, we exploit efficient duration modeling of the temporal behavior of AUs, and we propose hidden semi-Markov model (HSMM) and variable duration semi-Markov model (VDHMM) to recognize the dynamics of AU's. Such models allow the parameterization and inference of the AU's state duration distributions. Within our system, geometrical and appearance based measurements, as well as their first derivatives, modeling both the dynamics and the appearance of AUs, are applied to pair-wise SVM classifiers for a frame-based classification. The output of which are then fed as evidence to the HSMM or VDHMM for inferring AUs temporal phases. A thorough investigation into the aspect of duration modeling and its application to AU recognition through extensive comparison to state-of-art SVM-HMM approaches are presented. For comparison, an average recognition rate of 64.83 % and 64.66 % is achieved for the HSMM and VDHMM respectively. Our framework has several benefits: (1) it models the AU's temporal phases duration; (2) it does not require any assumption about the underlying structure of the AU events, and (3) compared to HMM, the proposed HSMM and VDHMM duration models reduce the duration error of the temporal phases of an AU, and they are especially better in recognizing the offset ending of an AU.

Long-term stability of sequential Monte Carlo methods under verifiable conditions

This paper discusses particle filtering in general hidden Markov models (HMMs) and presents novel theoretical results on the long-term stability of bootstrap-type particle filters. More specifically, we establish that the asymptotic variance of the Monte Carlo estimates produced by the bootstrap filter is uniformly bounded in time. On the contrary to most previous results of this type, which in general presuppose that the state space of the hidden state process is compact (an assumption that is rarely satisfied in practice), our very mild assumptions are satisfied for a large class of HMMs with possibly noncompact state space. In addition, we derive a similar time uniform bound on the asymptotic $\mathsf{L}^p$ error. Importantly, our results hold for misspecified models; that is, we do not at all assume that the data entering into the particle filter originate from the model governing the dynamics of the particles or not even from an HMM.

Handling signal variability with contextual markovian models

There are two popular families of statistical models for dealing with sequences and in particular with handwriting signals, either on-line or off-line, the well known generative hidden Markov models and the more recently proposed discriminative Hidden Conditional Random Fields.One key issue in such modeling frameworks is to efficiently handle variability. The traditional approach consists in first removing as much as possible signal variability in the preprocessing stage, and to use more complex models, for instance in the case of hidden Markov models one increases the number of states and the Gaussian mixture size.We focus here on another kind of approaches where the probability distribution implemented by the models depends on a number of additional contextual variables, that are assumed fixed or that vary slowly along a sequence. The context may stand for emotion features in speech recognition, physical features in gesture recognition, gender, age, etc.We propose a framework for deriving markovian models that make use of such contextual information. This yields new models that we call Contextual hidden Markov models and contextual Hidden Conditional Random Fields. We detail learning algorithms for both models and investigate their performances on the IAM off-line handwriting dataset.

Online Expectation Maximization based algorithms for inference in Hidden Markov Models

The Expectation Maximization (EM) algorithm is a versatile tool for model parameter estimation in latent data models. When processing large data sets or data stream however, EM becomes intractable since it requires the whole data set to be available at each iteration of the algorithm. In this contribution, a new generic online EM algorithm for model parameter inference in general Hidden Markov Model is proposed. This new algorithm updates the parameter estimate after a block of observations is processed (online). The convergence of this new algorithm is established, and the rate of convergence is studied showing the impact of the block-size sequence. An averaging procedure is also proposed to improve the rate of convergence. Finally, practical illustrations are presented to highlight the performance of these algorithms in comparison to other online maximum likelihood procedures.

Melody Harmonization With Interpolated Probabilistic Models

Most melody harmonization systems use the generative hidden Markov model (HMM), which model the relation between the hidden chords and the observed melody. Relations to other variables, such as the tonality or the metric structure, are handled by training multiple HMMs or are ignored. In this paper, we propose a discriminative means of combining multiple probabilistic models of various musical variables by means of model interpolation. We evaluate our models in terms of their cross-entropy and their performance in harmonization experiments. The proposed model offered higher chord root accuracy than the reference musicological rule-based harmonizer by up to 5% absolute.

Predictive models and generative complexity

The causal states of computational mechanics define the minimal sufficient memory for a given discrete stationary stochastic process. Their entropy is an important complexity measure called statistical complexity (or true measure complexity). They induce the ɛ-machine, which is a hidden Markov model (HMM) generating the process. But it is not the minimal one, although generative HMMs also have a natural predictive interpretation. This paper gives a mathematical proof of the idea that the ɛ-machine is the minimal HMM with an additional (partial) determinism condition. Minimal internal state entropy of a generative HMM is in analogy to statistical complexity called generative complexity. This paper also shows that generative complexity depends on the process in a nice way. It is, as a function of the process, lower semi-continuous (w.r.t. weak-* topology), concave, and behaves nice under ergodic decomposition of the process.

Sequential Monte Carlo smoothing for general state space hidden Markov models

Computing smoothing distributions, the distributions of one or more states conditional on past, present, and future observations is a recurring problem when operating on general hidden Markov models. The aim of this paper is to provide a foundation of particle-based approximation of such distributions and to analyze, in a common unifying framework, different schemes producing such approximations. In this setting, general convergence results, including exponential deviation inequalities and central limit theorems, are established. In particular, time uniform bounds on the marginal smoothing error are obtained under appropriate mixing conditions on the transition kernel of the latent chain. In addition, we propose an algorithm approximating the joint smoothing distribution at a cost that grows only linearly with the number of particles.

Maximum likelihood estimation for general hidden semi-Markov processes with backward recurrence time dependence

This paper concerns the study of asymptotic properties of the maximum likelihood estimator (MLE) for the general hidden semi-Markov model (HSMM) with backward recurrence time dependence. By transforming the general HSMM into a general hidden Markov model, we prove that under some regularity conditions, the MLE is strongly consistent and asymptotically normal. We also provide useful expressions for asymptotic covariance matrices, involving the MLE of the conditional sojourn times and the embedded Markov chain of the hidden semi-Markov chain. Bibliography: 17 titles.