Theory Of Hidden Markov Models Research Articles

DeepContext: Mobile Context Modeling and Prediction Via HMMs and Deep Learning

Mobile context determination is an important step for many context-aware services such as location-based services, enterprise policy enforcement, building/room occupancy detection for power/HVAC operation, etc. Especially in enterprise scenarios where policies (e.g., attending a confidential meeting only when the user is in &#x201C;Location X&#x201D;) are defined based on mobile context, it is paramount to verify the accuracy of the mobile context. Most of the existing solutions rely on context obtained directly from sensors which could be hacked, noisy or insufficient, which cannot be relied upon for security applications. In this article, we take a different approach by modeling mobile context based on past context data of related users and considering its unique challenges such as missing features. To this end, we propose three models for modeling mobile context based on symbolic time series data of feature-value pairs&#x2014;two stochastic models based on the theory of Hidden Markov Models&#x00A0;(HMMs) and one model based on deep learning&#x2014;<i>personalized model</i> (<i>HPContext</i>), <i>collaborative filtering model</i> (<i>(HCFContext)</i>), and <i>deep learning model</i> (<i>DeepContext</i>) to eventually replace <i>HPContext</i>. <i>HPContext</i> and <i>DeepContext</i> predict the current context using sequential history of the user&#x0027;s own past context observations; while <i>HCFContext</i> enhances the former with collaborative filtering features, which enables it to predict the current context of the primary user based on the context observations of users related to the primary user, e.g., same team colleagues in the company, gym friends, family members, etc. <i>DeepContext</i> models mobile context based on symbolic (i.e., categorical valued rather than continuous-valued) time series data using deep learning techniques. These models are then used to determine the context of the primary user at the current instant or some timesteps in to the future. Each of the proposed models can also be used to enhance or complement the context obtained from sensors. Finally, these models are thoroughly validated on a real-life dataset.

Hidden Markov Model of Lane-Changing-Based Car-Following Behavior on Freeways using Naturalistic Driving Data

This paper develops a methodology for simultaneously modeling lane-changing and car-following behavior of automated vehicles on freeways. Naturalistic driving data from the Safety Pilot Model Deployment (SPMD) program are used. First, a framework to process the SPMD data is proposed using various data analytics techniques including data fusion, data mining, and machine learning. Second, pairs of automated host vehicle and their corresponding front vehicle are identified along with their lane-change and car-following relationship data. Using these data, a lane-changing-based car-following (LCCF) model, which explicitly considers lane-change and car-following behavior simultaneously, is developed. The LCCF model is based on Gaussian-mixture-based hidden Markov model theory and is disaggregated into two processes: LCCF association and LCCF dissociation. These categories are based on the result of the lane change. The overall goal is to predict a driver’s lane-change intention using the LCCF model. Results show that the model can predict the lane-change event in the order of 0.6 to 1.3 s before the moment of the vehicle body across the lane boundary. In addition, the execution times of lane-change maneuvers average between 0.55 and 0.86 s. The LCCF model allows the intention time and execution time of driver’s lane-change behavior to be forecast, which will help to develop better advanced driver assistance systems for vehicle controls with respect to lane-change and car-following warning functions.

Hidden Markov Model of Two-Component System with Group Instantly Replenished Time Reserve

Most systems allow the construction of a semi-Markov model. However, during the operation of the system, full information contained in the state encoding is not always available, but it is possible to obtain some signal (information). Tasks arise to assess the consistency of the model with the received data (signals), to refine the model and its parameters. Such parameters can be characteristics of random values characterizing system operation, time reserve value, etc. The theory of hidden Markov models allows solving these problems. In order to move from a semi-Markov model of the system to its hidden Markov model, it is proposed to first the semi-Markov model merge using a stationary phase merging algorithm. In this paper, on the basis of the semi-Markov model with a common phase state space of a two-component system with a group instantly replenished timereserve, we construct a hidden Markov model of a two-component system with a group instantly replenished time reserve. It is used to evaluate the characteristics and predict the states of the system in question based on the received vector of signals. The influence of the time reserve value on the probability of occurrence of the obtained vector of signals is shown.

Clustering of samples with a tree-shaped dependence structure, with an application to microscopic time lapse imaging.

Recent imaging technologies allow for high-throughput tracking of cells as they migrate, divide, express fluorescent markers and change their morphology. The interpretation of these data requires unbiased, efficient statistical methods that model the dynamics of cell phenotypes. We introduce treeHFM, a probabilistic model which generalizes the theory of hidden Markov models to tree structured data. While accounting for the entire genealogy of a cell, treeHFM categorizes cells according to their primary phenotypic features. It models all relevant events in a cell's life, including cell division, and thereby enables the analysis of event order and cell fate heterogeneity. Simulations show higher accuracy in predicting correct state labels when modeling the more complex, tree-shaped dependency of samples over standard HMM modeling. Applying treeHFM to time lapse images of hematopoietic progenitor cell differentiation, we demonstrate that progenitor cells undergo a well-ordered sequence of differentiation events. The treeHFM is implemented in C++. We provide wrapper functions for the programming languages R (CRAN package, https://CRAN.R-project.org/package=treeHFM) and Matlab (available at Mathworks Central, http://se.mathworks.com/matlabcentral/fileexchange/57575-treehfml). Supplementary data are available at Bioinformatics online.

Minimal Realization Problems for Hidden Markov Models

This paper addresses two fundamental problems in the context of hidden Markov models (HMMs). The first problem is concerned with the characterization and computation of a minimal order HMM that realizes the exact joint densities of an output process based on only finite strings of such densities (known as HMM partial realization problem). The second problem is concerned with learning a HMM from finite output observations of a stochastic process. We review and connect two fields of studies: realization theory of HMMs, and the recent development in spectral methods for learning latent variable models. Our main results in this paper focus on generic situations, namely, statements that will be true for almost all HMMs, excluding a measure zero set in the parameter space. In the main theorem, we show that both the minimal quasi-HMM realization and the minimal HMM realization can be efficiently computed based on the joint probabilities of length $N$ strings, for $N$ in the order of ${\cal O}(\log_{d}(k))$ . In other words, learning a quasi-HMM and an HMM have comparable complexity for almost all HMMs.

A Hidden Markov Models Approach for Crop Classification: Linking Crop Phenology to Time Series of Multi-Sensor Remote Sensing Data

Vegetation monitoring and mapping based on multi-temporal imagery has recently received much attention due to the plethora of medium-high spatial resolution satellites and the improved classification accuracies attained compared to uni-temporal approaches. Efficient image processing strategies are needed to exploit the phenological information present in temporal image sequences and to limit data redundancy and computational complexity. Within this framework, we implement the theory of Hidden Markov Models in crop classification, based on the time-series analysis of phenological states, inferred by a sequence of remote sensing observations. More specifically, we model the dynamics of vegetation over an agricultural area of Greece, characterized by spatio-temporal heterogeneity and small-sized fields, using RapidEye and Landsat ETM+ imagery. In addition, the classification performance of image sequences with variable spatial and temporal characteristics is evaluated and compared. The classification model considering one RapidEye and four pan-sharpened Landsat ETM+ images was found superior, resulting in a conditional kappa from 0.77 to 0.94 per class and an overall accuracy of 89.7%. The results highlight the potential of the method for operational crop mapping in Euro-Mediterranean areas and provide some hints for optimal image acquisition windows regarding major crop types in Greece.

An inverse Yarbus process: predicting observers' task from eye movement patterns.

In this paper we develop a probabilistic method to infer the visual-task of a viewer given measured eye movement trajectories. This method is based on the theory of hidden Markov models (HMM) that employs a first order Markov process to predict the coordinates of fixations given the task. The prediction confidence level of each task-dependent model is used in a Bayesian inference formulation, whereby the task with the maximum a posteriori (MAP) probability is selected. We applied this technique to a challenging dataset consisting of eye movement trajectories obtained from subjects viewing monochrome images of real scenes tasked with answering questions regarding the scenes. The results show that the HMM approach, combined with a clustering technique, can be a reliable way to infer visual-task from eye movements data.

Hidden Markov models revealing the stress field underlying the earthquake generation

The application of the hidden Markov models (HMMs) is attempted for revealing key features for the earthquake generation which are not accessible to direct observation. Considering that the states of the HMM correspond to levels of the stress field, our objective is to identify these states. The observations are considered after grouping earthquake magnitudes and the cases of different number of states are examined. The problems of HMMs theory are solved and the ensuing HMMs are compared on the basis of Akaike and Bayesian information criteria. A new insight on the evaluation of future seismic hazard is given by calculating the mean number of steps for the first visit to a particular state, along with the respective variance. We further calculate an estimator of the mean number of steps for the first visit to a particular state and we construct its confidence interval. Additionally, a second approach to the problem is followed by assuming a different determination of observations. The HMMs applied to both approaches, contribute significantly to seismic hazard assessment via revealing the number of the stress levels as well as the way in which these levels are associated with certain earthquake occurrence.

A New Stochastic Model of Electric Arc Furnace Based on Hidden Markov Model: A Study of Its Effects on the Power System

Electric arc furnaces (EAFs) are nonlinear and time-variant loads that cause severe fluctuations of current, voltage, and current harmonics, voltage flicker, frequency changes, etc. The requirement is to present an accurate model to analyze the function of these types of loads. In this paper, a new model based on hidden Markov model theory is presented for an electric arc. For this purpose, the characteristic of the arc voltage-current is divided into four regions. Then, on the basis of actual measured samples of an electric arc in several functioning cycles of EAF, different operating points are generated in the form of statistical probability, corresponding to hidden Markov theory in the aforementioned four regions. The arc model is designed in the form of a current-dependent voltage source. Therefore, the proposed model is without linear approximation and corresponds to the actual state. Also, in order to be accurate in flicker modeling, a voltage is randomly considered for the arc model. After designing the aforementioned model, the effects resulting from this electric load in the power system are studied and analyzed. The simulation results are compared with the measured arc to show the accuracy and soundness of the model.

Identifying bubble collapse in a hydrothermal system using hidden Markov models

Beginning in July 2003 and lasting through September 2003, the Norris Geyser Basin in Yellowstone National Park exhibited an unusual increase in ground temperature and hydrothermal activity. Using hidden Markov model theory, we identify over five million high‐frequency (&gt;15 Hz) seismic events observed at a temporary seismic station deployed in the basin in response to the increase in hydrothermal activity. The source of these seismic events is constrained to within ∼100 m of the station, and produced ∼3500–5500 events per hour with mean durations of ∼0.35–0.45 s. The seismic event rate, air temperature, hydrologic temperatures, and surficial water flow of the geyser basin exhibited a marked diurnal pattern that was closely associated with solar thermal radiance. We interpret the source of the seismicity to be due to the collapse of small steam bubbles in the hydrothermal system, with the rate of collapse being controlled by surficial temperatures and daytime evaporation rates.

Efficient methods for sampling spike trains in networks of coupled neurons

Monte Carlo approaches have recently been proposed to quantify connectivity in neuronal networks. The key problem is to sample from the conditional distribution of a single neuronal spike train, given the activity of the other neurons in the network. Dependencies between neurons are usually relatively weak; however, temporal dependencies within the spike train of a single neuron are typically strong. In this paper we develop several specialized Metropolis--Hastings samplers which take advantage of this dependency structure. These samplers are based on two ideas: (1) an adaptation of fast forward--backward algorithms from the theory of hidden Markov models to take advantage of the local dependencies inherent in spike trains, and (2) a first-order expansion of the conditional likelihood which allows for efficient exact sampling in the limit of weak coupling between neurons. We also demonstrate that these samplers can effectively incorporate side information, in particular, noisy fluorescence observations in the context of calcium-sensitive imaging experiments. We quantify the efficiency of these samplers in a variety of simulated experiments in which the network parameters are closely matched to data measured in real cortical networks, and also demonstrate the sampler applied to real calcium imaging data.

A condition-based maintenance policy and input parameters estimation for deteriorating systems under periodic inspection

This paper combines an optimization model and input parameters estimation from empirical data, in order to propose condition-based maintenance policies. The system deterioration is described by discrete states ordered from the state “as good as new” to the state “completely failed”. At each periodic inspection, whose outcome might not be accurate, a decision has to be made between continuing to operate the system or stopping and performing its preventive maintenance. We explore the problem of how to estimate the model input parameters, i.e., how to adequate the model inputs to the empirical data available. For this purpose, we use the Hidden Markov Model theory. The literature has not explored the combination of optimization techniques and model input parameters, through historical data, for problems with imperfect information such as the one considered in this paper. We thoroughly discuss our approach, illustrate it with empirical data and also point out directions for future research.

Analysis of electron tunneling events with the hidden Markov model

The charge fluctuations of a quantum dot defined by depletion gates in a semiconductor heterostructure can be observed using a charge sensor. The charge sensor can observe electrons transiting on and off of the quantum dot in real time. From such data information about the quantum states of electrons on the dot can be inferred. We present an approach to analyzing charge sensor data based on the hidden Markov model (HMM). HMM theory provides a mathematical approach for inferring the details of a stochastic process from indirect observations. We discuss how this applies to the problem of charge sensor data analysis. We apply HMMs to data from a previous quantum dot experiment and demonstrate its usefulness in extracting the electron transition rates. Further potential for the HMM in the context of quantum dot experiments is discussed.

A new nonlinear filter for parameters identification in dynamic systems and application to a transmission channel

In this paper we propose general nonlinear models for off-line and on-line parameters identification in dynamic systems. These numerical filters can be applied to any nonlinear system represented by a state equation and an observation equation both nonlinear. The theory of hidden Markov models is used to derive these algorithms starting from a of Baum & Welch type method. The proposed identification algorithm has two levels: the first level is an iterative and global algorithm (IGA), it estimates iteratively the parameters from a block of data. The second level is an ergodic recursive algorithm (ERA), it estimates the parameters in an adaptive manner. The estimators defined by these algorithms converge almost surely to the true values of the model parameters studied in [A. Khoukhi, T. Aliziane, M. Souilah, Un Algorithme Multi-Niveau d’Identification d’un Canal en Communication Numérique, JESA 36(4) (2002) 519–537; M. Souilah, A. Khoukhi, T. Aliziane, A new multi-level algorithm for identification and stochastic adaptive control of industrial manipulators, Eng. Simulation 26(4) (2004) 83– 98; M. Souilah, A new strategy for identification and control of mobile robots, Eng. Simulation 28(3) (2006) 35–48]. The advantage of the proposed nonlinear filters in relation to classical autoregressive models is the fact that the nonlinearity of the model is taken into account as it is and no linearizations are made around a nominal position. The variances of the added noises are also estimated. The mathematical convergence of the algorithms IGA and ERA is an open problem in the general case. We propose to this end an interesting conjecture based on ergodic theory. These algorithms are applied to identify the parameters of a transmission channel in data communication. Some simulation results showing the convergence of these algorithms are given.

Probabilistic methods of identifying genes in prokaryotic genomes: connections to the HMM theory.

In this paper, we review developments in probabilistic methods of gene recognition in prokaryotic genomes with the emphasis on connections to the general theory of hidden Markov models (HMM). We show that the Bayesian method implemented in GeneMark, a frequently used gene-finding tool, can be augmented and reintroduced as a rigorous forward-backward (FB) algorithm for local posterior decoding described in the HMM theory. Another earlier developed method, prokaryotic GeneMark.hmm, uses a modification of the Viterbi algorithm for HMM with duration to identify the most likely global path through hidden functional states given the DNA sequence. GeneMark and GeneMark.hmm programs are worth using in concert for analysing prokaryotic DNA sequences that arguably do not follow any exact mathematical model. The new extension of GeneMark using the FB algorithm was implemented in the software program GeneMark.fba. Given the DNA sequence, this program determines an a posteriori probability for each nucleotide to belong to coding or non-coding region. Also, for any open reading frame (ORF), it assigns a score defined as a probabilistic measure of all paths through hidden states that traverse the ORF as a coding region. The prediction accuracy of GeneMark.fba determined in our tests was compared favourably to the accuracy of the initial (standard) GeneMark program. Comparison to the prokaryotic GeneMark.hmm has also demonstrated a certain, yet species-specific, degree of improvement in raw gene detection, ie detection of correct reading frame (and stop codon). The accuracy of exact gene prediction, which is concerned about precise prediction of gene start (which in a prokaryotic genome unambiguously defines the reading frame and stop codon, thus, the whole protein product), still remains more accurate in GeneMarkS, which uses more elaborate HMM to specifically address this task.

A method for portfolio choice

AbstractThis paper shows how one can use the theory of hidden Markov models for portfolio optimization. We illustrate our method by a ball and urn experiment. An application to historical data is examined. Copyright © 2003 John Wiley &amp; Sons, Ltd.

The consensus string problem and the complexity of comparing hidden Markov models

The basic theory of hidden Markov models was developed and applied to problems in speech recognition in the late 1960s, and has since then been applied to numerous problems, e.g. biological sequence analysis. Most applications of hidden Markov models are based on efficient algorithms for computing the probability of generating a given string, or computing the most likely path generating a given string. In this paper we consider the problem of computing the most likely string, or consensus string, generated by a given model, and its implications on the complexity of comparing hidden Markov models. We show that computing the consensus string, and approximating its probability within any constant factor, is NP-hard, and that the same holds for the closely related labeling problem for class hidden Markov models. Furthermore, we establish the NP-hardness of comparing two hidden Markov models under the L ∞- and L 1-norms. We discuss the applicability of the technique used for proving the hardness of comparing two hidden Markov models under the L 1-norm to other measures of distance between probability distributions. In particular, we show that it cannot be used for proving NP-hardness of determining the Kullback–Leibler distance between two hidden Markov models, or of comparing them under the L k -norm for any fixed even integer k.

On the inclusion of channel's time dependence in a hidden Markov model for blind channel estimation

The theory of hidden Markov models (HMM) is applied to the problem of blind (without training sequences) channel estimation and data detection. Within a HMM framework, the Baum-Welch (1970) identification algorithm is frequently used to find out maximum-likelihood (ML) estimates of the corresponding model. However, such a procedure assumes the model (i.e., the channel response) to be static throughout the observation sequence. By means of introducing a parametric model for time-varying channel responses, a version of the algorithm, which is more appropriate for mobile channels [time-dependent Baum-Welch (TDBW)] is derived. Aiming to compare algorithm behavior, a set of computer simulations for a GSM scenario is provided. Results indicate that, in comparison to other Baum-Welch (BW) versions of the algorithm, the TDBW approach attains a remarkable enhancement in performance. For that purpose, only a moderate increase in computational complexity is needed.

A Bayesian model for detecting past recombination events in DNA multiple alignments.

Most phylogenetic tree estimation methods assume that there is a single set of hierarchical relationships among sequences in a data set for all sites along an alignment. Mosaic sequences produced by past recombination events will violate this assumption and may lead to misleading results from a phylogenetic analysis due to the imposition of a single tree along the entire alignment. Therefore, the detection of past recombination is an important first step in an analysis. A Bayesian model for the changes in topology caused by recombination events is described here. This model relaxes the assumption of one topology for all sites in an alignment and uses the theory of Hidden Markov models to facilitate calculations, the hidden states being the underlying topologies at each site in the data set. Changes in topology along the multiple sequence alignment are estimated by means of the maximum a posteriori (MAP) estimate. The performance of the MAP estimate is assessed by application of the model to data sets of four sequences, both simulated and real.

Learning discrete hidden Markov models

This paper describes a learning program for discrete Hidden Markov Models (HMM). The learning of the theory of hidden Markov models is a hard task and many students have been disappointed after the first equations. This program attempts to facilitate the learning of the theory by means of a spread sheet interface for interactive user simulations. The spread sheet of observation probabilities simplify the understanding of several discrete HMM equations. The program allows the student to observe several convergence properties of the training algorithm. © 2000 John Wiley & Sons, Inc. Comput Appl Eng Educ 8: 141–149, 2000