Hidden Markov Model Structure Research Articles

A Latent Hidden Markov Model for Process Data.

Response process data from computer-based problem-solving items describe respondents' problem-solving processes as sequences of actions. Such data provide a valuable source for understanding respondents' problem-solving behaviors. Recently, data-driven feature extraction methods have been developed to compress the information in unstructured process data into relatively low-dimensional features. Although the extracted features can be used as covariates in regression or other models to understand respondents' response behaviors, the results are often not easy to interpret since the relationship between the extracted features, and the original response process is often not explicitly defined. In this paper, we propose a statistical model for describing response processes and how they vary across respondents. The proposed model assumes a response process follows a hidden Markov model given the respondent's latent traits. The structure of hidden Markov models resembles problem-solving processes, with the hidden states interpreted as problem-solving subtasks or stages. Incorporating the latent traits in hidden Markov models enables us to characterize the heterogeneity of response processes across respondents in a parsimonious and interpretable way. We demonstrate the performance of the proposed model through simulation experiments and case studies of PISA process data.

China's GDP forecasting using Long Short Term Memory Recurrent Neural Network and Hidden Markov Model.

This paper presents a Long Short Term Memory Recurrent Neural Network and Hidden Markov Model (LSTM-HMM) to predict China's Gross Domestic Product (GDP) fluctuation state within a rolling time window. We compare the predictive power of LSTM-HMM with other dynamic forecast systems within different time windows, which involves the Hidden Markov Model (HMM), Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) and LSTM-HMM with an input of monthly Consumer Price Index (CPI) or quarterly CPI within 4-year, 6-year, 8-year and 10-year time window. These forecasting models employed in our empirical analysis share the basic HMM structure but differ in the generation of observable CPI fluctuation states. Our forecasting results suggest that (1) among all the models, LSTM-HMM generally performs better than the other models; (2) the model performance can be improved when model input transforms from quarterly to monthly; (3) among all the time windows, models within 10-year time window have better overall performance; (4) within 10-year time window, the LSTM-HMM, with either quarterly or monthly input, has the best accuracy and consistency.

Towards Constructing HMM Structure for Speech Recognition With Deep Neural Fenonic Baseform Growing

For decades, acoustic models in speech recognition systems pivot on Hidden Markov Models (HMMs), e.g., Gaussian Mixture Model-HMM system, Deep Neural Network-HMM system, etc., and achieve remarkable results. However, the popular HMM model is the three-state left-to-right structure, without the superiority certainty. There are multiple studies on the HMM structure’s optimization, but none of them addresses this problem leveraging deep learning algorithms. For the first time, this paper proposes a new training method based on Deep Neural Fenonic Baseform Growing to optimize the HMM structure, which is concisely designed and computationally cheap. Moreover, this data-driven method customizes the HMM structure for each phone precisely without external assumptions concerning the number of states or transition patterns. Experimental results on both TIMIT and TEDliumv2 corpora indicate that the proposed HMM structure improves both the monophone system and the triphone system substantially. Besides, its adoption further improves state-of-the-art speech recognition systems with remarkably reduced parameters.

Reward Learning From Very Few Demonstrations

This article introduces a novel skill learning framework that learns rewards from very few demonstrations and uses them in policy search (PS) to improve the skill. The demonstrations are used to learn a parameterized policy to execute the skill and a goal model, as a hidden Markov model (HMM), to monitor executions. The rewards are learned from the HMM structure and its monitoring capability. The HMM is converted to a finite-horizon Markov reward process (MRP). A Monte Carlo approach is used to calculate its values. Then, the HMM and the values are merged into a partially observable MRP to obtain execution returns to be used with PS for improving the policy. In addition to reward learning, a black box PS method with an adaptive exploration strategy is adopted. The resulting framework is evaluated with five PS approaches and two skills in simulation. The results show that the learned dense rewards lead to better performance compared to sparse monitoring signals, and using an adaptive exploration lead to faster convergence with higher success rates and lower variance. The efficacy of the framework is validated in a real-robot settings by improving three skills to complete success from complete failure using learned rewards where sparse rewards failed completely.

A hidden semi-Markov model for indoor radio source localization using received signal strength

Multipath propagation makes the use of received signal strength (RSS) unreliable as a signal propagation model for localization of a radio source based on RSS data. An approach to mitigation of this problem is the use of a Hidden Markov model (HMM) to represent the relationship between RSS and the radio source location by incorporating an environment prior and RSS source dynamics. The HMM structure forces a geometric form for the distribution for the sojourn time. This, combined with missing data problems, reduces confidence in location estimation. It is found, in this paper, that Hidden semi-Markov Models (HsMMs), with a more flexible sojourn time distribution are more able to represent source dynamics while retaining the advantages of HMMs for environmental constraints and improving resilience to missing measurements. The performance of the proposed HsMM algorithm is compared with a standard HMM via experiments and simulations involving indoor radio source localization using RSS data. Simulations use a ray tracing software-based simulator and the experiments for transmitter localization with RSS data are collected by software defined radios.

Lower Bound of variance for Multipartite Structure of Hidden Markov Model

Lower Bound of variance for Multipartite Structure of Hidden Markov Model

Lower Bound of Variance for Multipartite Structure of Hidden Markov Model

Objective: This study derives an expression for obtaining lower bound of variance for estimates of hidden Markov model. Also, this study provides a parametric procedure for lower bound of variance. Methods/Statistical Analysis: In study a multipartite form of hidden Markov model considered. The states of model observed at time “2t-1” and “2t” respectively. The lower bound expression obtained by Louis methodology. The secondary data is used to study the validity of proposed lower bound. This dataset is a time series and named as “tree ring width”. Study also defines a parametric procedure for computing lower bound of variance. Findings: The study obtained the lower bound of variance for the maximum likelihood estimates of model using real-world data. The study compares the results of variance obtained by two procedures for various combinations of the states. It is found that for some combination of states, the lower bound of variance by two approaches found almost same. While for many combinations of states the lower bound of variance from proposed procedure found smaller than parametric procedure. The overall comparison lower bound of variance for maximum likelihood by proposed method explaining less variation in dataset then conventional i.e. parametric procedure. Conclusion: The study concluded that results for variance by derived expression found sharpen than those of parametric procedure. Keywords: Hidden Markov Model, Lower Bound Variance, Maximum Likelihood, Multipartite Structure, Parametric Procedure

Improved HMM by Deep Learning for Ear Classification

Ear recognition is one of the most relevant applications of image analysis. It’s a true challenge to build an automated system which exceeds human ability to recognize ears. Humans do not identify the ears ordinarily, so we are not skilled when we must deal with a large number of unknown ears. The modern computers, with an almost limitless memory and computational speed, should overcome humans' limitations. This work uses Ear classification problem to improve Deviance Information Criterion- Structural Hidden Markov Model (DIC-SHMM) by Convolutional Neural Network (CNN). HMM is a strong model for large size of features. While the CNN, as a most important technology for deep learning, used for image classification, to recognize persons by their ears as a one of unique biometric physiological characteristics. Three systems will be used to classify ear images, deep learning for the original image directly, deep learning for eigenvector as Principle Components Analysis (PCA) of the original image to compare them with proposed combining convolution layers of CNN with improved HMM for the original Image. The proposed system shows the best correction rate as 97.5%.

Bayesian hidden Markov models in DNA sequence segmentation using R: the case of Simian Vacuolating virus (SV40)

ABSTRACTSegmentation models aim to partition compositionally heterogeneous domains into homogeneous segments which may be reflective of biological function. Due to the latent nature of the segments a natural approach to segmentation that has gained favour recently uses Bayesian hidden Markov models (HMMs). Concomitantly in the last few decades, the free R programming language has become a dominant tool for computational statistics, visualization and data science. Therefore, this paper aims to fully exploit R to fit a Bayesian HMM for DNA segmentation. The joint posterior distribution of parameters in the model to be considered is derived followed by the algorithms that can be used for estimation. Functions following these algorithms (Gibbs Sampling, Data Augmentation and Label Switching) are then fully implemented in R. The methodology is assessed through extensive simulation studies and then being applied to analyse Simian Vacuolating virus (SV40). It is concluded that: (1) the algorithms and functions in R can correctly estimate sequence segmentation if the HMM structure is assumed; (2) the performance of the model improves with sequence length; (3) R is reasonably fast for short to medium sequence lengths and number of segments and (4) the segmentation of SV40 appears to correspond with the two major transcripts, early and late, that regulate the expression of SV40 genes.

Using hidden Markov model to identify oscillation temporal pattern for control loops

Oscillations in the control loops can result in high variability of product quality, more energy consumption, and accelerated equipment aging. The existing oscillation detection methods use only part information of signal, for example, the regularity of zero-crossings, but do not take the temporal sequence into account. This research studies the introduction of a priori structure to extract the repeated pattern from a time series based on the hidden Markov model (HMM), particularly in recognition of the oscillation behavior. To get the representative hidden state sequence, a strategy of merging hidden states to find a good HMM structure for a given time sequence is developed. Then a new framework for analyzing the hidden state series data called oscillation detection algorithm (ODA) is presented. This ODA adapts innovative data mining concepts to analyze the state series data. In particular, it can reveal hidden temporal patterns. Case studies on a numeric example and a refinery process are performed to illustrate the effectiveness of the proposed method.

Evolutionary structure of hidden Markov models for audio-visual Arabic speech recognition

In this paper, we present an Audio-Visual Automatic Speech Recognition System that combines the acoustic and the visual data. The proposed algorithm here, for modelling the multimodal data, is a Hidden Markov Model (HMM) hybridised with the Genetic Algorithm (GA) to determine its optimal structure. This algorithm is combined with the Baum-Welch algorithm, which allows an effective re-estimation of the probabilities of the HMM. Our experiments show the improvement in the performance of the most promising audio-visual system, based on the combination of GA/HMM model compared to the traditional HMM.

Recognition System for Home-Service-Related Sign Language Using Entropy-Based <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula>-Means Algorithm and ABC-Based HMM

This paper presents a recognition system for understanding the words of home-service-related sign language. Because the data received from a sensor are sequential, the hidden Markov model (HMM) that has been successfully applied to speech signals is chosen as a classifier. However, the number of states in the HMM model should be decided upon first before constructing the HMM classifier. To solve this problem, an entropy-based $K$ -means algorithm is proposed to evaluate the number of states in the HMM model with an entropy diagram. Four real datasets are utilized to verify the developed entropy-based $K$ -means algorithm. Moreover, a data-driven method is given to combine the artificial bee colony algorithm with the Baum–Welch algorithm to determine the structure of HMM. The database contains 11 home-service-related Taiwan sign language words and each word is performed ten times, five males and five females are invited to perform such words. Finally, the recognition system is established by 11 HMM models, and the cross-validation demonstrates an average recognition rate of 91.3%.

Particle Swarm Optimization For Hidden Markov Model

Hidden Markov Model (HMM) is a very well known method as a statistical model used for intelligent systems applications. Due to its involvement in various applications, it would be very important to have a good representation of HMM for the given problem to achieve good results. In this paper, we propose a theoretical approach that can be followed to obtain the best structure of HMM based on Particle Swarm Optimization (PSO) concepts. Given a set of comprehensive visible and invisible states, we propose a method based on PSO concepts to evolve an optimum HMM structure design. The proposed approach deals with two factors related to HMM, generating new states and updating probability values. The main steps followed in the proposed approach involve three main phases, the first phase is generating randomly a population of HMMs, the second phase is converting the generated HMM to PSO required format and the third phase is the application of PSO to find out the optimum HMM . The importance of the proposed approach over other previous approaches is that other approaches deal only with probability updating.

Online decoding of hidden Markov models for gait event detection using foot-mounted gyroscopes.

In this paper, we present an approach to the online implementation of a gait event detector based on machine learning algorithms. Gait events were detected using a uniaxial gyro that measured the foot instep angular velocity in the sagittal plane to feed a four-state left-right hidden Markov model (HMM). The short-time Viterbi algorithm was used to overcome the limitation of the standard Viterbi algorithm, which does not allow the online decoding of hidden state sequences. Supervised learning of the HMM structure and validation with the leave-one-subject-out validation method were performed using treadmill gait reference data from an optical motion capture system. The four gait events were foot strike, flat foot (FF), heel off (HO), and toe off. The accuracy ranged, on average, from 45 ms (early detection, FF) to 35 ms (late detection, HO); the latency of detection was less than 100 ms for all gait events but the HO, where the probability that it was greater than 100 ms was 25%. Overground walking tests of the HMM-based gait event detector were also successfully performed.

Hidden Markov models for the activity profile of terrorist groups

The main focus of this work is on developing models for the activity profile of a terrorist group, detecting sudden spurts and downfalls in this profile, and, in general, tracking it over a period of time. Toward this goal, a $d$-state hidden Markov model (HMM) that captures the latent states underlying the dynamics of the group and thus its activity profile is developed. The simplest setting of $d=2$ corresponds to the case where the dynamics are coarsely quantized as Active and Inactive, respectively. A state estimation strategy that exploits the underlying HMM structure is then developed for spurt detection and tracking. This strategy is shown to track even nonpersistent changes that last only for a short duration at the cost of learning the underlying model. Case studies with real terrorism data from open-source databases are provided to illustrate the performance of the proposed methodology.

Increasing mapping based hidden Markov model for dynamic process monitoring and diagnosis

Hidden Markov models (HMMs) perform parameter estimation based on the forward–backward (FB) procedure and the Baum–Welch (BW) algorithm. The two algorithms together may increase the computational complexity and the difficulty to understand the algorithm structure of HMMs clearly. In this study, an increasing mapping based hidden Markov model (IMHMM) is proposed. Between the observation sequence and possible state sequence an increasing mapping is established. The re-estimation formulas for the model parameters are derived straightforwardly based on these mappings instead of FB variables. The IMHMM has simpler algorithm structure and lower storage requirement than the HMM. Based on IMHMM, an expandable process monitoring and fault diagnosis framework for large-scale dynamical process is developed. To characterize the dynamic process, a novel index considering serial correlation is used to evaluate process state. The presented methodology is carried out in Tennessee Eastman process (TEP). The results show improvement over HMM in terms of memory complexity and training time of the model. Also, the power of IMHMM can be observed compared with principal component analysis (PCA) based methods.

A detection of danger sounds based on variable-state hidden Markov models

To detect hazardous situations with danger sounds, the acoustic surveillance system is an ideal candidate. The conventional systems recognize environmental sounds with hidden Markov model (HMM) in order to detect danger sounds. It is however difficult to accurately recognize the environmental sounds, because the optimum HMM parameters for environmental sounds have not been identified. It is important factor for accurately recognizing them to ideally determine the number of states, one of the HMM parameter. On the other hand, environmental sounds which include danger sounds have a wider characteristic as the structure, the complexity, the length, etc. The variable states should be therefore an optimum HMM structure to detect the danger sounds. We thus propose the danger sound detection based on variable-state HMMs corresponding to a number of inflection points with the delta power of environmental sounds. We first investigate the recognition performance of environmental sounds including danger sounds with ...

Power Quality Disturbance Classification Using S-transform and Hidden Markov Model

In this article, a combination method based on S-transform and the hidden Markov model is presented for power quality disturbance classification. S-transform is a new method that has been used in signal processing and power quality disturbance classification applications. S-transform not only has the advantages of both wavelet transform and fast Fourier transform but also characteristics superior than both mentioned transform methods. Besides, the hidden Markov model is a powerful and effective method on signal processing applications. This method computes the maximum likelihood probability between training and testing data signals for identification. The proposed method makes the classification more simple and accurate. The identification procedure has two major steps. First, features are extracted by S-transform, and in the second step, classification is done using a decision tree structure of hidden Markov models. Simulation results by simulated and experimental test signals reveal the robustness of proposed method and the possibility of using it for real applications.

Ozone Prediction Performance of HMM Based on the Data Compressed by Different Wavelet Basis Functions

Surface ozone in the air boundary layer is one of the most harmful air pollutants produced by photochemical reaction between nitrogen oxides and volatile hydrocarbons, which causes great damage to human beings and environment. The prediction of surface ozone levels plays an important role in the control and the reduction of air pollutants. As model-driven statistical prediction models, hidden Markov Models (HMMs) are rich in mathematical structure and work well in many important applications. Due to the complex structure of HMM, long observation sequences would increase computational load by geometric ratio. In order to reduce training time, wavelet decomposition is used to compress the original observations into shorter ones. During compression step, observation sequences compressed by different wavelet basis functions keep different information content. This may have impact on prediction results. In this paper, ozone prediction performance of HMM based on different wavelet basis functions are discussed. Shannon entropy is employed to measure how much information content is kept in the new sequence compared to the original one. Data from Houston Metropolitan Area, TX are used in this paper. Results show that wavelet basis functions used in data compression step can affect the HMM model performance significantly. The new sequence with the maximum Shannon entropy generates the best prediction result.

Evolving Fisher Kernels for Biological Sequence Classification

Fisher kernels have been successfully applied to many problems in bioinformatics. However, their success depends on the quality of the generative model upon which they are built. For Fisher kernel techniques to be used on novel problems, a mechanism for creating accurate generative models is required. A novel framework is presented for automatically creating domain-specific generative models that can be used to produce Fisher kernels for support vector machines (SVMs) and other kernel methods. The framework enables the capture of prior knowledge and addresses the issue of domain-specific kernels, both of which are current areas that are lacking in many kernel-based methods. To obtain the generative model, genetic algorithms are used to evolve the structure of hidden Markov models (HMMs). A Fisher kernel is subsequently created from the HMM, and used in conjunction with an SVM, to improve the discriminative power. This paper investigates the effectiveness of the proposed method, named GA-SVM. We show that its performance is comparable if not better than other state of the art methods in classifying secretory protein sequences of malaria. More interestingly, it showed better results than the sequence-similarity-based approach, without the need for additional homologous sequence information in protein enzyme family classification. The experiments clearly demonstrate that the GA-SVM is a novel way to find features with good performance from biological sequences, that does not require extensive tuning of a complex model.