Hidden Markov Model Algorithm Research Articles

A Hybrid Approach for Persian Named Entity Recognition

Named entity recognition (NER) is an information extraction subtask that attempts to recognize and categorize named entities in unstructured text into predefined categories such as the names of people, organizations, and locations. Recently, machine learning approaches, such as hidden Markov model (HMM) as well as hybrid methods, are frequently used to solve Name Entity Recognition. To the best of our knowledge, publicly available data sets for NER in Persian do not exist in any machine learning-based Persian NER system. Because of HMM innate weaknesses, in this paper, we have used both hidden Markov model and rule-based method to recognize named entities in Persian texts. The combination of rule-based method and machine learning method results in a high accurate recognition. The proposed system in its machine learning section uses HMM and Viterbi algorithms, and in its rule-based section employs a set of lexical resources and pattern bases for the recognition of named entities including the names of people, locations and organizations. During this study, we annotate our own training and testing data sets for use in the related phases. Our hybrid approach performs on Persian language with 89.73% precision, 82.44% recall, and 85.93% F-measure using an annotated test corpus including 32,606 tokens.

Efficient and Effective Learning of HMMs Based on Identification of Hidden States

The predominant learning algorithm for Hidden Markov Models (HMMs) is local search heuristics, of which the Baum-Welch (BW) algorithm is mostly used. It is an iterative learning procedure starting with a predefined size of state spaces and randomly chosen initial parameters. However, wrongly chosen initial parameters may cause the risk of falling into a local optimum and a low convergence speed. To overcome these drawbacks, we propose to use a more suitable model initialization approach, a Segmentation-Clustering and Transient analysis (SCT) framework, to estimate the number of states and model parameters directly from the input data. Based on an analysis of the information flow through HMMs, we demystify the structure of models and show that high-impact states are directly identifiable from the properties of observation sequences. States having a high impact on the log-likelihood make HMMs highly specific. Experimental results show that even though the identification accuracy drops to 87.9% when random models are considered, the SCT method is around 50 to 260 times faster than the BW algorithm with 100% correct identification for highly specific models whose specificity is greater than 0.06.

An optimization algorithm for single-molecule fluorescence resonance (smFRET) data processing

The single-molecule fluorescence resonance energy transfer (smFRET) technique plays an important role in the development of biophysics. Measuring the changes of the fluorescence intensities of donor and acceptor and of the FRET efficiency can reveal the changes of distance between the labeling positions. The smFRET may be used to study conformational changes of DNA, proteins and other biomolecules. Traditional algorithm for smFRET data processing is highly dependent on manual operation, leading to high noise, low efficiency and low reliability of the outputs. In the present work, we propose an automatic and more accurate algorithm for smFRET data processing. It consists of three parts: algorithm for automatic pairing of donor and acceptor fluorescence spots based on negative correlation between their intensities; algorithm for data screening by eliminating invalid fluorescence spots sections; algorithm for global data fitting based on Baum-Welch algorithm of hidden Markov model (HMM). Based on the law of energy conservation, the light intensity of one pair of donor and acceptor shows a negative correlation. We can use this feature to find the active smFRET pairs automatically. The algorithm will first find out three active smFRET pairs with correlation coefficient lower than the threshold we set. This three active smFRET pairs will provide enough coordinate data for the algorithm to calculate the pairing matrix in the rest of automatic pairing work. After obtaining all the smFRET pairs, the algorithm for data screening will check the correlation coefficient for each pair. The invalid pairs with correlation coefficient higher than the threshold value will be eliminated. The rest of smFRET pairs will be analyzed by the data fitting algorithm. The Baum-Welch algorithm can be used for learning the global parameters. The global parameters we obtained will then be used to fit each FRET-time curve with Viterbi algorithm. The global parameter learning part will help us find the specific FRET efficiency for each state and the curve fitting part will provide more kinetic parameters. The optimization algorithm significantly simplifies the procedures of manual operation in the traditional algorithm and eliminate several types of noises from the experimental data automatically. We apply the new optimization algorithm to the analyses of folding kinetics data for human telomere repeat sequence, the G-quadruplex DNA. It is demonstrated that the optimization algorithm is more efficient to produce data with higher S/N ratio than the traditional algorithm. The final results reveal clearly the folding of G-quadruplex DNA in multiple states that are influenced by the K+ concentration.

Metagenome and Metatranscriptome Analyses Using Protein Family Profiles.

Analyses of metagenome data (MG) and metatranscriptome data (MT) are often challenged by a paucity of complete reference genome sequences and the uneven/low sequencing depth of the constituent organisms in the microbial community, which respectively limit the power of reference-based alignment and de novo sequence assembly. These limitations make accurate protein family classification and abundance estimation challenging, which in turn hamper downstream analyses such as abundance profiling of metabolic pathways, identification of differentially encoded/expressed genes, and de novo reconstruction of complete gene and protein sequences from the protein family of interest. The profile hidden Markov model (HMM) framework enables the construction of very useful probabilistic models for protein families that allow for accurate modeling of position specific matches, insertions, and deletions. We present a novel homology detection algorithm that integrates banded Viterbi algorithm for profile HMM parsing with an iterative simultaneous alignment and assembly computational framework. The algorithm searches a given profile HMM of a protein family against a database of fragmentary MG/MT sequencing data and simultaneously assembles complete or near-complete gene and protein sequences of the protein family. The resulting program, HMM-GRASPx, demonstrates superior performance in aligning and assembling homologs when benchmarked on both simulated marine MG and real human saliva MG datasets. On real supragingival plaque and stool MG datasets that were generated from healthy individuals, HMM-GRASPx accurately estimates the abundances of the antimicrobial resistance (AMR) gene families and enables accurate characterization of the resistome profiles of these microbial communities. For real human oral microbiome MT datasets, using the HMM-GRASPx estimated transcript abundances significantly improves detection of differentially expressed (DE) genes. Finally, HMM-GRASPx was used to reconstruct comprehensive sets of complete or near-complete protein and nucleotide sequences for the query protein families. HMM-GRASPx is freely available online from http://sourceforge.net/projects/hmm-graspx.

A heuristic optimization algorithm for HMM based on SA and EM in machinery diagnosis

This paper proposes a novel hidden Markov model (HMM) based on simulated annealing (SA) algorithm and expectation maximization (EM) algorithm for machinery diagnosis. As traditional HMM is sensitive to initial values and EM is easy to trap into a local optimization, SA is combined to improve HMM which can overcome local optimization searching problem. The proposed HMM has strong ability of global convergence, and optimizes the process of parameters estimation. Finally, through a case study, the computation results illustrate this SAEM-HMM has high efficiency and accuracy, which could help machinery diagnosis in practical.

Observable Operator Models

This paper describes a new approach to model discrete stochastic processes, called observable operator models (OOMs). The OOMs were introduced by Jaeger as a generalization of hidden Markov models (HMMs). The theory of OOMs makes use of both probabilistic and linear algebraic tools, which has an important advantage: using the tools of linear algebra a very simple and efficient learning algorithm can be developed for OOMs. This seems to be better than the known algorithms for HMMs. This learningalgorithm is presented in detail in the second part of the article.

A Discrete HMM-Based Feature Sequence Model Approach for Star Identification

This paper presents a novel star pattern recognition algorithm based on a discrete hidden Markov model (HMM) for autonomous spacecraft attitude determination in the lost-in-space mode. A two-layer structure is proposed to build an HMM-based star pattern for every guide star. The hidden layer describes the unique geometric distributions of stars in the field of view via the transitions among hidden states, and the observation layer uses discrete and rounded feature sequences consisting of the angular distances of star pairs as outputs. An HMM-based star pattern with maximum probabilities in generating the sequences obtained in the recognition process is uniquely matched. Experiments of synthesized and real star images are tested for the HMM algorithm. Results show that the proposed algorithm has a fast average identification time and is highly robust toward star positions, magnitude noise, and false stars. In the simulation, the HMM algorithm exhibits 99.65% identification rate with 0.4 pixel standard deviations of positional noises, 0.323-Mv magnitude noises, and 1.5-ms average identification time. Moreover, the identification rate is 99.11% and the average identification time is 1.2 ms when testing real star images.

Improving the Consistency of Multitemporal Land Cover Maps Using a Hidden Markov Model

Land cover and land use affect a wide range of regional-scale to global-scale ecosystem processes, and many Earth system models rely on accurate land cover information. However, multitemporal land cover products often show unrealistically high levels of year-to-year label change, particularly at coarse spatial resolution (i.e., 300–500 m). Much of this apparent land cover change arises from errors in classification and does not indicate real change in land cover or land use. In this paper, we present a novel framework that uses a hidden Markov model (HMM) to help distinguish real land cover change from spurious land cover changes in classification time series. We apply the HMM as a postprocessing step to supervised classification, and we solve for the optimal label sequence using existing HMM algorithms. Our results demonstrate that the HMM provides a rigorous framework for capturing temporal context and likelihood of land cover change at each pixel. We evaluated our approach using the MODIS Collection 5.1 Land Cover Type product (MCD12Q1), focusing on areas that have experienced little change over the MODIS time series, and areas that have experienced well characterized change (e.g., deforestation). We show that the HMM method provides label sequences that are more accurate and that exhibit less year-to-year variability than label sequences produced by ensemble-decision-tree classification or by postprocessing heuristics that have been used in recent versions of MCD12Q1 product. The framework that we present offers an improvement over conventional multitemporal land cover classification methods, and it is widely applicable to problems in multitemporal land cover and land use monitoring.

State recognition method for machining process of a large spot welder based on improved genetic algorithm and hidden Markov model

Large spot welder is an important equipment in rail transit equipment manufacturing industry, but having the problem of low utilization rate and low effectlvely machining rate. State monitoring can master its operating states real time and comprehensively, and providing data support for state recognition. Hidden Markov model is a state classification method, but it is sensitive to the initial model parameters and easy to trap into a local optima. Genetic algorithm is a global searching method; however, it is quite poor at hill climbing and also has the problem of premature convergence. In this paper, proposing the improved genetic algorithm, and combining improved genetic algorithm and hidden Markov model, a new method of state recognition method named improved genetic algorithm–hidden Markov model is proposed. In the proposed method, improved genetic algorithm is used for optimizing the initial parameters, and hidden Markov model as a classifier to recognize the operating states for machining process. This method is also compared with the other two recognition methods named adaptive genetic algorithm–hidden Markov model and hidden Markov model, in which adaptive genetic algorithm is similarly used for optimizing the initial parameters, however hidden Markov model (in both methods) as a classifier. Experimental results show that the proposed method is very effective, and the improved genetic algorithm–hidden Markov model recognition method is superior to the adaptive genetic algorithm–hidden Markov model and hidden Markov model recognition method.

DPHK: real-time distributed predicted data collecting based on activity pattern knowledge mined from trajectories in smart environments

In this paper, we have proposed and designed DPHK (data prediction based on HMM according to activity pattern knowledge mined from trajectories), a real-time distributed predicted data collection system to solve the congestion and data loss caused by too many connections to sink node in indoor smart environment scenarios (like Smart Home, Smart Wireless Healthcare and so on). DPHK predicts and sends predicted data at one time instead of sending the triggered data of these sensor nodes which people is going to pass in several times. Firstly, our system learns the knowledge of transition probability among sensor nodes from the historical binary motion data through data mining. Secondly, it stores the corresponding knowledge in each sensor node based on a special storage mechanism. Thirdly, each sensor node applies HMM (hidden Markov model) algorithm to predict the sensor node locations people will arrive at according to the receivedmessage. At last, these sensor nodes send their triggered data and the predicted data to the sink node. The significances of DPHK are as follows: (a) the procedure of DPHK is distributed; (b) it effectively reduces the connection between sensor nodes and sink node. The time complexities of the proposed algorithms are analyzed and the performance is evaluated by some designed experiments in a smart environment.

Algebraic Dynamic Programming over general data structures.

BackgroundDynamic programming algorithms provide exact solutions to many problems in computational biology, such as sequence alignment, RNA folding, hidden Markov models (HMMs), and scoring of phylogenetic trees. Structurally analogous algorithms compute optimal solutions, evaluate score distributions, and perform stochastic sampling. This is explained in the theory of Algebraic Dynamic Programming (ADP) by a strict separation of state space traversal (usually represented by a context free grammar), scoring (encoded as an algebra), and choice rule. A key ingredient in this theory is the use of yield parsers that operate on the ordered input data structure, usually strings or ordered trees. The computation of ensemble properties, such as a posteriori probabilities of HMMs or partition functions in RNA folding, requires the combination of two distinct, but intimately related algorithms, known as the inside and the outside recursion. Only the inside recursions are covered by the classical ADP theory.ResultsThe ideas of ADP are generalized to a much wider scope of data structures by relaxing the concept of parsing. This allows us to formalize the conceptual complementarity of inside and outside variables in a natural way. We demonstrate that outside recursions are generically derivable from inside decomposition schemes. In addition to rephrasing the well-known algorithms for HMMs, pairwise sequence alignment, and RNA folding we show how the TSP and the shortest Hamiltonian path problem can be implemented efficiently in the extended ADP framework. As a showcase application we investigate the ancient evolution of HOX gene clusters in terms of shortest Hamiltonian paths.ConclusionsThe generalized ADP framework presented here greatly facilitates the development and implementation of dynamic programming algorithms for a wide spectrum of applications.

An adaptive design method for understanding tolerance in the precision stamping process

With the development of precision manufacturing, the understanding of tolerance has become a research hotspot in the field of manufacturing. An adaptable design method for understanding tolerance in the precision stamping process is proposed in this study. First, fluctuations of tolerance which are caused by differences in the stamping process are analyzed, such as differences in material and thickness, which can lead to changes in the metal flow stress curve. Second, a condition-driven adaptive design method is constructed based on a monitoring system and hydraulic control system. The mapping rules between multiple disturbance factors and the execution strategy are established by the hidden Markov model algorithm. Third, executive parameters, such as velocity, pressure, and gaps, are calculated and optimized by the data statistics of partial tolerance fluctuations in the control module. Then disturbances of various conditions could be adaptively controlled timely and effectively by the executive parameters. Finally, the adaptive design method for tolerance of one precision stamping part is applied, and the effect of the application is proved by the optimized results.

Feature Selection Using Fisher's Ratio Technique for Automatic Speech Recognition

Automatic Speech Recognition involves mainly two steps; feature extraction and classification . Mel Frequency Cepstral Coefficient is used as one of the prominent feature extraction techniques in ASR. Usually, the set of all 12 MFCC coefficients is used as the feature vector in the classification step. But the question is whether the same or improved classification accuracy can be achieved by using a subset of 12 MFCC as feature vector. In this paper, Fisher's ratio technique is used for selecting a subset of 12 MFCC coefficients that contribute more in discriminating a pattern. The selected coefficients are used in classification with Hidden Markov Model algorithm. The classification accuracies that we get by using 12 coefficients and by using the selected coefficients are compared.

Quantifying and Protecting Location Privacy

Quantifying and Protecting Location Privacy

Optimizing the Forward Algorithm for Hidden Markov Model on IBM Roadrunner clusters

In this paper we present a parallel solution of the Forward Algorithm for Hidden Markov Models. The Forward algorithm compute a probability of a hidden state from Markov model at a cer ...

Hybrid Metaheuristic Approaches to the Expectation Maximization for Estimation of the Hidden Markov Model for Signal Modeling

The expectation maximization (EM) is the standard training algorithm for hidden Markov model (HMM). However, EM faces a local convergence problem in HMM estimation. This paper attempts to overcome this problem of EM and proposes hybrid metaheuristic approaches to EM for HMM. In our earlier research, a hybrid of a constraint-based evolutionary learning approach to EM (CEL-EM) improved HMM estimation. In this paper, we propose a hybrid simulated annealing stochastic version of EM (SASEM) that combines simulated annealing (SA) with EM. The novelty of our approach is that we develop a mathematical reformulation of HMM estimation by introducing a stochastic step between the EM steps and combine SA with EM to provide better control over the acceptance of stochastic and EM steps for better HMM estimation. We also extend our earlier work and propose a second hybrid which is a combination of an EA and the proposed SASEM, (EA-SASEM). The proposed EA-SASEM uses the best constraint-based EA strategies from CEL-EM and stochastic reformulation of HMM. The complementary properties of EA and SA and stochastic reformulation of HMM of SASEM provide EA-SASEM with sufficient potential to find better estimation for HMM. To the best of our knowledge, this type of hybridization and mathematical reformulation have not been explored in the context of EM and HMM training. The proposed approaches have been evaluated through comprehensive experiments to justify their effectiveness in signal modeling using the speech corpus: TIMIT. Experimental results show that proposed approaches obtain higher recognition accuracies than the EM algorithm and CEL-EM as well.

A Syntax Parsing Method Based on Adaptive Genetic Annealing Optimization HMM

In order to further enhance the performance of syntax parsing, for the shortcomings of hidden Markov model (HMM) in the parameter optimization, an improved syntax parsing method based on adaptive genetic annealing and HMM was presented. First, an adaptive hybrid genetic annealing algorithm was adopted to optimize HMM initial parameters. Second, the improved HMM was trained by Baum Welch algorithm, and then a modified Viterbi algorithm was used to recognize various types of phrases at the same layer, finally a hierarchical analysis algorithm and Viterbi algorithm were combined together to solve hierarchy and recursion in the sentence. In the adaptive genetic annealing HMM algorithm, genetic operators and parameters of simulated annealing (SA) were first respectively improved, subpopulations were classified according to the adaptive crossover and mutation probability of GA in order to realize the multi-group parallel search and information exchange, which could avoid premature and accelerate convergence, then SA was taken as a GA operator to strengthen the local search capability. Compared with several new approaches, Fβ = 1 value is averagely increased by 3%. The experiment results prove that this method is very effective for syntactic parsing.

A View-Invariant Action Recognition Based on Multi-View Space Hidden Markov Models

Visual-based action recognition has already been widely used in human–machine interfaces. However, it is a challenging research to recognize the human actions from different viewpoints. In order to solve this issue, a novel multi-view space hidden Markov models (HMMs) algorithm for view-invariant action recognition is proposed. First, a view-insensitive feature representation by combining the bag-of-words of interest point with the amplitude histogram of optical flow is utilized for describing the human action sequences. The combined features could not only solve the problem that there was no possibility in establishing an association between traditional bag-of-words of interest point method and HMMs, but also greatly reduce the redundancy in the video. Second, the view space is partitioned into multiple sub-view space according to the camera rotation viewpoint. Human action models are trained by HMMs algorithm in each sub-view space. By computing the probabilities of the test sequence (i.e., observation sequence) for the given multi-view space HMMs, the similarity between the sub-view space and the test sequence viewpoint are analyzed during the recognition process. Finally, the action with unknown viewpoint is recognized via the probability weighted combination. The experimental results on multi-view action dataset IXMAS demonstrate that the proposed approach is highly efficient and effective in view-invariant action recognition.

Wavelet analysis in ecology and epidemiology: impact of statistical tests

Wavelet analysis is now frequently used to extract information from ecological and epidemiological time series. Statistical hypothesis tests are conducted on associated wavelet quantities to assess the likelihood that they are due to a random process. Such random processes represent null models and are generally based on synthetic data that share some statistical characteristics with the original time series. This allows the comparison of null statistics with those obtained from original time series. When creating synthetic datasets, different techniques of resampling result in different characteristics shared by the synthetic time series. Therefore, it becomes crucial to consider the impact of the resampling method on the results. We have addressed this point by comparing seven different statistical testing methods applied with different real and simulated data. Our results show that statistical assessment of periodic patterns is strongly affected by the choice of the resampling method, so two different resampling techniques could lead to two different conclusions about the same time series. Moreover, our results clearly show the inadequacy of resampling series generated by white noise and red noise that are nevertheless the methods currently used in the wide majority of wavelets applications. Our results highlight that the characteristics of a time series, namely its Fourier spectrum and autocorrelation, are important to consider when choosing the resampling technique. Results suggest that data-driven resampling methods should be used such as the hidden Markov model algorithm and the 'beta-surrogate' method.

Bisulfighter: accurate detection of methylated cytosines and differentially methylated regions

Analysis of bisulfite sequencing data usually requires two tasks: to call methylated cytosines (mCs) in a sample, and to detect differentially methylated regions (DMRs) between paired samples. Although numerous tools have been proposed for mC calling, methods for DMR detection have been largely limited. Here, we present Bisulfighter, a new software package for detecting mCs and DMRs from bisulfite sequencing data. Bisulfighter combines the LAST alignment tool for mC calling, and a novel framework for DMR detection based on hidden Markov models (HMMs). Unlike previous attempts that depend on empirical parameters, Bisulfighter can use the expectation-maximization algorithm for HMMs to adjust parameters for each data set. We conduct extensive experiments in which accuracy of mC calling and DMR detection is evaluated on simulated data with various mC contexts, read qualities, sequencing depths and DMR lengths, as well as on real data from a wide range of biological processes. We demonstrate that Bisulfighter consistently achieves better accuracy than other published tools, providing greater sensitivity for mCs with fewer false positives, more precise estimates of mC levels, more exact locations of DMRs and better agreement of DMRs with gene expression and DNase I hypersensitivity. The source code is available at http://epigenome.cbrc.jp/bisulfighter.