Generalized Hidden Markov Model Research Articles

Auxiliary two-filter particle smoothing for one generalized hidden Markov model

This paper develops two-filter particle smoothing (TFPS) algorithms for the nonlinear fixed-interval smoothing problem of one generalized hidden Markov model (GHMM), where the current observation depends not only on the current state, but also on one-step previous state. Firstly, by Bayesian approach, the two-filter smoothing (TFS) formula for GHMM is established to calculate smoothing densities. In this TFS formula, the backward information prediction density is generally not a density of the state. This results in a difficulty that the normal sequential Monte Carlo (SMC) sampling technique cannot be directly applied to design corresponding TFPS algorithms based on the TFS formula. To solve this difficulty, a generalized TFS formula for GHMM is then proposed by introducing a sequence of artificial densities. By combining this generalized TFS formula, SMC, and the auxiliary variable sampling technique, a basic auxiliary TFPS (ATFPS) algorithm with quadratic computational complexity is proposed, and a simplified ATFPS algorithm with linear computational complexity is further developed. Finally, the effectiveness and superiority of the two proposed ATFPS algorithms for GHMM are verified via simulation examples and real experimental data.

A generalized hidden Markov model and its applications in recognition of cutting states

In the traditional hidden Markov model (HMM) for statistical learning and classification, aleatory uncertainty because of randomness and epistemic uncertainty that is due to the lack of knowledge are not differentiated. In this paper, a generalized hidden Markov model (GHMM) in the context of generalized interval probability theory is proposed. The parameters of GHMM are in the form of generalized interval probability, which provides a concise representation for the two kinds of uncertainty simultaneously. The generalized versions of the forward-backward, Viterbi, and Baum-Welch algorithms in GHMM are developed. The proposed algorithms take advantage of the algebraic property in the generalized interval probability. These algorithms provide an efficient approach to train the GHMM. The trained GHMM is used for state recognition with the criterion of maximum log-likelihood. A case study on recognizing the cutting states in manufacturing processes is provided to demonstrate the application of GHMM. The interval forms of the cutting signal are considered as the inputting of GHMM. The cutting states are recognized based on the learning algorithm of GHMM. With the two uncertainty components quantified, the reliability of GHMM in state recognition is superior to that of HMM.

Prediction of plant mRNA polyadenylation sites.

Messenger RNA polyadenylation is one of the essential processing steps during eukaryotic gene expression. The site of polyadenylation [poly(A) site] marks the end of a transcript, which is also the end of a gene in most cases. A computation program that is able to recognize poly(A) sites would not only be useful for genome annotation in finding genes ends, but also for predicting alternative poly(A) sites. PASS [Poly(A) Site Sleuth] and PAC [Poly(A) site Classifier] were developed to predict poly(A) sites in plants. PASS was built based on the Generalized Hidden Markov Model (GHMM), which consists of four functional modules: input model, poly(A) site recognition module, graphic process module, and output module. PAC is a classification model, integrating several features that define the poly(A) sites including K-gram pattern, Z-curve, position-specific scoring matrix, and first-order inhomogeneous Markov sub-model. PAC can be used to predict poly(A) sites from species whose polyadenylation profile is unknown. The result of PASS and PAC is an output of a few files with one of them containing the score or probability of being a poly(A) site for each position of a given sequence. While the models were built mostly based on poly(A) profile data from Arabidopsis, it is also functional in other higher plants since their profiles are quite similar.

A generalized interval probability-based optimization method for training generalized hidden Markov model

Recently a generalized hidden Markov model (GHMM) was proposed for solving the information fusion problems under aleatory and epistemic uncertainties in engineering application. In GHMM, aleatory uncertainty is captured by the probability measure whereas epistemic uncertainty is modeled by generalized interval. In this paper, the problem of how to train the GHMM with a small amount of observation data is studied. An optimization method as a generalization of the Baum–Welch algorithm is proposed. With a generalized Baum–Welch′s auxiliary function and the Jensen inequality based on generalized interval, the GHMM parameters are estimated and updated by the lower and upper bounds of observation sequences. A set of training and re-estimation formulas are developed. With a multiple observation expectation maximization (EM) algorithm, the training method guarantees the local maxima of the lower and the upper bounds. Two case studies of recognizing the tool wear and cutting states in manufacturing is described to demonstrate the proposed method. The results show that the optimized GHMM has a good recognition performance.

Multiscale Uncertainty Quantification Based on a Generalized Hidden Markov Model

Variability is the inherent randomness in systems, whereas incertitude is due to lack of knowledge. In this paper, a generalized hidden Markov model (GHMM) is proposed to quantify aleatory and epistemic uncertainties simultaneously in multiscale system analysis. The GHMM is based on a new imprecise probability theory that has the form of generalized interval. The new interval probability resembles the precise probability and has a similar calculus structure. The proposed GHMM allows us to quantify cross-scale dependency and information loss between scales. Based on a generalized interval Bayes’ rule, three cross-scale information assimilation approaches that incorporate uncertainty propagation are also developed.

MGEScan-non-LTR: computational identification and classification of autonomous non-LTR retrotransposons in eukaryotic genomes

Computational methods for genome-wide identification of mobile genetic elements (MGEs) have become increasingly necessary for both genome annotation and evolutionary studies. Non-long terminal repeat (non-LTR) retrotransposons are a class of MGEs that have been found in most eukaryotic genomes, sometimes in extremely high numbers. In this article, we present a computational tool, MGEScan-non-LTR, for the identification of non-LTR retrotransposons in genomic sequences, following a computational approach inspired by a generalized hidden Markov model (GHMM). Three different states represent two different protein domains and inter-domain linker regions encoded in the non-LTR retrotransposons, and their scores are evaluated by using profile hidden Markov models (for protein domains) and Gaussian Bayes classifiers (for linker regions), respectively. In order to classify the non-LTR retrotransposons into one of the 12 previously characterized clades using the same model, we defined separate states for different clades. MGEScan-non-LTR was tested on the genome sequences of four eukaryotic organisms, Drosophila melanogaster, Daphnia pulex, Ciona intestinalis and Strongylocentrotus purpuratus. For the D. melanogaster genome, MGEScan-non-LTR found all known ‘full-length’ elements and simultaneously classified them into the clades CR1, I, Jockey, LOA and R1. Notably, for the D. pulex genome, in which no non-LTR retrotransposon has been annotated, MGEScan-non-LTR found a significantly larger number of elements than did RepeatMasker, using the current version of the RepBase Update library. We also identified novel elements in the other two genomes, which have only been partially studied for non-LTR retrotransposons.

A Generalized Hidden Markov Model Approach to Transmembrane Region Prediction with Poisson Distribution as State Duration Probabilities

We present a novel algorithm to predict transmembrane regions from a primary amino acid sequence. Previous studies have shown that the Hidden Markov Model (HMM) is one of the powerful tools known to predict transmembrane regions; however, one of the conceptual drawbacks of the standard HMM is the fact that the state duration, i.e., the duration for which the hidden dynamics remains in a particular state follows the geometric distribution. Real data, however, does not always indicate such a geometric distribution. The proposed algorithm utilizes a Generalized Hidden Markov Model (GHMM), an extension of the HMM, to cope with this problem. In the GHMM, the state duration probability can be any discrete distribution, including a geometric distribution. The proposed algorithm employs a state duration probability based on a Poisson distribution. We consider the two-dimensional vector trajectory consisting of hydropathy index and charge associated with amino acids, instead of the 20 letter symbol sequences. Also a Monte Carlo method (Forward/Backward Sampling method) is adopted for the transmembrane region prediction step. Prediction accuracies using publicly available data sets show that the proposed algorithm yields reasonably good results when compared against some existing algorithms.

Gene prediction in eukaryotes with a generalized hidden Markov model that uses hints from external sources.

BackgroundIn order to improve gene prediction, extrinsic evidence on the gene structure can be collected from various sources of information such as genome-genome comparisons and EST and protein alignments. However, such evidence is often incomplete and usually uncertain. The extrinsic evidence is usually not sufficient to recover the complete gene structure of all genes completely and the available evidence is often unreliable. Therefore extrinsic evidence is most valuable when it is balanced with sequence-intrinsic evidence.ResultsWe present a fairly general method for integration of external information. Our method is based on the evaluation of hints to potentially protein-coding regions by means of a Generalized Hidden Markov Model (GHMM) that takes both intrinsic and extrinsic information into account. We used this method to extend the ab initio gene prediction program AUGUSTUS to a versatile tool that we call AUGUSTUS+. In this study, we focus on hints derived from matches to an EST or protein database, but our approach can be used to include arbitrary user-defined hints. Our method is only moderately effected by the length of a database match. Further, it exploits the information that can be derived from the absence of such matches. As a special case, AUGUSTUS+ can predict genes under user-defined constraints, e.g. if the positions of certain exons are known. With hints from EST and protein databases, our new approach was able to predict 89% of the exons in human chromosome 22 correctly.ConclusionSensitive probabilistic modeling of extrinsic evidence such as sequence database matches can increase gene prediction accuracy. When a match of a sequence interval to an EST or protein sequence is used it should be treated as compound information rather than as information about individual positions.

Efficient decoding algorithms for generalized hidden Markov model gene finders

BackgroundThe Generalized Hidden Markov Model (GHMM) has proven a useful framework for the task of computational gene prediction in eukaryotic genomes, due to its flexibility and probabilistic underpinnings. As the focus of the gene finding community shifts toward the use of homology information to improve prediction accuracy, extensions to the basic GHMM model are being explored as possible ways to integrate this homology information into the prediction process. Particularly prominent among these extensions are those techniques which call for the simultaneous prediction of genes in two or more genomes at once, thereby increasing significantly the computational cost of prediction and highlighting the importance of speed and memory efficiency in the implementation of the underlying GHMM algorithms. Unfortunately, the task of implementing an efficient GHMM-based gene finder is already a nontrivial one, and it can be expected that this task will only grow more onerous as our models increase in complexity.ResultsAs a first step toward addressing the implementation challenges of these next-generation systems, we describe in detail two software architectures for GHMM-based gene finders, one comprising the common array-based approach, and the other a highly optimized algorithm which requires significantly less memory while achieving virtually identical speed. We then show how both of these architectures can be accelerated by a factor of two by optimizing their content sensors. We finish with a brief illustration of the impact these optimizations have had on the feasibility of our new homology-based gene finder, TWAIN.ConclusionsIn describing a number of optimizations for GHMM-based gene finders and making available two complete open-source software systems embodying these methods, it is our hope that others will be more enabled to explore promising extensions to the GHMM framework, thereby improving the state-of-the-art in gene prediction techniques.

GlimmerM, Exonomy and Unveil: three ab initio eukaryotic genefinders.

We present three programs for ab initio gene prediction in eukaryotes: Exonomy, Unveil and GlimmerM. Exonomy is a 23-state Generalized Hidden Markov Model (GHMM), Unveil is a 283-state standard Hidden Markov Model (HMM) and GlimmerM is a previously-described genefinder which utilizes decision trees and Interpolated Markov Models (IMMs). All three are readily re-trainable for new organisms and have been found to perform well compared to other genefinders. Results are presented for Arabidopsis thaliana. Cases have been found where each of the genefinders outperforms each of the others, demonstrating the collective value of this ensemble of genefinders. These programs are all accessible through webservers at http://www.tigr.org/software.

MIXING AND MOMENT PROPERTIES OF VARIOUS GARCH AND STOCHASTIC VOLATILITY MODELS

This paper first provides some useful results on a generalized random coefficient autoregressive model and a generalized hidden Markov model. These results simultaneously imply strict stationarity, existence of higher order moments, geometric ergodicity, and β-mixing with exponential decay rates, which are important properties for statistical inference. As applications, we then provide easy-to-verify sufficient conditions to ensure β-mixing and finite higher order moments for various linear and nonlinear GARCH(1,1), linear and power GARCH(p,q), stochastic volatility, and autoregressive conditional duration models. For many of these models, our sufficient conditions for existence of second moments and exponential β-mixing are also necessary. For several GARCH(1,1) models, our sufficient conditions for existence of higher order moments again coincide with the necessary ones in He and Terasvirta (1999, Journal of Econometrics 92, 173–192).

Phat—a gene finding program for Plasmodium falciparum

We describe and assess the performance of the gene finding program pretty handy annotation tool (Phat) on sequence from the malaria parasite Plasmodium falciparum. Phat is based on a generalized hidden Markov model (GHMM) similar to the models used in GENSCAN, Genie and HMMgene. In a test set of 44 confirmed gene structures Phat achieves nucleotide-level sensitivity and specificity of greater than 95%, performing as well as the other P. falciparum gene finding programs Hexamer and GlimmerM. Phat is particularly useful for P. falciparum and other eukaryotes for which there are few gene finding programs available as it is distributed with code for retraining it on new organisms. Moreover, the full source code is freely available under the GNU General Public License, allowing for users to further develop and customize it.

Improved splice site detection in Genie.

We present an improved splice site predictor for the genefinding program Genie. Genie is based on a generalized Hidden Markov Model (GHMM) that describes the grammar of a legal parse of a multi-exon gene in a DNA sequence. In Genie, probabilities are estimated for gene features by using dynamic programming to combine information from multiple content and signal sensors, including sensors that integrate matches to homologous sequences from a database. One of the hardest problems in genefinding is to determine the complete gene structure correctly. The splice site sensors are the key signal sensors that address this problem. We replaced the existing splice site sensors in Genie with two novel neural networks based on dinucleotide frequencies. Using these novel sensors, Genie shows significant improvements in the sensitivity and specificity of gene structure identification. Experimental results in tests using a standard set of annotated genes showed that Genie identified 86% of coding nucleotides correctly with a specificity of 85%, versus 80% and 84% in the older system. In further splice site experiments, we also looked at correlations between splice site scores and intron and exon lengths, as well as at the effect of distance to the nearest splice site on false positive rates.