Higher-order Markov Models Research Articles

Improving Prediction of Tobacco Use Over Time: Findings from Waves 1-4 of the Population Assessment of Tobacco and Health Study.

First-order Markov models assume future tobacco use behavior is dependent on current tobacco use and are often used to characterize patterns of tobacco use over time. Higher-order Markov models that assume future behavior is dependent on current and prior tobacco use may better estimate patterns of tobacco use. This study compared Markov models of different orders to examine whether incorporating information about tobacco use history improves model estimation of tobacco use and estimated tobacco use transition probabilities. We used data from four waves of the Population Assessment of Tobacco and Health Study. In each Wave, a participant was categorized into one of the following tobacco use states: never smoker, former smoker, menthol cigarette smoker, non-menthol cigarette smoker, or e-cigarette/dual user. We compared first-, second-, and third-order Markov models using multinomial logistic regression and estimated transition probabilities between tobacco use states. The third-order model was the best fit for the data. The percentage of former smokers, menthol cigarette smokers, non-menthol cigarette smokers, and e-cigarette/dual users in Wave 3 that remained in the same tobacco use state in Wave 4 ranged from 63.4% to 97.2%, 29.2% to 89.8%, 34.8% to 89.7%, and 20.5% to 80.0%, respectively, dependent on tobacco use history. Individuals who were current tobacco users, but former smokers in the prior two years, were most likely to quit. Transition probabilities between tobacco use states varied widely depending on tobacco use history. Higher-order Markov models improve estimation of tobacco use over time and can inform understanding of trajectories of tobacco use behavior. Findings from this study suggest that transition probabilities between tobacco use states vary widely depending on tobacco use history. Tobacco product users (cigarette or e-cigarette/dual users) who were in the same tobacco use state in the prior two years were least likely to quit. Individuals who were current tobacco users, but former smokers in the prior two years, were most likely to quit. Quitting smoking for at least two years is an important milestone in the process of cessation.

A Higher-Order Markov Model for a Hybrid Inventory System with Probabilistic Remanufacturing Demand

Abstract This study develops a higher-order Markov model (HOM) for an inventory system with remanufacturing, substitution, and lost sales. Defective and disposed items are other factors that are considered in addition to probabilistic demand for both manufacturing and remanufacturing items. One year is the warranty period for items manufactured, and items sold return from customers to the manufacturer in increasing cumulative percentages over the months of the year. To the best our knowledge, a higher-order Markov model has rarely been used in a hybrid inventory system. The challenge is how to determine the steady state of the system with the probable demand for manufacturing and remanufacturing. We propose a new search algorithm to select the best control strategy from several strategies, and then compare it with the two-phase local search algorithm. Each state deals with (12) a probabilistic demand (policy), so the system steady state is set to (22632) policies in total for each production plan. The results showed profit maximization using the new search algorithm compared with the two-phase local search algorithm. Also, an increase in defective and returned items over time, and therefore an increase in remanufactured items. But it does not satisfy all the demand, so manufacturing increases over time due to substitution. Substitution strategy leads to increase the expected average profit.

Gated Attention Recurrent Neural Network: A Deeping Learning Approach for Radar-Based Precipitation Nowcasting

Precipitation nowcasting predicts the future rainfall intensity in local areas in a brief time that impacts directly on human life. In this paper, we express the precipitation nowcasting as a spatiotemporal sequence prediction problem. Predictive learning for a spatiotemporal sequence aims to construct a model of natural spatiotemporal processes to predict the future frames based on historical frames. The spatiotemporal process is an abstraction of some of the spatial things in nature that change with time, and they usually do not change very dramatically. To simplify the model and facilitate the training, we considered that the spatiotemporal process satisfies the generalized Markov properties. The natural spatiotemporal processes are nonlinear and non-stationary in many aspects. The processes are not satisfied with the first-order Markov properties when making predictions, such as the nonlinear movement, expansion, dissipation, and intensity enhancement of echoes. To describe such complex spatiotemporal variations, higher-order Markov models need to be used for the modeling. However, many of the previous models for spatiotemporal prediction constructed were based on first-order Markov properties, losing information on the higher-order variations. Thus, we propose a recurrent neural network which satisfies the multi-order Markov properties to create more accurate spatiotemporal predictions. In this network, the core component is the memory cell structure of the gated attention mechanism, which combines the current input information, extracts the historical state that best matches the existing input from the historical multi-period memory information, and then predicts the future. Through this principle of the gated attention, we could extract the historical state information that is richer and deeper to predict the future and more accurately describe the changing characteristics of motion. The experiments show that our GARNN network captures the spatiotemporal characteristics better and obtains excellent results in the precipitation forecasting with radar echoes.

Factors That Influence the Choice of Markov Model Order in Discriminating DNA Sequences from Different Sources.

Markov models have frequently been used in genetic sequence analysis. The number of parameters of a Markov model increases exponentially with model order, so it is often recommended that the order be chosen based on the size of data being modeled, lower orders for small and higher orders for large dataset sizes. Approaches based on model selection criterion have also been proposed. An important problem in microbiology and evolutionary biology is to decipher chimeric genomes of microbes, particularly, identify segments of distinct ancestries in genomes and reconstruct the plausible evolutionary scenarios that might have shaped the chimeric genomes in the microbial world. In this study, we assessed a Markov model-based segmentation method for its ability to detect compositionally disparate segments in chimeric sequence constructs as a function of model order, sequence length, and phylogenetic divergence. Our results show that the choice of Markov model order depends on both sequence size and composition. Higher order Markov models were found to be more effective in delineating sequence segments arising from closely related organisms in longer constructs; on the other hand, lower order Markov models were found to be more appropriate in delineating sequence segments arising from distantly related organisms in shorter constructs. These findings are important and timely, with broad implications in fields such as epidemiology that has to deal with the emergence of novel pathogenic chimeras that arise by foreign DNA acquisition, and ecology where chimeric structures may arise in various ecosystems, necessitating more robust approaches for their deconstruction and interpretation.

Bearing-only 2D maneuvering target tracking using smart interacting multiple model filter

In this paper, along with reviewing and analyzing the maneuvering target tracking model, the multiple-model Interacting Multiple Model algorithm is used to solve the maneuvering target tracking problem in the presence of measurement noise. In addition, for reliable estimation another method is proposed, which uses higher-order Markov models to describe the system behavior precisely. It means that the previous two models are used to predict the next model of target in order to present a more better algorithm than the first-order IMM algorithm. In this approach, two models are employed. For each model Extended Kalman Filter is used to randomly estimate states of the target. The final estimation of the maneuvering target consists of these two models. Final target estimation is obtained from a weighted sum of all state estimates. In addition, target tracking is presented with two modes for noise measurement: one is an adaptive method and the other is an assignment of an integer amount considering problem circumstances. In the end, the results are compared.

Webpage Recommendation System Based on the Social Media Semantic Details of the Website

The web page recommendation is generated by using the navigational history from web server log files. Semantic Variable Length Markov Chain Model (SVLMC) is a web page recommendation system used to generate recommendation by combining a higher order Markov model with rich semantic data. The problem of state space complexity and time complexity in SVLMC was resolved by Semantic Variable Length confidence pruned Markov Chain Model (SVLCPMC) and Support vector machine based SVLCPMC (SSVLCPMC) meth-ods respectively. The recommendation accuracy was further improved by quickest change detection using Kullback-Leibler Divergence method. In this paper, socio semantic information is included with the similarity score which improves the recommendation accuracy. The social information from the social websites such as twitter is considered for web page recommendation. Initially number of web pages is collected and the similari-ty between web pages is computed by comparing their semantic information. The term frequency and inverse document frequency (tf-idf) is used to produce a composite weight, the most important terms in the web pages are extracted. Then the Pointwise Mutual Information (PMI) between the most important terms and the terms in the twitter dataset are calculated. The PMI metric measures the closeness between the twitter terms and the most important terms in the web pages. Then this measure is added with the similarity score matrix to provide the socio semantic search information for recommendation generation. The experimental results show that the pro-posed method has better performance in terms of prediction accuracy, precision, F1 measure, R measure and coverage.

High-order Markov model for prediction of secondary crash likelihood considering incident duration

Traffic incidents can create major non-recurring congestion and have the potential to be fatal. Traffic Engineers and researchers have worked vigorously to reduce and prevent traffic crashes and ma...

PKM3: an optimal Markov model for predicting future navigation sequences of the web surfers

Predicting the browsing behavior of the user on the web has gained significant importance, as it improves the productivity of the website owners and also raises the interest of web users. The Markov model has been used immensely for user’s web navigation prediction. To enhance the coverage and accuracy of the Markov model, higher order Markov models are integrated with lower order models. However, this integration results in large state-space complexity. To reduce the state-space complexity, this paper proposes a novel technique, namely Pruned all-Kth modified Markov model (PKM3). PKM3 eliminates the irrelevant states from a higher order model, which have a negligible contribution toward prediction. The proposed model is evaluated on four standard weblogs: BMS, MSWEB, CTI and MSNBC. PKM3 performance was optimal for the website in which pages were closely placed and share high interlinking. This pruning-based optimal model achieves a significant reduction in state-space complexity while maintaining comparable accuracy.

Higher-order Markov models for metagenomic sequence classification.

Alignment-free, stochastic models derived from k-mer distributions representing reference genome sequences have a rich history in the classification of DNA sequences. In particular, the variants of Markov models have previously been used extensively. Higher-order Markov models have been used with caution, perhaps sparingly, primarily because of the lack of enough training data and computational power. Advances in sequencing technology and computation have enabled exploitation of the predictive power of higher-order models. We, therefore, revisited higher-order Markov models and assessed their performance in classifying metagenomic sequences. Comparative assessment of higher-order models (HOMs, 9th order or higher) with interpolated Markov model, interpolated context model and lower-order models (8th order or lower) was performed on metagenomic datasets constructed using sequenced prokaryotic genomes. Our results show that HOMs outperform other models in classifying metagenomic fragments as short as 100 nt at all taxonomic ranks, and at lower ranks when the fragment size was increased to 250 nt. HOMs were also found to be significantly more accurate than local alignment which is widely relied upon for taxonomic classification of metagenomic sequences. A novel software implementation written in C++ performs classification faster than the existing Markovian metagenomic classifiers and can therefore be used as a standalone classifier or in conjunction with existing taxonomic classifiers for more robust classification of metagenomic sequences. The software has been made available at https://github.com/djburks/SMM. Rajeev.Azad@unt.edu. Supplementary data are available at Bioinformatics online.

On some Extensions of the Sequential Monte Carlo methods in high-order Hidden Markov Models

Nous analysons quelques extensions des methodes de Monte Carlo sequentielles (SMC) dans le contexte des modeles a espace d'etats non-lineaires. Precisement,, nous adaptons les methodes SMC pour traiter les HMM d'ordre superieur a travers les recursions habituelles des distributions a posteriori. Cela procede par mimer la procedure en deux etapes, c'est-a-dire l'etape de prediction et l'etape de mise a jour, dans la derivation de la distribution du filtre. Une fois obtenu, nous etendons certaines recursions de lissage comme l'algorithme Forward-Backward et l'algorithme Backward Smoother pour traiter les distributions de lissage dans les HMM d'ordre superieur. Enfin, nous donnons quelques exemples de l'application de ces extensions.

BrownieAligner: accurate alignment of Illumina sequencing data to de Bruijn graphs

BackgroundAligning short reads to a reference genome is an important task in many genome analysis pipelines. This task is computationally more complex when the reference genome is provided in the form of a de Bruijn graph instead of a linear sequence string.ResultsWe present a branch and bound alignment algorithm that uses the seed-and-extend paradigm to accurately align short Illumina reads to a graph. Given a seed, the algorithm greedily explores all branches of the tree until the optimal alignment path is found. To reduce the search space we compute upper bounds to the alignment score for each branch and discard the branch if it cannot improve the best solution found so far. Additionally, by using a two-pass alignment strategy and a higher-order Markov model, paths in the de Bruijn graph that do not represent a subsequence in the original reference genome are discarded from the search procedure.ConclusionsBrownieAligner is applied to both synthetic and real datasets. It generally outperforms other state-of-the-art tools in terms of accuracy, while having similar runtime and memory requirements. Our results show that using the higher-order Markov model in BrownieAligner improves the accuracy, while the branch and bound algorithm reduces runtime. BrownieAligner is written in standard C++11 and released under GPL license. BrownieAligner relies on multithreading to take advantage of multi-core/multi-CPU systems. The source code is available at: https://github.com/biointec/browniealigner

Evaluation and Monitoring of Free Running Oscillators Serving as Source of Randomness

In this paper, we evaluate clock signals generated in ring oscillators and self-timed rings and the way their jitter can be transformed into random numbers. We show that counting the periods of the jittery clock signal produces random numbers of significantly better quality than the methods in which the jittery signal is simply sampled (the case in almost all current methods). Moreover, we use the counter values to characterize and continuously monitor the source of randomness. However, instead of using the widely used statistical variance, we propose to use Allan variance to do so. There are two main advantages: Allan variance is insensitive to low frequency noises such as flicker noise that are known to be autocorrelated and significantly less circuitry is required for its computation than that used to compute commonly used variance. We also show that it is essential to use a differential principle of randomness extraction from the jitter based on the use of two identical oscillators to avoid autocorrelations originating from external and internal global jitter sources and that this fact is valid for both kinds of rings. Last but not least, we propose a method of statistical testing based on high order Markov model to show the reduced dependencies when the proposed randomness extraction is applied.

A Novel Entropy-Based Decoding Algorithm for a Generalized High-Order Discrete Hidden Markov Model

The optimal state sequence of a generalized High-Order Hidden Markov Model (HHMM) is tracked from a given observational sequence using the classical Viterbi algorithm. This classical algorithm is based on maximum likelihood criterion. We introduce an entropy-based Viterbi algorithm for tracking the optimal state sequence of a HHMM. The entropy of a state sequence is a useful quantity, providing a measure of the uncertainty of a HHMM. There will be no uncertainty if there is only one possible optimal state sequence for HHMM. This entropy-based decoding algorithm can be formulated in an extended or a reduction approach. We extend the entropy-based algorithm for computing the optimal state sequence that was developed from a first-order to a generalized HHMM with a single observational sequence. This extended algorithm performs the computation exponentially with respect to the order of HMM. The computational complexity of this extended algorithm is due to the growth of the model parameters. We introduce an efficient entropy-based decoding algorithm that used reduction approach, namely, entropy-based order-transformation forward algorithm (EOTFA) to compute the optimal state sequence of any generalized HHMM. This EOTFA algorithm involves a transformation of a generalized high-order HMM into an equivalent first-order HMM and an entropy-based decoding algorithm is developed based on the equivalent first-order HMM. This algorithm performs the computation based on the observational sequence and it requires OTN~2 calculations, where N~ is the number of states in an equivalent first-order model and T is the length of observational sequence.

KVLMM: A Trajectory Prediction Method Based on a Variable-Order Markov Model With Kernel Smoothing

With the dramatic proliferation of global positioning system (GPS) devices, a rich range of research has been conducted on the analysis of GPS trajectories. Research on trajectory prediction uses historical trajectory data to forecast future positions. The typical method is to use a statistical model based on the Markov chain. However, existing models are inefficient in two aspects. The methods of using lower-order Markov models use only current information and ignore historical information, degrading the prediction accuracy. In contrast, higher-order Markov models can improve the prediction accuracy but incur increased time and space complexity. Here, we propose the kernel variable length Markov model (KVLMM), a variable-order Markov model based on kernel smoothing, which combines sequence analysis with the Markov statistical model. The KVLMM can adaptively train trajectory data and learn rules from the training results. When training a large data sample, the KVLMM can rapidly execute training in linearly complex time and space. Moreover, this model uses kernel smoothing when training fewer data samples. In other words, the KVLMM improves the prediction accuracy and reduces the overhead of the data process. Our experimental results show that KVLMM has a lower algorithm complexity and a higher prediction accuracy.

Research of Spread Spectrum Steganography based on High-order Markov Model

The spread spectrum communication technology is widely applied with the spread spectrum principle and very high steganography robustness and security. With the research object of the digital image (DI), this paper carries out a research on the spread spectrum steganalysis. In the research process, this paper adopts the high-order Markov model (HOMM) and designs an experiment of spread spectrum steganalysis by using this model. The research result shows that, compared with the conventional e-secure security index, the high-order statistical measure of the original image (OI) and the carrier image (CI) acquired by using DI HOMM is more sensitive; HOMM based on CI has a superior applicability in analysis of SSIS steganography, presenting a rising trend of accuracy rate and declining trend of false positive with the increase of the model order; HOMM has a quite high applicability in analysis of SSIS steganography. Such applicability is not only embodied in the computation speed and accuracy, but also embodied in the convenience of the actual operation for the relevant technical staff.

Vague Sets Security Measure for Steganographic System Based on High-Order Markov Model

Security measure is of great importance in both steganography and steganalysis. Considering that statistical feature perturbations caused by steganography in an image are always nondeterministic and that an image is considered nonstationary, in this paper, the steganography is regarded as a fuzzy process. Here a steganographic security measure is proposed. This security measure evaluates the similarity between two vague sets of cover images and stego images in terms of n-order Markov chain to capture the interpixel correlation. The new security measure has proven to have the properties of boundedness, commutativity, and unity. Furthermore, the security measures of zero order, first order, second order, third order, and so forth are obtained by adjusting the order value of n-order Markov chain. Experimental results indicate that the larger n is, the better the measuring ability of the proposed security measure will be. The proposed security measure is more sensitive than other security measures defined under a deterministic distribution model, when the embedding is low. It is expected to provide a helpful guidance for designing secure steganographic algorithms or reliable steganalytic methods.

Using higher-order Markov models to reveal flow-based communities in networks.

Complex systems made of interacting elements are commonly abstracted as networks, in which nodes are associated with dynamic state variables, whose evolution is driven by interactions mediated by the edges. Markov processes have been the prevailing paradigm to model such a network-based dynamics, for instance in the form of random walks or other types of diffusions. Despite the success of this modelling perspective for numerous applications, it represents an over-simplification of several real-world systems. Importantly, simple Markov models lack memory in their dynamics, an assumption often not realistic in practice. Here, we explore possibilities to enrich the system description by means of second-order Markov models, exploiting empirical pathway information. We focus on the problem of community detection and show that standard network algorithms can be generalized in order to extract novel temporal information about the system under investigation. We also apply our methodology to temporal networks, where we can uncover communities shaped by the temporal correlations in the system. Finally, we discuss relations of the framework of second order Markov processes and the recently proposed formalism of using non-backtracking matrices for community detection.

Whole genome/proteome based phylogeny reconstruction for prokaryotes using higher order Markov model and chaos game representation

Traditional methods for sequence comparison and phylogeny reconstruction rely on pair wise and multiple sequence alignments. But alignment could not be directly applied to whole genome/proteome comparison and phylogenomic studies due to their high computational complexity. Hence alignment-free methods became popular in recent years. Here we propose a fast alignment-free method for whole genome/proteome comparison and phylogeny reconstruction using higher order Markov model and chaos game representation. In the present method, we use the transition matrices of higher order Markov models to characterize amino acid or DNA sequences for their comparison. The order of the Markov model is uniquely identified by maximizing the average Shannon entropy of conditional probability distributions. Using one-dimensional chaos game representation and linked list, this method can reduce large memory and time consumption which is due to the large-scale conditional probability distributions. To illustrate the effectiveness of our method, we employ it for fast phylogeny reconstruction based on genome/proteome sequences of two species data sets used in previous published papers. Our results demonstrate that the present method is useful and efficient.Availability and implementation: The source codes for our algorithm to get the distance matrix and genome/proteome sequences can be downloaded from ftp://121.199.20.25/. The software Phylip and EvolView we used to construct phylogenetic trees can be referred from their websites.

Predicting Gene Structure with the Use of Mixtures of Probability Distributions

The authors consider the problem of reconstruction of hidden state sequences for mixture distributions with constituents described by the generalization of high-order Markov chains and hidden Markov models. A new algorithm to solve the problem using dynamic programming is proposed, as well as its modifications to eliminate recursion and reduce search. The results are applied to the problem of gene fragment recognition in plants.

Retracted Note

The article "A Study on the Spread Spectrum Steganography Based on the High-order Markov Model" has been withdrawn by the authors. Request approved by the editors and the publisher on September 1st, 2015.