Kth-order Markov Research Articles

A sufficient condition for a unique invariant distribution of a higher-order Markov chain

We derive a sufficient condition for a kth order homogeneous Markov chain Z with finite alphabet Z to have a unique invariant distribution on Zk. Specifically, let X be a first-order, stationary Markov chain with finite alphabet X and a single recurrent class, let g:X→Z be non-injective, and define the (possibly non-Markovian) process Y:=g(X) (where g is applied coordinate-wise). If Z is the kth order Markov approximation of Y, its invariant distribution is unique. We generalize this to non-Markovian processes X.

Kth-order Markov extremal models for assessing heatwave risks

Heatwaves are defined as a set of hot days and nights that cause a marked short-term increase in mortality. Obtaining accurate estimates of the probability of an event lasting many days is important. Previous studies of temporal dependence of extremes have assumed either a first-order Markov model or a particularly strong form of extremal dependence, known as asymptotic dependence. Neither of these assumptions is appropriate for the heatwaves that we observe for our data. A first-order Markov assumption does not capture whether the previous temperature values have been increasing or decreasing and asymptotic dependence does not allow for asymptotic independence, a broad class of extremal dependence exhibited by many processes including all non-trivial Gaussian processes. This paper provides a kth-order Markov model framework that can encompass both asymptotic dependence and asymptotic independence structures. It uses a conditional approach developed for multivariate extremes coupled with copula methods for time series. We provide novel methods for the selection of the order of the Markov process that are based upon only the structure of the extreme events. Under this new framework, the observed daily maximum temperatures at Orleans, in central France, are found to be well modelled by an asymptotically independent third-order extremal Markov model. We estimate extremal quantities, such as the probability of a heatwave event lasting as long as the devastating European 2003 heatwave event. Critically our method enables the first reliable assessment of the sensitivity of such estimates to the choice of the order of the Markov process.

Growth and Dissolution of Macromolecular Markov Chains

The kinetics and thermodynamics of free living copolymerization are studied for processes with rates depending on k monomeric units of the macromolecular chain behind the unit that is attached or detached. In this case, the sequence of monomeric units in the growing copolymer is a kth-order Markov chain. In the regime of steady growth, the statistical properties of the sequence are determined analytically in terms of the attachment and detachment rates. In this way, the mean growth velocity as well as the thermodynamic entropy production and the sequence disorder can be calculated systematically. These different properties are also investigated in the regime of depolymerization where the macromolecular chain is dissolved by the surrounding solution. In this regime, the entropy production is shown to satisfy Landauer's principle.

Lumpings of Markov Chains, Entropy Rate Preservation, and Higher-Order Lumpability

A lumping of a Markov chain is a coordinatewise projection of the chain. We characterise the entropy rate preservation of a lumping of an aperiodic and irreducible Markov chain on a finite state space by the random growth rate of the cardinality of the realisable preimage of a finite-length trajectory of the lumped chain and by the information needed to reconstruct original trajectories from their lumped images. Both are purely combinatorial criteria, depending only on the transition graph of the Markov chain and the lumping function. A lumping is strongly k-lumpable, if and only if the lumped process is a kth-order Markov chain for each starting distribution of the original Markov chain. We characterise strong k-lumpability via tightness of stationary entropic bounds. In the sparse setting, we give sufficient conditions on the lumping to both preserve the entropy rate and be strongly k-lumpable.

Markov Models in the Analysis of Frequent Patterns in Financial Data

Frequent sequence mining is one of the main challenges in data mining and especially in large databases, which consist of millions of records. There is a number of different applications where frequent sequence mining is very important: medicine, finance, internet behavioural data, marketing data, etc. Exact frequent sequence mining methods make multiple passes over the database and if the database is large, then it is a time consuming and expensive task. Approximate methods for frequent sequence mining are faster than exact methods because instead of doing multiple passes over the original database, they analyze a much shorter sample of the original database formed in a specific way. This paper presents Markov Property Based Method (MPBM) – an approximate method for mining frequent sequences based on kth order Markov models, which makes only several passes over the original database. The method has been implemented and evaluated using real-world foreign exchange database and compared to exact and approximate frequent sequent mining algorithms.

Accurate and effective inference of network link loss from unicast end-to-end measurements

Understanding the network link loss is particularly important for optimising delay-sensitive applications. This study addresses the issue of estimating temporal dependence characteristic of link loss by using network tomography. Different from existing works of network loss tomography, the authors use a kth order Markov Chain (k-MC for short, k>1) to model the packet loss process, and propose a constrained optimisation-based method to estimate the state transmission probabilities of the k-MC link loss model. The authors also propose a top–down algorithm in order to ensure that our method can be applied to large networks. Compared with existing loss tomography methods, our method is capable of obtaining more accurate packet loss probability estimates. The ns-2 simulation results show the good performance of our method.

Optimal instruments and models for noisy chaos

Analysis of finite, noisy time series data leads to modern statistical inference methods. Here we adapt Bayesian inference for applied symbolic dynamics. We show that reconciling Kolmogorov's maximum-entropy partition with the methods of Bayesian model selection requires the use of two separate optimizations. First, instrument design produces a maximum-entropy symbolic representation of time series data. Second, Bayesian model comparison with a uniform prior selects a minimum-entropy model, with respect to the considered Markov chain orders, of the symbolic data. We illustrate these steps using a binary partition of time series data from the logistic and Henon maps as well as the Rössler and Lorenz attractors with dynamical noise. In each case we demonstrate the inference of effectively generating partitions and kth-order Markov chain models.

Asymptotic Redundancy of the MTF Scheme for Stationary Ergodic Sources

The Move-to-front (MTF) scheme is a data-compression method which converts each symbol of a source sequence to a positive integer sequentially, and encodes it to a binary codeword. The compression performance of this algorithm has been analyzed usually under the assumption of the so-called symbol extension. But, in this paper, upper and lower bounds are derived for the redundancy of the MTF scheme without the symbol extension for stationary ergodic sources and Markov sources. It is also proved that for the stationary ergodic first-order Markov sources, the MTF scheme can attain the entropy rate if and only if the transition matrix of the source is a kind of doubly stochastic matrix. Moreover, if the source is a Kth-order Markov source (K/spl ges/2), the MTF scheme cannot attain the entropy rate of the source generally.

Generalizations of Markov model to characterize biological sequences

BackgroundThe currently used kth order Markov models estimate the probability of generating a single nucleotide conditional upon the immediately preceding (gap = 0) k units. However, this neither takes into account the joint dependency of multiple neighboring nucleotides, nor does it consider the long range dependency with gap>0.ResultWe describe a configurable tool to explore generalizations of the standard Markov model. We evaluated whether the sequence classification accuracy can be improved by using an alternative set of model parameters. The evaluation was done on four classes of biological sequences – CpG-poor promoters, all promoters, exons and nucleosome positioning sequences. Using di- and tri-nucleotide as the model unit significantly improved the sequence classification accuracy relative to the standard single nucleotide model. In the case of nucleosome positioning sequences, optimal accuracy was achieved at a gap length of 4. Furthermore in the plot of classification accuracy versus the gap, a periodicity of 10–11 bps was observed which might indicate structural preferences in the nucleosome positioning sequence. The tool is implemented in Java and is available for download at .ConclusionMarkov modeling is an important component of many sequence analysis tools. We have extended the standard Markov model to incorporate joint and long range dependencies between the sequence elements. The proposed generalizations of the Markov model are likely to improve the overall accuracy of sequence analysis tools.

Order Estimation of Markov Chains

Estimators /spl chi//sub n/(X/sub 0/, X/sub 1/, ..., X/sub n/), are described which, when applied to an unknown stationary process taking values from a countable alphabet /spl chi/, converge almost surely to k in case the process is a kth-order Markov chain and to infinity otherwise.

Finite-State Markov Modeling of Correlated Rician-Fading Channels

Stochastic properties of the binary channel that describe the successes and failures of the transmission of a modulated signal over a time-correlated flat-fading channel are considered for investigation. This analysis is employed to develop Kth-order Markov models for such a burst channel. The order of the Markov model that generates accurate analytical models is estimated for a broad range of fading environments. The parameterization and accuracy of an important class of hidden Markov models, known as the Gilbert-Elliott channel (GEC), are also investigated. Fading rates are identified in which the Kth-order Markov model and the GEC model approximate the fading channel with similar accuracy. The latter model is useful for approximating slowly fading processes, since it provides a more compact parameterization.

Modeling frame-level errors in GSM wireless channels

We compare four different approaches to modeling frame-level errors in GSM channels. One of these, the Markov-based Trace Analysis (MTA) model, was developed explicitly for this purpose. The next two, kth-order Markov models and hidden Markov models (HMMs), have been widely used to model loss in wired networks. All three of these have difficulty in modeling empirical GSM frame-level error traces. The MTA model and HMM predict frame-error rates substantially different from that measured from the trace, and all three models have difficulty in capturing the long term temporal correlation structure. We propose a fourth model, the extended On/Off model, which alternates between an On (error-free) and an Off (error-filled) state. The state holding times are taken from mixtures of geometric distributions. We show that this model, with mixtures of 4–7 geometric distributions captures first- and second-order statistics significantly better than the preceding three approaches.

The extremal index of a higher-order stationary Markov chain

The paper presents a method of computing the extremal index of a real-valued, higher-order (kth-order, $k \geq 1$) stationary Markov chain ${X_n}$. The method is based on the assumption that the joint distribution of $k +1$ consecutive variables is in the domain of attraction of some multivariate extreme value distribution. We introduce limiting distributions of some rescaled stationary transition kernels, which are used to define a new $k -1$th-order Markov chain ${Y_n}$, say. Then, the kth-order Markov chain ${Z_n}$ defined by $Z_n = Y_1 + \dots + Y_n$ is used to derive a representation for the extremal index of ${X_n}$. We further establish convergence in distribution of multilevel exceedance point processes for ${X_n}$ in terms of ${Z_n}$. The representations for the extremal index and for quantities characterizing the distributional limits are well suited for Monte Carlo simulation.

Approximating kth-order two-state Markov chains

In this paper, we consider kth-order two-state Markov chains {Xi } with stationary transition probabilities. For k = 1, we construct in detail an upper bound for the total variation d(Sn, Y) = Σ x |𝐏(Sn = x) − 𝐏(Y = x)|, where S n = X 1 + · ··+ Xn and Y is a compound Poisson random variable. We also show that, under certain conditions, d(Sn, Y) converges to 0 as n tends to ∞. For k = 2, the corresponding results are given without derivation. For general k ≧ 3, a conjecture is proposed.

Approximating kth-order two-state Markov chains

In this paper, we consider kth-order two-state Markov chains {Xi} with stationary transition probabilities. For k = 1, we construct in detail an upper bound for the total variation d(Sn, Y) = Σx |𝐏(Sn = x) − 𝐏(Y = x)|, where Sn = X1 + · ··+ Xn and Y is a compound Poisson random variable. We also show that, under certain conditions, d(Sn, Y) converges to 0 as n tends to ∞. For k = 2, the corresponding results are given without derivation. For general k ≧ 3, a conjecture is proposed.

Variable-to-fixed length codes are better than fixed-to-variable length codes for Markov sources

It is demonstrated that for finite-alphabet, kth-order ergodic Markov sources (i.e. memory of k letters), a variable-to-fixed length code is better than the best fixed-to-variable length code (Huffman code). It is shown how to construct a variable-to-fixed length code for a kth order ergodic Markov source, which compresses more effectively than the best fixed-to-variable code. >

Asymptotically optimal classification for multiple tests with empirically observed statistics

The decision problem of testing M hypotheses when the source is Kth-order Markov and there are M (or fewer) training sequences of length N and a single test sequence of length n is considered. K, M, n, N are all given. It is shown what the requirements are on M, n, N to achieve vanishing (exponential) error probabilities and how to determine or bound the exponent. A likelihood ratio test that is allowed to produce a no-match decision is shown to provide asymptotically optimal error probabilities and minimum no-match decisions. As an important serial case, the binary hypotheses problem without rejection is discussed. It is shown that, for this configuration, only one training sequence is needed to achieve an asymptotically optimal test.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

On the Structure of Real-Time Source Coders

The outputs of a discrete time source with memory are to be encoded (“quantized” or “compressed”) into a sequence of discrete variables. From this latter sequence, a receiver must attempt to approximate some features of the source sequence. Operation is in real time, and the distortion measure does not tolerate delays. Such a situation has been investigated over infinite time spans by B. McMillan. In the present work, only finite time spans are considered. The main result is the following. If the source is kth-order Markov, one may, without loss, assume that the encoder forms each output using only the last k source symbols and the present state of the receiver's memory. An example is constructed, which shows that the Markov property is essential. The case of delay is also considered.

Compound Bayes Predictors for Sequences with Apparent Markov Structure

Sequential predictors for binary sequences with no assumptions upon the existence of an underlying process are discussed. The rule offered here induces an expected proportion of errors which differs by 0(n-?) from the Bayes envelope with respect to the observed kth order Markov structure. This extends the compound sequential Bayes work of Robbins, Hannan and Blackwell from sequences with perceived 0th order structure to sequences with perceived kth order structure. The proof follows immediately from applying the 0th order theory to 2k separate subsequences. These results show the essential robustness of procedures which play Bayes with respect to (a perhaps randomized) version of an estimate of the distribution of the past. Such procedures still have asymptotically good properties even when the underlying assumptions for which they were originally developed no longer hold.