Higher-order Markov Processes Research Articles

The entropy rate of Linear Additive Markov Processes.

This work derives a theoretical value for the entropy of a Linear Additive Markov Process (LAMP), an expressive but simple model able to generate sequences with a given autocorrelation structure. Our research establishes that the theoretical entropy rate of a LAMP model is equivalent to the theoretical entropy rate of the underlying first-order Markov Chain. The LAMP model captures complex relationships and long-range dependencies in data with similar expressibility to a higher-order Markov process. While a higher-order Markov process has a polynomial parameter space, a LAMP model is characterised only by a probability distribution and the transition matrix of an underlying first-order Markov Chain. This surprising result can be explained by the information balance between the additional structure imposed by the next state distribution of the LAMP model, and the additional randomness of each new transition. Understanding the entropy of the LAMP model provides a tool to model complex dependencies in data while retaining useful theoretical results. To emphasise the practical applications, we use the LAMP model to estimate the entropy rate of the LastFM, BrightKite, Wikispeedia and Reuters-21578 datasets. We compare estimates calculated using frequency probability estimates, a first-order Markov model and the LAMP model, also considering two approaches to ensure the transition matrix is irreducible. In most cases the LAMP entropy rates are lower than those of the alternatives, suggesting that LAMP model is better at accommodating structural dependencies in the processes, achieving a more accurate estimate of the true entropy.

Partial Autoinformation to Characterize Symbolic Sequences.

An information-theoretic approach to numerically determine the Markov order of discrete stochastic processes defined over a finite state space is introduced. To measure statistical dependencies between different time points of symbolic time series, two information-theoretic measures are proposed. The first measure is time-lagged mutual information between the random variables Xn and Xn+k, representing the values of the process at time points n and n + k, respectively. The measure will be termed autoinformation, in analogy to the autocorrelation function for metric time series, but using Shannon entropy rather than linear correlation. This measure is complemented by the conditional mutual information between Xn and Xn+k, removing the influence of the intermediate values Xn+k−1, …, Xn+1. The second measure is termed partial autoinformation, in analogy to the partial autocorrelation function (PACF) in metric time series analysis. Mathematical relations with known quantities such as the entropy rate and active information storage are established. Both measures are applied to a number of examples, ranging from theoretical Markov and non-Markov processes with known stochastic properties, to models from statistical physics, and finally, to a discrete transform of an EEG data set. The combination of autoinformation and partial autoinformation yields important insights into the temporal structure of the data in all test cases. For first- and higher-order Markov processes, partial autoinformation correctly identifies the order parameter, but also suggests extended, non-Markovian effects in the examples that lack the Markov property. For three hidden Markov models (HMMs), the underlying Markov order is found. The combination of both quantities may be used as an early step in the analysis of experimental, non-metric time series and can be employed to discover higher-order Markov dependencies, non-Markovianity and periodicities in symbolic time series.

On the entropy of a hidden Markov process

We study the entropy rate of a hidden Markov process (HMP) defined by observing the output of a binary symmetric channel whose input is a first-order binary Markov process. Despite the simplicity of the models involved, the characterization of this entropy is a long standing open problem. By presenting the probability of a sequence under the model as a product of random matrices, one can see that the entropy rate sought is equal to a top Lyapunov exponent of the product. This offers an explanation for the elusiveness of explicit expressions for the HMP entropy rate, as Lyapunov exponents are notoriously difficult to compute. Consequently, we focus on asymptotic estimates, and apply the same product of random matrices to derive an explicit expression for a Taylor approximation of the entropy rate with respect to the parameter of the binary symmetric channel. The accuracy of the approximation is validated against empirical simulation results. We also extend our results to higher-order Markov processes and to Rényi entropies of any order.

A neural network model for generating complex birdsong syntax

The singing behavior of songbirds has been investigated as a model of sequence learning and production. The song of the Bengalese finch, Lonchura striata var. domestica, is well described by a finite state automaton including a stochastic transition of the note sequence, which can be regarded as a higher-order Markov process. Focusing on the neural structure of songbirds, we propose a neural network model that generates higher-order Markov processes. The neurons in the robust nucleus of the archistriatum (RA) encode each note; they are activated by RA-projecting neurons in the HVC (used as a proper name). We hypothesize that the same note included in different chunks is encoded by distinct RA-projecting neuron groups. From this assumption, the output sequence of RA is a higher-order Markov process, even though the RA-projecting neurons in the HVC fire on first-order Markov processes. We developed a neural network model of the local circuits in the HVC that explains the mechanism by which RA-projecting neurons transit stochastically on first-order Markov processes. Numerical simulation showed that this model can generate first-order Markov process song sequences.

Copula-Based Dependence Characteriztions and Modeling for Time Series

This paper develops a new unified approach to copula-based modeling and characterizations for time series and stochastic processes. We obtain complete characterizations of many time series dependence structures in terms of copulas corresponding to their finite-dimensional distributions. In particular, we focus on copula-based representations for Markov chains of arbitrary order, m-dependent and r-independent time series as well as martingales and conditionally symmetric processes. Our results provide new methods for modeling time series that have prescribed dependence structures such as, for instance, higher order Markov processes as well as non-Markovian processes that nevertheless satisfy Chapman-Kolmogorov stochastic equations. We also focus on the construction and analysis of new classes of copulas that have flexibility to combine many different dependence properties for time series. Among other results, we present a study of new classes of copulas based on expansions by linear functions (Eyraud-Farlie-Gumbel-Mongenstern copulas), power functions (power copulas) and Fourier polynomials (Fourier copulas) and introduce methods for modeling time series using these classes of dependence functions. We also focus on the study of weak convergence of empirical copula processes in the time series context and obtain new results on asymptotic gaussianity of such processes or a wide class of beta-mixing sequences.

Genome Phylogeny Based on Short-Range Correlations in DNA Sequences

The surprising fact that global statistical properties computed on a genomewide scale may reveal species information has first been observed in studies of dinucleotide frequencies. Here we will look at the same phenomenon with a totally different statistical approach. We show that patterns in the short-range statistical correlations in DNA sequences serve as evolutionary fingerprints of eukaryotes. All chromosomes of a species display the same characteristic pattern, markedly different from those of other species. The chromosomes of a species are sorted onto the same branch of a phylogenetic tree due to this correlation pattern. The average correlation between nucleotides at a distance k is quantified in two independent ways: (i) by estimating it from a higher-order Markov process and (ii) by computing the mutual information function at a distance k. We show how the quality of phylogenetic reconstruction depends on the range of correlation strengths and on the length of the underlying sequence segment. This concept of the correlation pattern as a phylogenetic signature of eukaryote species combines two rather distant domains of research, namely phylogenetic analysis based on molecular observation and the study of the correlation structure of DNA sequences.

A discrete autoregressive process as a model for short-range correlations in DNA sequences

We present a direct way to model short- and medium-range correlations in DNA sequences and to separate them from long-range correlations. To do so, we discuss symbol sequences generated by a discrete autoregressive process of order p, DAR( p). These sequences display higher-order Markov properties but are based on very few parameters. The aim of our investigation is (1) to introduce with such DAR( p) processes a parameter-efficient tool for generating higher-order Markov processes on a discrete alphabet, (2) to study, how the parameters of the process determine the statistical properties of the sequence and (3) to provide the mathematical tools for estimating the parameters from a given experimental sequence. The statistical properties of the generated sequences, expressed in terms of parameters in the DAR( p) process, are monitored with methods from information theory. The implications of our findings for DNA sequences are discussed and some application is given. In particular, it is shown, how short-range correlations in DNA sequences can be parameterised by such a process.

Using Markov models to incorporate serial dependence in studies of vegetation change

In this paper, we re-examine data from an intertidal salt-marsh, recording changes over 14 years. We use a method that explicitly accounts for possible temporal dependency between successive observations. This method employs first-order Markov models. We compare this with other methods used to explore these data without regard for serial dependence, and also investigate whether a higher-order Markov process is desirable. Although a salt-marsh has strong gradients, which might suggest continuous change, the results suggest that, in terms of major processes, two distinct communities are present with relatively weak linkages between them.

Stereoselectivity Aspects in the Condensation of Racemic NCA−Amino Acids in the Presence and Absence of Liposomes

In previous work dealing with the liposome-assisted polycondensation of NCA (N-carboxyanhydride)−amino acids, it has been shown that racemic NCA−Trp condensation affords chains with preferential homochiral sequences: the homochiral blocks become increasingly over-represented with increasing oligomer length if compared with the statistically (Bernoullian) distributions. The present paper deals with three questions related to this stereoselective NCA−Trp condensation. The first question concerns the kinetic mechanism that is responsible for the over-representation of homochiral sequences in the presence as well as in the absence of liposomes. To this aim, we have investigated whether and to what extent the selection of the entering chiral monomer unit is governed by the chirality of the penultimate residue. This analysis allows one to suggest that the polycondensation corresponds to a second- or higher-order Markov process. The second question considered in this paper is whether the preferential formation of homochiral sequences, originally found in NCA−Trp−amino acid condensations, is observed also for amino acids other than Trp as well as for mixtures of different amino acids. To clarify this point, we have analyzed by liquid chromatography mass spectrometry (LC-MS) the condensation products of racemic NCA−d-Leu/perdeuterated NCA−l-Leu and racemic NCA−d-Ile/perdeuterated NCA−l-Ile in an aqueous solution without liposomes. This analysis permits us to show that the preferential homochiral growth occurs also with Leu and Ile. By LC-MS analysis we can show that under the same conditions also mixtures of equal amounts of the l- and d-forms of different amino acids, such as Trp/Leu, Leu/Ile, and Trp/Leu/Ile, exhibit similar behavior. The third question considered in this paper is related to the effect of liposomes on the NCA−Trp condensation, addressing the question of whether the chemical structure of the lipid in the liposomes and consequently their physical state have an influence on the stereoselectivity of the NCA−Trp condensation. To this aim, experiments are carried out at different temperatures in the presence of DPPC (1,2-dipalmitoyl-sn-glycero-3-phosphorylcholine) liposomes, below as well as above the liquid-analogue/crystalline-analogue transition temperature. The answer to the third question is that basically the same results are obtained independently of the physical state of the lipid.

Bayes' estimators of generalized entropies

The order-q Tsallis and Renyi entropy receive broad applications in the statistical analysis of complex phenomena. A generic problem arises, however, when these entropies need to be estimated from observed data. The finite size of data sets can lead to serious systematic and statistical errors in numerical estimates. In this paper, we focus upon the problem of estimating generalized entropies from finite samples and derive the Bayes estimator of the order-q Tsallis entropy, including the order-1 (i.e. the Shannon) entropy, under the assumption of a uniform prior probability density. The Bayes estimator yields, in general, the smallest mean-quadratic deviation from the true parameter as compared with any other estimator. Exploiting the functional relationship between and , we use the Bayes estimator of to estimate the Renyi entropy . We compare these novel estimators with the frequency-count estimators for and . We find by numerical simulations that the Bayes estimator reduces statistical errors of order-q entropy estimates for Bernoulli as well as for higher-order Markov processes derived from the complete genome of the prokaryote Haemophilus influenzae.

On verifying the first-order Markovian assumption for a Rayleigh fading channel model

The use of received signal-to-noise ratio (SNR) as the side information in communication systems has been widely accepted especially when the channel quality is time varying. On many occasions, this side information is treated as the received SNR of the current channel symbol or that of previous symbols. In particular, the first-order Markov channel provides a mathematically tractable model for time-varying channels and uses only the received SNR of the symbol immediately preceding the current one. With the first-order Markovian assumption, given the information of the symbol immediately preceding the current one, any other previous symbol should be independent of the current one. Although the experimental measurements confirm the usefulness of the first-order Markovian assumption, one may argue that second or higher-order Markov processes should provide a more accurate model. We answer this question by showing that given the information corresponding to the previous symbol, the amount of uncertainty remaining in the current symbol should be negligible.

Evidence for Use of Continuous Angle Compensation as a Mechanism for Inerial Guidance by Sea Catfish Arius felis

-The use of inertial guidance for maintaining established directional headings by vertebrates in the absence of external directional cues remains controversial. Questions arise in assigning this capacity to vertebrates primarily because of the relatively low sensitivity of the labyrinth to angular and linear accelerations. Evidence of inertial guidance systems in fishes is limited. Recent studies, however, suggest that the use of inertial guidance by fishes in ocean migration is not as unlikely as was previously thought. This report is the first unambiguous documentation of the use of continuous angle compensation as a mechanism for inertial guidance in a migratory fish, sea catfish Ariusfelis. Although the underlying biophysical mechanism for inertial guidance may be debated, analyses indicate that higherorder Markov processes in the sequence of turns by sea catfish describe the phenomenon mathematically.