Markov Random Walk Research Articles

Velocity formulae between entropy and hitting time for Markov chains

In the absence of acceleration, the velocity formula gives “distance travelled equals speed multiplied by time”. For a broad class of Markov chains such as circulant Markov chains or random walk on complete graphs, we prove a probabilistic analogue of the velocity formula between entropy and hitting time, where distance is the entropy of the Markov trajectories from state i to state j in the sense of Ekroot and Cover (1993), speed is the classical entropy rate of the chain, and the time variable is the expected hitting time between i and j. This motivates us to define new entropic counterparts of various hitting time parameters such as average hitting time or commute time, and prove analogous velocity formulae and estimates between these quantities.

Random walks with long-range steps generated by functions of Laplacian matrices

In this paper, we explore different Markovian random walk strategies on networks with transition probabilities between nodes defined in terms of functions of the Laplacian matrix. We generalize random walk strategies with local information in the Laplacian matrix, that describes the connections of a network, to a dynamic determined by functions of this matrix. The resulting processes are non-local allowing transitions of the random walker from one node to nodes beyond its nearest neighbors. We find that only two types of Laplacian functions are admissible with distinct behaviors for long-range steps in the infinite network limit: type (i) functions generate Brownian motions, type (ii) functions Lévy flights. For this asymptotic long-range step behavior only the lowest non-vanishing order of the Laplacian function is relevant, namely first order for type (i), and fractional order for type (ii) functions.In the first part, we discuss spectral properties of the Laplacian matrix and a series of relations that are maintained by a particular type of functions that allow to define random walks on any type of undirected connected networks. Once described general properties, we explore characteristics of random walk strategies that emerge from particular cases with functions defined in terms of exponentials, logarithms and powers of the Laplacian as well as relations of these dynamics with non-local strategies like Lévy flights and fractional transport. Finally, we analyze the global capacity of these random walk strategies to explore networks like lattices and trees and different types of random and complex networks.

An arcsine law for Markov random walks

The classic arcsine law for the number Nn>≔n−1∑k=1n1{Sk>0} of positive terms, as n→∞, in an ordinary random walk (Sn)n≥0 is extended to the case when this random walk is governed by a positive recurrent Markov chain (Mn)n≥0 on a countable state space S, that is, for a Markov random walk (Mn,Sn)n≥0 with positive recurrent discrete driving chain. More precisely, it is shown that n−1Nn> converges in distribution to a generalized arcsine law with parameter ρ∈[0,1] (the classic arcsine law if ρ=1∕2) iff the Spitzer condition limn→∞1n∑k=1nPi(Sn>0)=ρholds true for some and then all i∈S, where Pi≔P(⋅|M0=i) for i∈S. It is also proved, under an extra assumption on the driving chain if 0<ρ<1, that this condition is equivalent to the stronger variant limn→∞Pi(Sn>0)=ρ.For an ordinary random walk, this was shown by Doney (1995) for 0<ρ<1 and by Bertoin and Doney (1997) for ρ∈{0,1}.

Locally weighted embedding topic modeling by markov random walk structure approximation and sparse regularization

Abstract Topic model is a practical method for learning interpretable models of text corpora and have become a key problem of document representation. Some recently proposed topic models incorporate the intrinsic geometrical information of the document manifold and yield a discriminative topic representation. However, the existing manifold-inspired topic models fail to provide the probability weighting information of local geometrical pattern, thus leads to a limitation to estimate intrinsic semantic information of topic representation. In this paper, we consider the problem of topic modeling with intrinsic structure of document manifold and propose an unsupervised AutoEncoder-based topic modeling framework, named locally weighted embedding topic model (LWE-TM). Different from existing manifold-inspired topic models, LWE-TM defines a group of probability coefficients to uncover the local geometrical pattern by the Markov random walk structure of affinity graph, and regularizes the training of sparse AutoEncoder (sAE) to explicitly recover such local geometrical pattern with the topics encoding. Under the regularized training framework, the encoding network becomes local-invariant around the neighborhood of the document manifold and enable us to perform a readily topic inference for out-of-sample documents, efficiently improving the generalization and discrimination of topics encoding. The experimental results on two widely-used corpus demonstrate the superiority of LWE-TM to comparative models in document modeling, document clustering and classification tasks.

On the mixing time in the Wang-Landau algorithm

We present preliminary results of the investigation of the properties of the Markov random walk in the energy space generated by the Wang-Landau probability. We build transition matrix in the energy space (TMES) using the exact density of states for one-dimensional and two-dimensional Ising models. The spectral gap of TMES is inversely proportional to the mixing time of the Markov chain. We estimate numerically the dependence of the mixing time on the lattice size, and extract the mixing exponent.

Recurrence of multidimensional persistent random walks. Fourier and series criteria

The recurrence and transience of persistent random walks built from variable length Markov chains are investigated. It turns out that these stochastic processes can be seen as Lévy walks for which the persistence times depend on some internal Markov chain: they admit Markov random walk skeletons. A recurrence versus transience dichotomy is highlighted. Assuming the positive recurrence of the driving chain, a sufficient Fourier criterion for the recurrence, close to the usual Chung–Fuchs one, is given and a series criterion is derived. The key tool is the Nagaev–Guivarc’h method. Finally, we focus on particular two-dimensional persistent random walks, including directionally reinforced random walks, for which necessary and sufficient Fourier and series criteria are obtained. Inspired by (Adv. Math. 208 (2007) 680–698), we produce a genuine counterexample to the conjecture of (Adv. Math. 117 (1996) 239–252). As for the one-dimensional case studied in (J. Theoret. Probab. 31 (2018) 232–243), it is easier for a persistent random walk than its skeleton to be recurrent. However, such examples are much more difficult to exhibit in the higher dimensional context. These results are based on a surprisingly novel – to our knowledge – upper bound for the Lévy concentration function associated with symmetric distributions.

Bivariate Markov chains converging to Lamperti transform Markov additive processes

Motivated by various applications, we describe the scaling limits of bivariate Markov chains (X,J) on Z+×{1,…,κ} where X can be viewed as a position marginal and {1,…,κ} is a set of κ types. The chain starts from an initial value (n,i)∈Z+×{1,…,κ}, with i fixed and n→∞, and typically we will assume that the macroscopic jumps of the marginal X are rare, i.e. arrive with a probability proportional to a negative power of the current state. We also assume that X is non-increasing. We then observe different asymptotic regimes according to whether the rate of type change is proportional to, faster than, or slower than the macroscopic jump rate. In these different situations, we obtain in the scaling limit Lamperti transforms of Markov additive processes, that sometimes reduce to standard positive self-similar Markov processes. As first examples of applications, we study the number of collisions in coalescents in varying environment and the scaling limits of Markov random walks with a barrier. This completes previous results obtained by Haas and Miermont (2011) and Bertoin and Kortchemski (2016) in the monotype setting. In a companion paper, we will use these results as a building block to study the scaling limits of multi-type Markov branching trees, with applications to growing models of random trees and multi-type Galton–Watson trees.

Recurrence of random walks with long-range steps generated by fractional Laplacian matrices on regular networks and simple cubic lattices

We analyze a Markovian random walk strategy on undirected regular networks involving power matrix functions of the type where L indicates a ‘simple’ Laplacian matrix. We refer to such walks as ‘fractional random walks’ with admissible interval . We deduce probability-generating functions (network Green’s functions) for the fractional random walk. From these analytical results we establish a generalization of Polya’s recurrence theorem for fractional random walks on d-dimensional infinite lattices: The fractional random walk is transient for dimensions (recurrent for ) of the lattice. As a consequence, for the fractional random walk is transient for all lattice dimensions and in the range for dimensions . Finally, for , Polya’s classical recurrence theorem is recovered, namely the walk is transient only for lattice dimensions . The generalization of Polya’s recurrence theorem remains valid for the class of random walks with Lévy flight asymptotics for long-range steps. We also analyze the mean first passage probabilities, mean residence times, mean first passage times and global mean first passage times (Kemeny constant) for the fractional random walk. For an infinite 1D lattice (infinite ring) we obtain for the transient regime closed form expressions for the fractional lattice Green’s function matrix containing the escape and ever passage probabilities. The ever passage probabilities (fractional lattice Green’s functions) in the transient regime fulfil Riesz potential power law decay asymptotic behavior for nodes far from the departure node. The non-locality of the fractional random walk is generated by the non-diagonality of the fractional Laplacian matrix with Lévy-type heavy tailed inverse power law decay for the probability of long-range moves. This non-local and asymptotic behavior of the fractional random walk introduces small-world properties with the emergence of Lévy flights on large (infinite) lattices.

Modeling Spatial Covariance Using the Limiting Distribution of Spatio-Temporal Random Walks

ABSTRACTWe present an approach for modeling areal spatial covariance in observed genetic allele data by considering the stationary (limiting) distribution of a spatio-temporal Markov random walk model for gene flow. This stationary distribution corresponds to an intrinsic simultaneous autoregressive (SAR) model for spatial correlation, and provides a principled approach to specifying areal spatial models when a spatio-temporal generating process can be assumed. We apply the approach to a study of spatial genetic variation of trout in a stream network in Connecticut, USA.

When Group Dispersal and Allee Effect Shape Metapopulation Dynamics

The dispersal ability of a species will be critical for how population dynamics are realized in spatially structured systems. To date, the effect of group dispersal on metapopulation dynamics is poorly understood. Here, we investigate how group dispersal and Allee effects shape metapopulation dynamics identifying conditions in which group dispersal can be an advantage over independent dispersal. We approach this question by building and analysing a Markovian random walk for metapopulation dynamics including group dispersal and Allee effect. This Markovian random walk is analogous to the discrete-time Stochastic Patch Occupancy Model (SPOM). We find that intermediate group sizes may lead to larger and more sustainable metapopulations in the presence of an Allee effect. Hence, understanding how group size variation and realized (meta)population dynamics are linked offers an exciting future venue for research that is expected to yield key insights into the ecology and evolution of populations occupying spati...

Fluctuation theorems for discrete kinetic models of molecular motors

Motivated by discrete kinetic models for non-cooperative molecular motors on periodic tracks, we consider random walks (also not Markov) on quasi one dimensional (1d) lattices, obtained by gluing several copies of a fundamental graph in a linear fashion. We show that, for a suitable class of quasi-1d lattices, the large deviation rate function associated to the position of the walker satisfies a Gallavotti–Cohen symmetry for any choice of the dynamical parameters defining the stochastic walk. This class includes the linear model considered in Lacoste et al (2008 Phys. Rev. E 78 011915). We also derive fluctuation theorems for the time-integrated cycle currents and discuss how the matrix approach of Lacoste et al (2008 Phys. Rev. E 78 011915) can be extended to derive the above Gallavotti–Cohen symmetry for any Markov random walk on with periodic jump rates. Finally, we review in the present context some large deviation results of Faggionato and Silvestri (2017 Ann. Inst. Henri Poincaré 53 46–78) and give some specific examples with explicit computations.

Upscaling of dilution and mixing using a trajectory based Spatial Markov random walk model in a periodic flow domain

The Spatial Markov Model (SMM) is an upscaled model that has been used successfully to predict effective mean transport across a broad range of hydrologic settings. Here we propose a novel variant of the SMM, applicable to spatially periodic systems. This SMM is built using particle trajectories, rather than travel times. By applying the proposed SMM to a simple benchmark problem we demonstrate that it can predict mean effective transport, when compared to data from fully resolved direct numerical simulations. Next we propose a methodology for using this SMM framework to predict measures of mixing and dilution, that do not just depend on mean concentrations, but are strongly impacted by pore-scale concentration fluctuations. We use information from trajectories of particles to downscale and reconstruct pore-scale approximate concentration fields from which mixing and dilution measures are then calculated. The comparison between measurements from fully resolved simulations and predictions with the SMM agree very favorably.

Stability of perpetuities in Markovian environment

The stability of iterations of affine linear maps , , is studied in the presence of a Markovian environment, more precisely, for the situation when is modulated by an ergodic Markov chain with countable state space and stationary distribution . We provide necessary and sufficient conditions for the a.s. and the distributional convergence of the backward iterations and also describe all possible limit laws as solutions to a certain Markovian stochastic fixed-point equation. As a consequence of the random environment, these limit laws are stochastic kernels from to rather than distributions on , thus reflecting their dependence on where the driving chain is started. We give also necessary and sufficient conditions for the distributional convergence of the forward iterations . The main differences caused by the Markovian environment as opposed to the extensively studied case of independent and identically distributed (iid) are that: (1) backward iterations may still converge in distribution if a.s. convergence fails, (2) the degenerate case when a.s. for suitable constants , , is by far more complex than the degenerate case for iid when a.s. for some , and (3) forward and backward iterations generally have different laws given for so that the former ones need a separate analysis. Our proofs draw on related results for the iid-case, notably by Vervaat, Grincevičius, and Goldie and Maller, in combination with recent results by the authors on fluctuation theory for Markov random walks.

Sovereign Credit Rating, Rating Migration, and the Risk-Free Rate: A Joint Markov Process and Random Walk Modelling of the Risk-Free Rate

The study proposes and tests a risk-free rate model that simultaneously lets the risk-free rate migrate between rating categories as risk-free rate ranges, and follow a random walk within rating categories as risk-free rate ranges. This is seen as a more accurate modelling of the risk-free rate, that better allows for random shifts in the risk-free rate, due to events and shocks. A benefit of the risk-free rate model is that it is entirely ahistorical. Also, future risk-free rate probability distributions can easily be generated by the model. This has particular benefits for exchange rate – currency forwards – valuation. The risk-free rate model greatly infers and builds on the relationship between the current risk-free rate, and likely future risk-free rates, as well as risk-free rate volatility. It also clusters risk-free rates into rating categories, and rating categories are assigned risk-free rate ranges. This then presumes that risk-free rates falling in the same rating category have comparable characteristics, most notably in terms of probable future risk-free rates and risk-free rate volatility. It also presumes that sovereign credit ratings as rating categories contain risk-free rate data on risk-free rate ranges, and risk-free rate volatility, and that sovereign credit ratings are or should be comparable or universal. Although the study arbitrarily assigned rating categories, and risk-free rate ranges to the rating categories, empirical research can clarify this, by examining the relationship between the risk-free rate and risk-free rate volatility, and by examining the relationship between sovereign credit ratings and risk-free rate ranges as well as risk-free rate volatility. Firstly, comparable risk-free rates should illustrate comparable risk-free rate volatility, and risk-free rates should cluster in terms of their risk-free rate volatility characteristics. Secondly, sovereign credit ratings should demonstrate risk-free rate ranges and risk-free rate volatility characteristics. To test the model, a risk-free bond portfolio, together with a risk-free rate rating migration matrix were simulated. The rating migration matrix governs the migration between risk-free rate rating categories. It is shown that the original migration matrix can again be decomposed with adequate accuracy, given that the appropriate constraints are used. It indicates that the model can be applied to empirical markets. Possible refinements to the model are noted.

A prediction-based handoff scheme for QoS in WLAN systems

In this paper, we introduce a technique to reduce the authentication overhead for handoff process in wireless local area networks (WLAN). It is based on user mobility prediction to reduce the network scanning area during the handoff procedure. The central server facilitates an intelligent mobility prediction mechanism to identify the next possible mobile access points for mobile subscriber. The predicted information is managed by an event log table, and it helps to reduce the authorisation overheads for cellular networks. It follows the reserve channel scheme to maintain the QoS for users during the call and handoff procedure and the handover-call has higher priority over local initiated call. The analytical model shows the effectiveness and efficiency of proposed work, we apply the Markov random walk model and M/M/1 queuing model to formulate our work. It is compared with the tradition technique, and has 40% better result than legacy techniques.

Modelling the Paradox in Stock Markets by Variance Ratio Volatility Estimator that Utilises Extreme Values of Asset Prices

We document the presence of the random walk effect in stock indices and, at the same time, find that the constituent stocks of the indices are excessively volatile. This gives rise to a paradox in stock markets between the behaviour of the stock index and its constituent stocks. We address this phenomenon in this article and reconcile the seemingly contradictory inferences by extending the Binomial Markov Random Walk (BMRW) model. JEL Classification: C15, C58, G15

Functional equation for the crossover in the model of one-dimensional Weierstrass random walks

We consider the problem of one-dimensional symmetric diffusion in the framework of Markov random walks of the Weierstrass type using two-parameter scaling for the transition probability. We construct a solution for the characteristic Lyapunov function as a sum of regular (homogeneous) and singular (nonhomogeneous) solutions and find the conditions for the crossover from normal to anomalous diffusion.

Reciprocal classes of random walks on graphs

The reciprocal class of a Markov path measure is the set of all the mixtures of its bridges. We give characterizations of the reciprocal class of a continuous-time Markov random walk on a graph. Our main result is in terms of some reciprocal characteristics whose expression only depends on the intensity of jump. We also characterize the reciprocal class by means of Taylor expansions in small time of some conditional probabilities.Our measure-theoretical approach allows to extend significantly already known results on the subject. The abstract results are illustrated by several examples.

Effective comparative analysis of protein-protein interaction networks by measuring the steady-state network flow using a Markov model.

BackgroundComparative analysis of protein-protein interaction (PPI) networks provides an effective means of detecting conserved functional network modules across different species. Such modules typically consist of orthologous proteins with conserved interactions, which can be exploited to computationally predict the modules through network comparison.ResultsIn this work, we propose a novel probabilistic framework for comparing PPI networks and effectively predicting the correspondence between proteins, represented as network nodes, that belong to conserved functional modules across the given PPI networks. The basic idea is to estimate the steady-state network flow between nodes that belong to different PPI networks based on a Markov random walk model. The random walker is designed to make random moves to adjacent nodes within a PPI network as well as cross-network moves between potential orthologous nodes with high sequence similarity. Based on this Markov random walk model, we estimate the steady-state network flow – or the long-term relative frequency of the transitions that the random walker makes – between nodes in different PPI networks, which can be used as a probabilistic score measuring their potential correspondence. Subsequently, the estimated scores can be used for detecting orthologous proteins in conserved functional modules through network alignment.ConclusionsThrough evaluations based on multiple real PPI networks, we demonstrate that the proposed scheme leads to improved alignment results that are biologically more meaningful at reduced computational cost, outperforming the current state-of-the-art algorithms. The source code and datasets can be downloaded from http://www.ece.tamu.edu/~bjyoon/CUFID.

Locally Discriminant Diffusion Projection and Its Application in Speech Emotion Recognition

The existing Diffusion Maps method brings diffusion to data samples by Markov random walk. In this paper, to provide a general solution form of Diffusion Maps, first, we propose the generalized single-graph-diffusion embedding framework on the basis of graph embedding framework. Second, by designing the embedding graph of the framework, an algorithm, namely Locally Discriminant Diffusion Projection (LDDP), is proposed for speech emotion recognition. This algorithm is the projection form of the improved Diffusion Maps, which includes both discriminant information and local information. The linear or kernelized form of LDDP (i.e., LLDDP or KLDDP) is used to achieve the dimensionality reduction of original speech emotion features. We validate the proposed algorithm on two widely used speech emotion databases, EMO-DB and eNTERFACE'05. The experimental results show that the proposed LDDP methods, including LLDDP and KLDDP, outperform some other state-of-the-art dimensionality reduction methods which are based on graph embedding or discriminant analysis.