Discrete-time Random Walk Research Articles

Noncolliding system of continuous-time random walks

The continuous-time random walk is defined as a Poissonization of discrete-time random walk. We study the noncolliding system of continuous-time simple and symmetric random walks on . We show that the system is determinantal for any finite initial configuration without multiple point. The spatio-temporal correlation kernel is expressed by using the modified Bessel functions. We extend the system to the noncolliding process with an infinite number of particles, when the initial configuration has equidistant spacing of particles, and show a relaxation phenomenon to the equilibrium determinantal point process with the sine kernel.

Diffusion maps, clustering and fuzzy Markov modeling in peptide folding transitions.

Using the helix-coil transitions of alanine pentapeptide as an illustrative example, we demonstrate the use of diffusion maps in the analysis of molecular dynamics simulation trajectories. Diffusion maps and other nonlinear data-mining techniques provide powerful tools to visualize the distribution of structures in conformation space. The resulting low-dimensional representations help in partitioning conformation space, and in constructing Markov state models that capture the conformational dynamics. In an initial step, we use diffusion maps to reduce the dimensionality of the conformational dynamics of Ala5. The resulting pretreated data are then used in a clustering step. The identified clusters show excellent overlap with clusters obtained previously by using the backbone dihedral angles as input, with small--but nontrivial--differences reflecting torsional degrees of freedom ignored in the earlier approach. We then construct a Markov state model describing the conformational dynamics in terms of a discrete-time random walk between the clusters. We show that by combining fuzzy C-means clustering with a transition-based assignment of states, we can construct robust Markov state models. This state-assignment procedure suppresses short-time memory effects that result from the non-Markovianity of the dynamics projected onto the space of clusters. In a comparison with previous work, we demonstrate how manifold learning techniques may complement and enhance informed intuition commonly used to construct reduced descriptions of the dynamics in molecular conformation space.

On the gap and time interval between the first two maxima of long random walks

.In the context of order statistics of discrete time random walks (RW), we investigate the statistics of the gap, Gn, and the number of time steps, Ln, between the two highest positions of a Markovian one-dimensional random walker, starting from x0 = 0, after n time steps (taking the x-axis vertical). The jumps ηi = xi −xi − 1 are independent and identically distributed random variables drawn from a symmetric probability distribution function (PDF), f(η), the Fourier transform of which has the small k behavior , with 0 < μ ⩽ 2. For μ = 2, the variance of the jump distribution is finite and the RW (properly scaled) converges to a Brownian motion. For 0 < μ < 2, the RW is a Lévy flight of index μ. We show that the joint PDF of Gn and Ln converges to a well defined stationary bi-variate distribution p(g, l) as the RW duration n goes to infinity. We present a thorough analytical study of the limiting joint distribution p(g, l), as well as of its associated marginals pgap(g) and ptime(l), revealing a rich variety of behaviors depending on the tail of f(η) (from slow decreasing algebraic tail to fast decreasing super-exponential tail). We also address the problem for a random bridge where the RW starts and ends at the origin after n time steps. We show that in the large n limit, the PDF of Gn and Ln converges to the same stationary distribution p(g, l) as in the case of the free-end RW. Finally, we present a numerical check of our analytical predictions. Some of these results were announced in a recent letter (Majumdar et al 2013 Phys. Rev. Lett. 111 070601)

A discrete time random walk model for anomalous diffusion

The continuous time random walk, introduced in the physics literature by Montroll and Weiss, has been widely used to model anomalous diffusion in external force fields. One of the features of this model is that the governing equations for the evolution of the probability density function, in the diffusion limit, can generally be simplified using fractional calculus. This has in turn led to intensive research efforts over the past decade to develop robust numerical methods for the governing equations, represented as fractional partial differential equations.Here we introduce a discrete time random walk that can also be used to model anomalous diffusion in an external force field. The governing evolution equations for the probability density function share the continuous time random walk diffusion limit. Thus the discrete time random walk provides a novel numerical method for solving anomalous diffusion equations in the diffusion limit, including the fractional Fokker–Planck equation. This method has the clear advantage that the discretisation of the diffusion limit equation, which is necessary for numerical analysis, is itself a well defined physical process. Some examples using the discrete time random walk to provide numerical solutions of the probability density function for anomalous subdiffusion, including forcing, are provided.

Strong Local Survival of Branching Random Walks is Not Monotone

In this paper we study the strong local survival property for discrete-time and continuous-time branching random walks. We study this property by means of an infinite-dimensional generating functionGand a maximum principle which, we prove, is satisfied by every fixed point ofG. We give results for the existence of a strong local survival regime and we prove that, unlike local and global survival, in continuous time, strong local survival is not a monotone property in the general case (though it is monotone if the branching random walk is quasitransitive). We provide an example of an irreducible branching random walk where the strong local property depends on the starting site of the process. By means of other counterexamples, we show that the existence of a pure global phase is not equivalent to nonamenability of the process, and that even an irreducible branching random walk with the same branching law at each site may exhibit nonstrong local survival. Finally, we show that the generating function of an irreducible branching random walk can have more than two fixed points; this disproves a previously known result.

Strong Local Survival of Branching Random Walks is Not Monotone

In this paper we study the strong local survival property for discrete-time and continuous-time branching random walks. We study this property by means of an infinite-dimensional generating functionGand a maximum principle which, we prove, is satisfied by every fixed point ofG. We give results for the existence of a strong local survival regime and we prove that, unlike local and global survival, in continuous time, strong local survival is not a monotone property in the general case (though it is monotone if the branching random walk is quasitransitive). We provide an example of an irreducible branching random walk where the strong local property depends on the starting site of the process. By means of other counterexamples, we show that the existence of a pure global phase is not equivalent to nonamenability of the process, and that even an irreducible branching random walk with the same branching law at each site may exhibit nonstrong local survival. Finally, we show that the generating function of an irreducible branching random walk can have more than two fixed points; this disproves a previously known result.

Online Markov Decision Processes With Kullback–Leibler Control Cost

This paper considers an online (real-time) control problem that involves an agent performing a discrete-time random walk over a finite state space. The agent's action at each time step is to specify the probability distribution for the next state given the current state. Following the set-up of Todorov, the state-action cost at each time step is a sum of a state cost and a control cost given by the Kullback-Leibler (KL) divergence between the agent's next-state distribution and that determined by some fixed passive dynamics. The online aspect of the problem is due to the fact that the state cost functions are generated by a dynamic environment, and the agent learns the current state cost only after selecting an action. An explicit construction of a computationally efficient strategy with small regret (i.e., expected difference between its actual total cost and the smallest cost attainable using noncausal knowledge of the state costs) under mild regularity conditions is presented, along with a demonstration of the performance of the proposed strategy on a simulated target tracking problem. A number of new results on Markov decision processes with KL control cost are also obtained.

Ultraslow diffusion in an exactly solvable non-Markovian random walk.

We study a one-dimensional discrete-time non-Markovian random walk with strong memory correlations subjected to pauses. Unlike the Scher-Montroll continuous-time random walk, which can be made Markovian by defining an operational time equal to the random-walk step number, the model we study keeps a record of the entire history of the walk. This new model is closely related to the one proposed recently by Kumar, Harbola, and Lindenberg [Phys. Rev. E 82, 021101 (2010)], with the difference that in our model the stochastic dynamics does not stop even in the extreme limit of subdiffusion. Surprisingly, this small difference leads to large consequences. The main results we report here are exact results showing ultraslow diffusion and a stationary diffusion regime (i.e., localization). Specifically, the equations of motion are solved analytically for the first two moments, allowing the determination of the Hurst exponent. Several anomalous diffusion regimes are apparent, ranging from superdiffusion to subdiffusion, as well as ultraslow and stationary regimes. We present the complete phase diffusion diagram, along with a study of the persistence and the statistics in the regions of interest.

On free lunches in random walk markets with short-sale constraints and small transaction costs, and weak convergence to Gaussian continuous-time processes

Abstract. This paper considers a sequence of discrete-time random walk mar-kets with a single risky asset, and gives conditions for the existence of arbitrageopportunitiesorfreeluncheswithvanishingrisk,oftheformofwaitingtobuyandsellingthenextperiod,withnoshorting,andfurthermoreforweakconvergenceoftherandomwalktoaGaussiancontinuous-timestochasticprocess. TheconditionsaregivenintermsofthekernelrepresentationwithrespecttoordinaryBrownianmotion and the discretisation chosen. Arbitrage examples are established wherethecontinuousanalogueisarbitrage-freeundersmalltransactioncosts–includingfor the semimartingale modifications of fractional Brownian motion suggested intheseminalRogers(1997)articleprovingarbitrageinfBmmodels.Keywords: Stockpricemodel,randomwalk,Gaussianprocesses,weakconver-gence,freelunchwithvanishingrisk,arbitrage,transactioncostsMSC(2010): 60B10,60E05,60F05,91G10JELclassification: C61,D53,D81,G11 0 Introduction As well known since Rogers [7], fractional Brownian motions is a troublesome model foruncertainty in price processes, as fBm will introduce arbitrage opportunities to canonicalmodels where the ordinary Brownian motion does not. To remedy this, Rogers proposesa parametrised semimartingale modification, whose moving average kernel converges to thefBm’s – in particular, the no-arbitrage property is not preserved under this limit. For thepurposesofstudyingmarketvaluegains(orlosses),onecanhoweverarguethatthepointwise

Moment Asymptotics for Multitype Branching Random Walks in Random Environment

We study a discrete-time multitype branching random walk on a finite space with finite set of types. Particles move in space according to a Markov chain whereas offspring distributions are given by a random field that is fixed throughout the evolution of the particles. Our main interest lies in the averaged (annealed) expectation of the population size, and its long-time asymptotics. We first derive, for fixed time, a formula for the expected population size with fixed offspring distributions, which is reminiscent of a Feynman–Kac formula. We choose Weibull-type distributions with parameter \(1/\rho _{ij}\) for the upper tail of the mean number of \(j\) type particles produced by an \(i\) type particle. We derive the first two terms of the long-time asymptotics, which are written as two coupled variational formulas, and interpret them in terms of the typical behavior of the system.

Logconcave reward functions and optimal stopping rules of threshold form

In the literature, the problem of maximizing the expected discounted reward over all stopping rules has been explicitly solved for a number of reward functions (including (maxfx;0g) , > 0, in particular) when the underlying process is either a random walk in discrete time or a Levy process in continuous time. All of such reward functions are increasing and logconcave while the corresponding optimal stopping rules have the threshold form. In this paper, we explore the close connection between increasing and logconcave reward functions and optimal stopping rules of threshold form. In the discrete case, we show that if a reward function defined on Z is nonnegative, increasing and logconcave, then the optimal stopping rule is of threshold form provided the underlying random walk is skip-free to the right. In the continuous case, it is shown that for a reward function defined on R which is nonnegative, increasing, logconcave and right-continuous, the optimal stopping rule is of threshold form provided the underlying process is a spectrally negative Levy process. Furthermore, we also establish the necessity of logconcavity and monotonicity of a reward function in order for the optimal stopping rule to be of threshold form in the discrete (continuous, resp.) case when the underlying process belongs to the class of Bernoulli random walks (Brownian motions, resp.) with a downward drift. These results together provide a partial characterization of the threshold structure of optimal stopping rules.

On the dynamics of trap models in ℤ d

We consider trap models on Z^d, namely continuous time Markov jump process on Z^d with embedded chain given by a generic discrete time random walk, and whose mean waiting time at x is given by tau_x, with tau = (tau_x, x in Z^d) a family of positive iid random variables in the basin of attraction of an alpha-stable law, 0<alpha<1. We may think of x as a trap, and tau_x as the depth of the trap at x. We are interested in the trap process, namely the process that associates to time t the depth of the currently visited trap. Our first result is the convergence of the law of that process under suitable scaling. The limit process is given by the jumps of a certain alpha-stable subordinator at the inverse of another alpha-stable subordinator, correlated with the first subordinator. For that result, the requirements for the embedded random walk are a) the validity of a law of large numbers for its range, and b) the slow variation at infinity of the tail of the distribution of its time of return to the origin: they include all transient random walks as well as all planar random walks, and also many one dimensional random walks. We then derive aging results for the process, namely scaling limits for some two-time correlation functions thereof, a strong form of which requires an assumption of transience, stronger than a, b. The above mentioned scaling limit result is an averaged result with respect to tau. Under an additional condition on the size of the intersection of the ranges of two independent copies of the embeddded random walk, roughly saying that it is small compared with the size of the range, we derive a stronger scaling limit result, roughly stating that it holds in probability with respect to tau. With that additional condition, we also strengthen the aging results, from the averaged version mentioned above, to convergence in probability with respect to tau.

Comparing Algorithms for Graph Isomorphism Using Discrete- and Continuous-Time Quantum Random Walks

Berry and Wang [Phys. Rev. A {\bf 83}, 042317 (2011)] show numerically that a discrete-time quantum random walk of two noninteracting particles is able to distinguish some non-isomorphic strongly regular graphs from the same family. Here we analytically demonstrate how it is possible for these walks to distinguish such graphs, while continuous-time quantum walks of two noninteracting particles cannot. We show analytically and numerically that even single-particle discrete-time quantum random walks can distinguish some strongly regular graphs, though not as many as two-particle noninteracting discrete-time walks. Additionally, we demonstrate how, given the same quantum random walk, subtle differences in the graph certificate construction algorithm can nontrivially impact the walk's distinguishing power. We also show that no continuous-time walk of a fixed number of particles can distinguish all strongly regular graphs when used in conjunction with any of the graph certificates we consider. We extend this constraint to discrete-time walks of fixed numbers of noninteracting particles for one kind of graph certificate; it remains an open question as to whether or not this constraint applies to the other graph certificates we consider.

From Discrete Time Quantum Walk to Continuous Time Quantum Walk in Limit Distribution

The discrete time quantum walk defined as a quantum-mechanical analogue of the discrete time random walk have recently been attracted from various and interdisciplinary fields. In this review, the weak limit theorem, that is, the asymptotic behavior, of the one-dimensional discrete time quantum walk is analytically shown. From the limit distribution of the discrete time quantum walk, the discrete time quantum walk can be taken as the quantum dynamical simulator of some physical systems.

Coalescing Random Walks and Voting on Connected Graphs

In a coalescing random walk, a set of particles make independent discrete-time random walks on a graph. Whenever one or more particles meet at a vertex, they unite to form a single particle, which then continues a random walk through the graph. Let $G=(V,E)$ be an undirected and connected graph with $n$ vertices and $m$ edges. The coalescence time, $C(n)$, is the expected time for all particles to coalesce, when initially one particle is located at each vertex. We study the problem of bounding the coalescence time for general connected graphs and prove that $C(n) = O\big(\frac{1}{1-\lambda_2}\big(\log^{4} n + \frac{n}{\nu}\big)\big)$. Here $\lambda_2$ is the second eigenvalue of the transition matrix of the random walk. To avoid problems arising from, e.g., lack of coalescence on bipartite graphs, we assume the random walk can be made lazy if required. The value of $\nu$ is given by $\nu= \sum_{v\in V} d^2(v)/(d^2n)$, where $d(v)$ is the degree of vertex $v$, and $d=2m/n$ is the average degree. The parame...

Effect of network structure on phase transitions in queuing networks

Recently, De Martino et al. [J. Stat. Mech. (2009) P08023; Phys. Rev. E 79, 015101 (2009)] have presented a general framework for the study of transportation phenomena on random networks with annealed disorder. One of their most significant achievements was a deeper understanding of the phase transition from the uncongested to the congested phase at a critical traffic load on uncorrelated networks. In this paper, we also study phase transition in transportation networks using a discrete time random walk model. Our aim is to establish a direct connection between the structure of an uncorrelated random graph with quenched disorder and the value of the critical traffic load. We show that if the network is dense, the quenched and annealed formulas for the critical loading probability coincide. For sparse graphs, higher-order corrections, related to the local structure of the network, appear.

Diffusion of Tracer in Altered Tonalite: Experiments and Simulations with Heterogeneous Distribution of Porosity

Numerical time-domain-diffusion simulations were used for studying the diffusion behavior of tracer molecules in rock matrix with homogeneous and heterogeneous porosity. As the heterogeneous sample in these simulations, a 3D tomographic image of altered tonalite was used, in which the mineral components and the pores resolved by X-ray microtomography were represented by their respective intragranular porosities determined previously by the 14C-PMMA method. The apparent diffusion coefficient of a tracer in altered tonalite was determined experimentally, and was then used in the simulations. In the altered tonalite analyzed, inclusion of heterogeneity in the porosity increased the diffusion coefficient by 16 %. Altered and pristine feldspar was the main mineral component in the sample (72 %), and it also provided the dominant contribution to tracer diffusion, explaining alone 52 % of the diffusion coefficient. The large pores resolved by microtomography (6 %) and altered and pristine mica (22 %) gave an equal contribution to the diffusion coefficient. The simulation method applied was also validated by comparing the results to both an analytical and a numerical solution to the diffusion equation in a homogenous medium. In addition, the method was compared to discrete-time random-walk simulations in the case of randomly placed overlapping spheres.

Record statistics for multiple random walks

We study the statistics of the number of records R(n,N) for N identical and independent symmetric discrete-time random walks of n steps in one dimension, all starting at the origin at step 0. At each time step, each walker jumps by a random length drawn independently from a symmetric and continuous distribution. We consider two cases: (I) when the variance σ(2) of the jump distribution is finite and (II) when σ(2) is divergent as in the case of Lévy flights with index 0<μ<2. In both cases we find that the mean record number R(n,N) grows universally as ~α(N) sqrt[n] for large n, but with a very different behavior of the amplitude α(N) for N>1 in the two cases. We find that for large N, α(N) ≈ 2sqrt[lnN] independently of σ(2) in case I. In contrast, in case II, the amplitude approaches to an N-independent constant for large N, α(N) ≈ 4/sqrt[π], independently of 0<μ<2. For finite σ(2) we argue-and this is confirmed by our numerical simulations-that the full distribution of (R(n,N)/sqrt[n]-2sqrt[lnN])sqrt[lnN] converges to a Gumbel law as n → ∞ and N → ∞. In case II, our numerical simulations indicate that the distribution of R(n,N)/sqrt[n] converges, for n → ∞ and N → ∞, to a universal nontrivial distribution independently of μ. We discuss the applications of our results to the study of the record statistics of 366 daily stock prices from the Standard & Poor's 500 index.

Thermal collapse of snowflake fractals

Snowflakes are thermodynamically unstable structures that would ultimately become ice balls. To investigate their dynamics, we mapped atomistic molecular dynamics simulations of small ice crystals – built as filled von Koch fractals – onto a discrete-time random walk model. Then the walkers explored the thermal evolution of high fractal generations. The in silico experiments showed that the evolution is not entirely random. The flakes step down one fractal generation before forfeiting their architecture. The effect may be used to trace the thermal history of snow.

ON THE ENSEMBLE AVERAGE OF GENERALIZED TRUNCATED RANDOM WALK

A new analytical solution is developed for the one- and two- dimensional generalized discrete time random walk with discrete and finite length of jumping. It is shown that the ensemble average of the mean square displacement has a linear relation with time. This result is general and independent of mathematical form of distribution function but magnitude of diffusion coefficient depends on the distribution function. This result confirms our previous numerical work. The truncated Levy flight is studied as a special case and the diffusion coefficient for different maximum length of jump is calculated numerically. The obtained results are in good agreement with the analytical solution.