non-Markovian Processes Research Articles

A One-Dimensional Model for Investigating Scale-separated Approaches to the Interaction of Oceanic Internal Waves

Abstract High-frequency wave propagation in near-inertial wave shear has, for four decades, been considered fundamental in setting the spectral character of the oceanic internal wave continuum and for transporting energy to wave-breaking. We compare idealized ray tracing numerical results with metrics derived using a wave turbulence derivation for the kinetic equation and a path integral to study this specific process. Statistical metrics include the changing ensemble mean vertical wavenumber, referred to as a mean drift, dispersion about the mean drift, time lagged correlation estimates of wavenumber and phase locking of the wave packets with the background. The path integral permits us to identify the mean drift as a resonant process and dispersion about that mean drift as non-resonant. At small inertial wave amplitudes; ray tracing, wave turbulence and the path integral provide consistent descriptions for the mean drift of wavepackets in the spectral domain and dispersion about the mean drift. Extrapolating these results to the background internal wavefield over-predicts downscale energy transports by an order of magnitude. At oceanic amplitudes, however, the numerics support diminished transport and dispersion that coincide with the mean drift time scale becoming similar to the lagged correlation time scale. We parse this as the transition to a non-Markovian process. Despite this decrease, numerical estimates of downscale energy transfer are still too large. We argue that residual differences result from an unwarranted discard of Bragg scattering resonances. Our results support replacing the long standing interpretive paradigm of extreme scale-separated interactions with a more nuanced slate of ‘local’ interactions in the kinetic equation.

Capturing Long-Range Memory Structures with Tree-Geometry Process Tensors

We introduce a class of quantum non-Markovian processes—dubbed process trees—that exhibit polynomially decaying temporal correlations and memory distributed across timescales. This class of processes is described by a tensor network with treelike geometry whose component tensors are (1) causality-preserving maps (superprocesses) and (2) locality-preserving temporal change-of-scale transformations. We show that the long-range correlations in this class of processes tends to originate almost entirely from memory effects and can accommodate genuinely quantum power-law correlations in time. Importantly, this class allows efficient computation of multitime correlation functions. To showcase the potential utility of this model-agnostic class for numerical simulation of physical models, we show how it can efficiently approximate the strong memory dynamics of the paradigmatic spin-boson model, in terms of arbitrary multitime features. In contrast to an equivalently costly matrix-product-operator representation, the ansatz produces a fiducial characterization of the relevant physics. Finally, leveraging 2D tensor-network renormalization-group methods, we detail an algorithm for deriving a process tree from an underlying Hamiltonian via the Feynmann-Vernon influence functional. Our work lays the foundation for the development of more efficient numerical techniques in the field of strongly interacting open quantum systems, as well as the theoretical development of a temporal renormalization-group scheme. Published by the American Physical Society 2024

Approximation identification for the stochastic time-delayed dynamical system

This paper addresses the challenges of analyzing stochastic dynamical systems with a single time-delay within a data-driven framework. The presence of time-delay introduces non-Markovian characteristics to the system, complicating the analysis of its dynamic behavior using traditional approaches. Drawing inspiration from the small delay approximation, we apply a sparse identification technique to approximate the non-Markovian process with a Markovian one. This innovative method circumvents limitations associated with the system's dimensionality and the complexity of delayed diffusion terms, offering a versatile tool for investigating the dynamics of stochastic time-delayed systems. Our approach begins by establishing a connection between the system's coefficients and simulated data using the Kramers-Moyal formula, which captures the essential statistical properties of the system. We then leverage sparse identification to extract a concise model of the stochastic dynamical system, effectively eliminating the time-delay from consideration. The practicality and efficacy of our method are substantiated through a series of illustrative examples that showcase its application and validate its performance. By introducing this method, we aim to provide a novel analytical framework for stochastic time-delayed systems, advancing the current capabilities for modeling and understanding such complex dynamics.

Infinite-Horizon Probability of Ruin for a Variable-Memory Counting Process (Hawkes Process)

The reserve R(t) of an insurance company denotes the accumulated capital up to time ttt throughout its operational span. While this reserve can be modeled using stochastic processes, the endeavor remains notably intricate. The mathematical complexity presents a considerable obstacle for researchers within this domain. Prior research has predominantly focused on reserve models constructed from Markovian processes. In this manuscript, our attention shifts to risk models derived from non-Markovian processes, particularly those with claim arrivals governed by Hawkes processes. Before delving into the core subject matter, we provide an extensive review of the literature on counting processes with variable memory. This includes a thorough exploration of Hawkes processes, Gerber-Shiu functions, integro-differential equations, and Laplace transforms. These elements are crucial for computing the probability of ruin over an infinite time horizon, especially in scenarios where interdependence exists between inter-claim times. Additionally, we present the requisite mathematical tools essential for comprehending the contents of this article, specifically tailored for individuals with a foundational understanding of actuarial science. Furthermore, we have successfully determined the probability of ruin, marking a significant milestone in our investigation.

Long-term memory induced correction to Arrhenius law

The Kramers escape problem is a paradigmatic model for the kinetics of rare events, which are usually characterized by Arrhenius law. So far, analytical approaches have failed to capture the kinetics of rare events in the important case of non-Markovian processes with long-term memory, as occurs in the context of reactions involving proteins, long polymers, or strongly viscoelastic fluids. Here, based on a minimal model of non-Markovian Gaussian process with long-term memory, we determine quantitatively the mean FPT to a rare configuration and provide its asymptotics in the limit of a large energy barrier E. Our analysis unveils a correction to Arrhenius law, induced by long-term memory, which we determine analytically. This correction, which we show can be quantitatively significant, takes the form of a second effective energy barrier E′<E\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$E^{\\prime} \\, < \\, E$$\\end{document} and captures the dependence of rare event kinetics on initial conditions, which is a hallmark of long-term memory. Altogether, our results quantify the impact of long-term memory on rare event kinetics, beyond Arrhenius law.

Stationary Fluctuations of Run-and-Tumble Particles

We study the stationary fluctuations of independent run-and-tumble particles. We prove that the joint densities of particles with given internal state converges to an infinite dimensional Ornstein-Uhlenbeck process. We also consider an interacting case, where the particles are subjected to exclusion. We then study the fluctuations of the total density, which is a non-Markovian Gaussian process, and obtain its covariance in closed form. By considering small noise limits of this non-Markovian Gaussian process, we obtain in a concrete example a large deviation rate function containing memory terms.

Numerical analysis and calculation of fractional neuron transmission model

Abstract It is well known that the transmission of nerve signals is closely related to the incidence and production of vascular diseases. However, due to the multilayered and complex structure of the vascular wall, although a series of studies have been carried out, the mechanism of vascular diseases is difficult to fully understand. In this paper, the fractional constitutive equation is used to study numerically the transmission of nerve signals in the vascular wall. On the basis of considering the effect of substance transport in the vascular layer, a new comprehensive model was established to describe and analyze the voltage trajectory of neuron activity using multi-time-scale dynamics induced by correlated membranes combined with non-Markov processes. The results show that our fractional order model can better describe the process of neurotransmission. By changing the fractional order, we can show in vitro the up-and-down spike adaptation found in the experiments of neocortical pyramidal cells and parietal neurons.

"Non-Markov Processes and Analytical Expressions of Markov Chains of Biomaterials"

"Non-Markov Processes and Analytical Expressions of Markov Chains of Biomaterials"

Asymptotic optimality of a class of controlled non-Markov processes

Motivated by applications in power systems and problems arising in simulation of large scale complex system optimizations, this work is concerned with controlled stochastic switching systems. The system of interest displays a multi-time scale structure. In contrast to the so-called singularly perturbed diffusions and multi-scale Markov decision processes, controlled non-Markov processes (also known as non-Markov decision processes) are treated. The novelty of our work is the treatment of the non-Markov controlled processes and the time-scale used. The fast and slow processes are coupled through a stochastic differential equation. Using averaging, it is first shown that the non-Markov switching process has a weak limit that is a Markov decision process. Then asymptotic optimal control of the non-Markov process is obtained by using the limit process.

On positively divisible non-Markovian processes

There are some positively divisible non-Markovian processes whose transition matrices satisfy the Chapman–Kolmogorov equation. These processes should also satisfy the Kolmogorov consistency conditions, an essential requirement for a process to be classified as a stochastic process. Combining the Kolmogorov consistency conditions with the Chapman–Kolmogorov equation, we derive a necessary condition for positively divisible stochastic processes on a finite sample space. This necessary condition enables a systematic approach to the manipulation of certain Markov processes in order to obtain a positively divisible non-Markovian process. We illustrate this idea by an example and, in addition, analyze a classic example given by Feller in the light of our approach.

Standard form of master equations for general non-Markovian jump processes: The Laplace-space embedding framework and asymptotic solution

We present a standard form of master equations (MEs) for general one-dimensional non-Markovian (history-dependent) jump processes, complemented by an asymptotic solution derived from an expanded system-size approach. The ME is obtained by developing a general Markovian embedding using a suitable set of auxiliary field variables. This Markovian embedding uses a Laplace-convolution operation applied to the velocity trajectory. We introduce an asymptotic method tailored for this ME standard, generalizing the system-size expansion for these jump processes. Under specific stability conditions tied to a single noise source, upon coarse graining, the generalized Langevin equation (GLE) emerges as a universal approximate model for point processes in the weak-coupling limit. This methodology offers a unified analytical tool set for general non-Markovian processes, reinforcing the universal applicability of the GLE founded in microdynamics and the principles of statistical physics. Published by the American Physical Society 2024

Statistics of quantum heat in the Caldeira-Leggett model.

Nonequilibrium fluctuation relation lies at the heart of the quantum thermodynamics. Many previous studies have demonstrated that the heat exchange between a quantum system and a thermal bath initially prepared in their own Gibbs states at different temperatures obeys the famous Jarzynski-Wójcik fluctuation theorem. However, this conclusion is obtained under the assumption of Born-Markovian approximation. In this paper, going beyond the Born-Markovian limitation, we investigate the statistics of quantum heat in an exactly non-Markovian relaxation process described by the well-known Caldeira-Leggett model. It is revealed that the Jarzynski-Wójcik fluctuation theorem breaks down in the strongly non-Markovian regime. Moreover, we find the steady-state quantum heat within the non-Markovian framework can be widely tunable by using the quantum reservoir-engineering technique. These results are sharply contrary to the common Born-Markovian predictions. Our results presented in this paper may update the understanding of the quantum thermodynamics in strongly coupled and low-temperature systems. Moreover, the controllable heat may have some potential applications in improving the performance of a quantum heat engine.

Assessing dengue risk globally using non-Markovian models

Dengue is a vector-borne disease transmitted by Aedes mosquitoes. The worldwide spread of these mosquitoes and the increasing disease burden have emphasized the need for a spatio-temporal risk map capable of assessing dengue outbreak conditions and quantifying the outbreak risk. Given that the life cycle of Aedes mosquitoes is strongly influenced by habitat temperature, numerous studies have utilized temperature-dependent development rates of these mosquitoes to construct virus transmission and outbreak risk models. In this study, we contribute to existing research by developing a mechanistic model for the mosquito life cycle that accurately captures its non-Markovian nature. Beginning with integral equations to track the mosquito population across different life cycle stages, we demonstrate how to derive the corresponding differential equations using phase-type distributions. This approach can be further applied to similar non-Markovian processes that are currently described with less accurate Markovian models. By fitting the model to data on human dengue cases, we estimate several model parameters, allowing the development of a global spatiotemporal dengue risk map. This risk model employs temperature and precipitation data to assess the environmental suitability for dengue outbreaks in a given area.

Confluence of Random Walks, Interacting Particle Systems, and Distributed Machine Learning: Federated Learning through Crawling over Networks

In this work, we aim to unveil a new class of intermediate FL architectures between centralized and decentralized schemes called “FedCrawl.” FedCrawl takes advantage of benefits of D2D communications similar to decentralized schemes; however, it uses them in a nuanced way. FedCrawl is inspired by web crawlers, which effectively explore the websites to find updated/new content posted on the internet. The cornerstone of FedCrawl is its innovative conceptualization of neural networks (NNs) or other used ML models as autonomous entities, called random walkers, with the capability to move or jump across nodes in the network through peer-to-peer (P2P) or device-to-device (D2D) connections. We introduce five research aspects to study the nuanced intricacies governing random walker behavior in these environments. The first research aspect addresses the interplay between network topology and data distribution, emphasizing the importance of considering both factors for designing efficient random walks in FedCrawl. The second research aspect explores the applicability of node importance metrics in optimizing random walker paths for FedCrawl. We propose a dynamic perception-aware design, discussed in the third research aspect, where transition matrices adapt to the evolving state of random walkers, balancing exploration and exploitation. The fourth research aspect introduces innovative features like skipping, memory look-back, and caching/trailing to enhance random walker performance. Lastly, the fifth research aspect delves into the dynamics of multiple random walkers in networked environments, introducing the concept of multi-pole random walkers. Complementing these five research aspects, we present five conjectures, each introducing novel perspectives and methodologies in the domain of decentralized learning. These conjectures encompass areas such as temperature-based characterization of random walkers and network nodes, dynamic transition matrices, non-Markovian processes, and an evolutionary framework for random walker patterns.

A unifying representation of path integrals for fractional Brownian motions

Fractional Brownian motion (fBm) is an experimentally-relevant, non-Markovian Gaussian stochastic process with long-ranged correlations between the increments, parametrised by the so-called Hurst exponent H; depending on its value the process can be sub-diffusive (0<H<1/2) , diffusive (H=1/2) or super-diffusive (1/2<H<1) . There exist three alternative equally often used definitions of fBm—due to Lévy and due to Mandelbrot and van Ness (MvN), which differ by the interval on which the time variable is formally defined. Respectively, the covariance functions of these fBms have different functional forms. Moreover, the MvN fBms have stationary increments, while for the Lévy fBm this is not the case. One may therefore be tempted to conclude that these are, in fact, different processes which only accidentally bear the same name. Recently determined explicit path integral representations also appear to have very different functional forms, which only reinforces the latter conclusion. Here we develop a unifying equivalent path integral representation of all three fBms in terms of Riemann–Liouville fractional integrals, which links the fBms and proves that they indeed belong to the same family. We show that the action in such a representation involves the fractional integral of the same form and order (dependent on whether H<1/2 or H>1/2 ) for all three cases, and differs only by the integration limits.

Active particle in one dimension subjected to resetting with memory.

The study of diffusion with preferential returns to places visited in the past has attracted increased attention in recent years. In these highly non-Markov processes, a standard diffusive particle intermittently resets at a given rate to previously visited positions. At each reset, a position to be revisited is randomly chosen with a probability proportional to the accumulated amount of time spent by the particle at that position. These preferential revisits typically generate a very slow diffusion, logarithmic in time, but still with a Gaussian position distribution at late times. Here we consider an active version of this model, where between resets the particle is self-propelled with constant speed and switches direction in one dimension according to a telegraphic noise. Hence there are two sources of non-Markovianity in the problem. We exactly derive the position distribution in Fourier space, as well as the variance of the position at all times. The crossover from the short-time ballistic regime, dominated by activity, to the long-time anomalous logarithmic growth induced by memory is studied. We also analytically derive a large deviation principle for the position, which exhibits a logarithmic time scaling instead of the usual algebraic form. Interestingly, at large distances, the large deviations become independent of time and match the nonequilibrium steady state of a particle under resetting to its starting position only.

Machine Learning Memory Kernels as Closure for Non-Markovian Stochastic Processes.

Finding the dynamical law of observable quantities lies at the core of physics. Within the particular field of statistical mechanics, the generalized Langevin equation (GLE) comprises a general model for the evolution of observables covering a great deal of physical systems with many degrees of freedom and an inherently stochastic nature. Although formally exact, GLE brings its own great challenges. It depends on the complete history of the observables under scrutiny, as well as the microscopic degrees of freedom, all of which are often inaccessible. We show that these drawbacks can be overcome by adopting elements of machine learning from empirical data, in particular coupling a multilayer perceptron (MLP) with the formal structure of GLE and calibrating the MLP with the data. This yields a powerful computational tool capable of describing noisy complex systems beyond the realms of statistical mechanics. It is exemplified with a number of representative examples from different fields: from a single colloidal particle and particle chains in a thermal bath to climatology and finance, showing in all cases excellent agreement with the actual observable dynamics. The new framework offers an alternative perspective for the study of nonequilibrium processes opening also a new route for stochastic modeling.

Green Measures for a Class of Non-Markov Processes

In this paper, we investigate the Green measure for a class of non-Gaussian processes in Rd. These measures are associated with the family of generalized grey Brownian motions Bβ,α, 0&lt;β≤1, 0&lt;α≤2. This family includes both fractional Brownian motion, Brownian motion, and other non-Gaussian processes. We show that the perpetual integral exists with probability 1 for dα&gt;2 and 1&lt;α≤2. The Green measure then generalizes those measures of all these classes.

Tailoring nondiffracting fields with a non-Markovian phase imprint.

We experimentally generate nondiffracting speckles that carry non-Markovian properties by encoding the wavefront of a monochromatic laser beam with ring-shaped non-Markovian phases. The resulting non-Markovian nondiffracting fields present a ring-shaped pattern and central dark notches, which are analyzed with an expression of the orbital angular momentum spectra of the wavefront possessing ring-shaped non-Markovian phases. Furthermore, we demonstrate that the intensity profiles of these non-Markovian nondiffracting fields exhibit stability over multiple Rayleigh ranges, and their statistical properties could be controlled with the non-Markovianity of the input phase masks. This work presents an approach for simultaneously tailoring the diffracting property and non-Markovianity of optical fields and provides a deeper understanding of non-Markovian processes.

Omega-Regular Decision Processes

Regular decision processes (RDPs) are a subclass of non-Markovian decision processes where the transition and reward functions are guarded by some regular property of the past (a lookback). While RDPs enable intuitive and succinct representation of non-Markovian decision processes, their expressive power coincides with finite-state Markov decision processes (MDPs). We introduce omega-regular decision processes (ODPs) where the non-Markovian aspect of the transition and reward functions are extended to an omega-regular lookahead over the system evolution. Semantically, these lookaheads can be considered as promises made by the decision maker or the learning agent about her future behavior. In particular, we assume that, if the promised lookaheads are not met, then the payoff to the decision maker is falsum (least desirable payoff), overriding any rewards collected by the decision maker. We enable optimization and learning for ODPs under the discounted-reward objective by reducing them to lexicographic optimization and learning over finite MDPs. We present experimental results demonstrating the effectiveness of the proposed reduction.