Nonlinear Markov Processes Research Articles

Bivariate Jacobi polynomials depending on four parameters and their effect on solutions of time-fractional Burgers’ equations

The utilization of time-fractional Burgers’ equations is widespread, employed in modeling various phenomena such as heat conduction, acoustic wave propagation, gas turbulence, and the propagation of chaos in non-linear Markov processes. This study introduces a novel pseudo-operational collocation method, leveraging two-variable Jacobi polynomials. These polynomials are obtained through the Kronecker product of their one-variable counterparts, concerning both spatial (x) and temporal (t) domains. The study explores the impact of four parameters (θ,ϑ,σ,ς>−1) on the accuracy of resulting approximate solutions, marking the first examination of such influence. Collocation nodes in a tensor approach are constructed employing the roots of one-variable Jacobi polynomials of varying degrees in x and t. The study delves into analyzing how the distribution of these roots affects the outcomes. Consequently, pseudo-operational matrices are devised to integrate both integer and fractional orders, presenting a novel methodological advancement. By employing these matrices and appropriate approximations, the governing equations transform into an algebraic system, facilitating computational analysis. Furthermore, the existence and uniqueness of the equations under study are investigated and the study estimates error bounds within a Jacobi-weighted space for the obtained approximate solutions. Numerical simulations underscore the simplicity, applicability, and efficiency of the proposed matrix spectral scheme.

Lyapunov function for interacting reinforced stochastic processes via Hopfield’s energy function

In 1984, Hopfield introduced an artificial neural network to understand the memory in living organisms. Under a condition of symmetry, he showed that there exists a Lyapunov function (known as energy function) for the network. In 2015, Budhiraja, Dupuis and Fischer introduced an idea to construct Lyapunov functions for nonlinear Markov processes via relative entropy. In this article, we introduce an approach based on the energy function of Hopfield networks to obtain Lyapunov functions for a class of interacting reinforced stochastic processes. Our result is an alternative to Budhiraja, Dupuis and Fischer’s approach and it works for the case of processes with finitely many 2-dimensional probability measures. We also bring from the neural network framework results about the total stability of differential equations. Finally, we include an application of the results to the study of differential equations associated to n≥2 vertex reinforced random walks on two-vertex graphs.

Non-reversible processes: GENERIC, hypocoercivity and fluctuations

We consider two approaches to study non-reversible Markov processes, namely the hypocoercivity theory and general equations for non-equilibrium reversible–irreversible coupling; the basic idea behind both of them is to split the process into a reversible component and a non-reversible one, and then quantify the way in which they interact. We compare such theories and provide explicit formulas to pass from one formulation to the other; as a bi-product we give a simple proof of the link between reversibility of the dynamics and gradient flow structure of the associated Fokker–Planck equation. We do this both for linear Markov processes and for a class of nonlinear Markov process as well. We then characterise the structure of the large deviation functional of generalised-reversible processes; this is a class of non-reversible processes of large relevance in applications. Finally, we show how our results apply to two classes of Markov processes, namely non-reversible diffusion processes and a class of piecewise deterministic Markov processes (PDMPs), which have recently attracted the attention of the statistical sampling community. In particular, for the PDMPs we consider we prove entropy decay.

Resetting photons

Starting from a frequency diffusion process for a tagged photon which simulates relaxation to the Planck law, we introduce a resetting where photons lower their frequency at random times. We consider two versions, one where the resetting to low frequency is independent of the existing frequency and a second case where the reduction in frequency scales with the original frequency. The result is a nonlinear Markov process where the stationary distribution modifies the Planck law by abundance of low-frequency occupation. The physical relevance of such photon resetting processes can be found in explorations of nonequilibrium effects, e.g., via random expansions of a confined plasma or photon gas or via strongly inelastic scattering with matter.

Nonlinear Markov chains with finite state space: invariant distributions and long-term behaviour

AbstractNonlinear Markov chains with finite state space were introduced by Kolokoltsov (Nonlinear Markov Processes and Kinetic Equations, 2010). The characteristic property of these processes is that the transition probabilities depend not only on the state, but also on the distribution of the process. Here we provide first results regarding their invariant distributions and long-term behaviour: we show that under a continuity assumption an invariant distribution exists and provide a sufficient criterion for the uniqueness of the invariant distribution. Moreover, we present examples of peculiar limit behaviour that cannot occur for classical linear Markov chains. Finally, we present for the case of small state spaces sufficient (and easy-to-verify) criteria for the ergodicity of the process.

Photon Frequency Diffusion Process

We introduce a stochastic multi-photon dynamics on reciprocal space. Assuming isotropy, we derive the diffusion limit for a tagged photon to be a nonlinear Markov process on frequency. The nonlinearity stems from the stimulated emission. In the case of Compton scattering with thermal electrons, the limiting process describes the dynamical fluctuations around the Kompaneets equation. More generally, we construct a photon frequency diffusion process which enables to include nonequilibrium effects. Modifications of the Planck Law may thus be explored, where we focus on the low-frequency regime.

Household epidemic models and McKean–Vlasov Poisson driven stochastic differential equations

This paper presents a new view of household epidemic models, where the interaction between the households is of mean field type. We thus obtain in the limit of infinitely many households a nonlinear Markov process solution of a McKean - Vlasov type Poisson driven SDE, and a propagation of chaos result. We also define a basic reproduction number R0, and show that if R0>1, then the nonlinear Markov process has a unique non trivial ergodic invariant probability measure, whereas if R0<=1, it converges to 0 as t tends to infinity.

Effective Hamiltonians and Lagrangians for Conditioned Markov Processes at Large Volume

When analysing statistical systems or stochastic processes, it is often interesting to ask how they behave given that some observable takes some prescribed value. This conditioning problem is well understood within the linear operator formalism based on rate matrices or Fokker-Planck operators, which describes the dynamics of many independent random walkers. Relying on certain spectral properties of the biased linear operators, guaranteed by the Perron-Frobenius theorem, an effective process can be found such that its path probability is equivalent to the conditional path probability. In this paper, we extend those results for nonlinear Markov processes that appear when the many random walkers are no longer independent, and which can be described naturally through a Lagrangian-Hamiltonian formalism within the theory of large deviations at large volume. We identify the appropriate spectral problem as being a Hamilton-Jacobi equation for a biased Hamiltonian, for which we conjecture that two special global solutions exist, replacing the Perron-Frobenius theorem concerning the positivity of the dominant eigenvector. We then devise a rectification procedure based on a canonical gauge transformation of the biased Hamiltonian, yielding an effective dynamics in agreement with the original conditioning. Along the way, we present simple examples in support of our conjecture, we examine its consequences on important physical objects such as the fluctuation symmetries of the biased and rectified processes as well as the dual dynamics obtained through time-reversal. We apply all those results to simple independent and interacting models, including a stochastic chemical reaction network and a population process called the Brownian Donkey.

Abstract McKean–Vlasov and Hamilton–Jacobi–Bellman Equations, Their Fractional Versions and Related Forward–Backward Systems on Riemannian Manifolds

We introduce a class of abstract nonlinear fractional pseudo-differential equations in Banach spaces that includes both the Mc-Kean-Vlasov-type equations describing nonlinear Markov processes and the Hamilton-Jacobi-Bellman(HJB)-Isaacs equation of stochastic control and games thus allowing for a unified analysis of these equations. This leads to an effective theory of coupled forward-backward systems (forward McKean-Vlasov evolution and backward HJB-Isaacs evolution) that are central to the modern theory of mean-field games.

Probabilistic properties and parametric inference of small variance nonlinear self-stabilizing stochastic differential equations

We consider a process (Xt) solution of a one-dimensional nonlinear self-stabilizing stochastic differential equation, with classical drift term V(α,x) depending on an unknown parameter α, self-stabilizing term Φ(β,x) depending on another unknown parameter β and small noise amplitude ɛ. Self-Stabilization is the effect of including a mean-field interaction in addition to the state-dependent drift. Adding this term leads to a nonlinear or McKean-Vlasov Markov process with transitions depending on the distribution of Xt. We study the probabilistic properties of (Xt) as ɛ tends to 0 and exhibit a Gaussian approximating process for (Xt). Next, we study the estimation of (α,β) from a continuous observation of (Xt,t∈[0,T]). We build explicit estimators using an approximate log-likelihood function obtained from the exact log-likelihood function of a proxi-model. We prove that, for fixed T, as ɛ tends to 0, α can be consistently estimated with rate ɛ−1but notβ. Then, considering ni.i.d. sample paths (Xti,i=1,…,n), we build consistent and asymptotically Gaussian estimators of (α,β) with rates nɛ−1 for α and n for β. Finally, we prove that the statistical experiments generated by (Xt) and the proxi-model are asymptotically equivalent in the sense of the Le Cam Δ-distance both for the continuous observation of one path and for ni.i.d. paths under the condition nɛ→0, which justifies our statistical method.

Empirical Measure and Small Noise Asymptotics Under Large Deviation Scaling for Interacting Diffusions

Consider a collection of particles whose state evolution is described through a system of interacting diffusions in which each particle is driven by an independent individual source of noise and also by a small amount of noise that is common to all particles. The interaction between the particles is due to the common noise and also through the drift and diffusion coefficients that depend on the state empirical measure. We study large deviation behavior of the empirical measure process which is governed by two types of scaling, one corresponding to mean field asymptotics and the other to the Freidlin–Wentzell small noise asymptotics. Different levels of intensity of the small common noise lead to different types of large deviation behavior, and we provide a precise characterization of the various regimes. The rate functions can be interpreted as the value functions of certain stochastic control problems in which there are two types of controls; one of the controls is random and nonanticipative and arises from the aggregated contributions of the individual Brownian noises, whereas the second control is nonrandom and corresponds to the small common Brownian noise that impacts all particles. We also study large deviation behavior of interacting particle systems approximating various types of Feynman–Kac functionals. Proofs are based on stochastic control representations for exponential functionals of Brownian motions and on uniqueness results for weak solutions of stochastic differential equations associated with controlled nonlinear Markov processes

Stabilization of Multi-Agent Systems to Target Distributions using Local Interactions

In this paper, we construct a mean-field discrete-time Markov process evolving on a compact subset of ℝ d can be stabilized to an arbitrary target distribution that has a continuous density. This density, unlike in our previous works, need not have a connected support on the state space. Our main application of interest is characterizing the distribution of a multi-agent system that evolves according to a discrete-time Markov process. Even if the Markov process converges to an equilibrium distribution, the agents may continue to switch between states, potentially wasting energy. In order to prevent this unnecessary switching, we show that the Markov process can be constructed in such a way that the operator that pushes forward measures is the identity operator at the target measure. The challenge in the stability analysis of the system arises from the fact that the transition kernel is a function of the current distribution, resulting in a nonlinear Markov process. Moreover, we aim to design the transition kernel, which is the feedback control law for the Markov process, to be decentralized in the sense that it depends on the local density of agents. We prove by construction that there exists a control law that is decentralized and globally stabilizes the desired measure. In order to implement this control law, the individual agents must estimate the local population density. We validate our control law with numerical simulations of multi-agent systems with different population sizes. We observe that the number of agent state transitions at equilibrium significantly decreases as the population size increases.

The dynamics of entropy in the COVID-19 outbreaks

With the unfolding of the COVID-19 pandemic, mathematical modelling of epidemics has been perceived and used as a central element in understanding, predicting, and governing the pandemic event. However, soon it became clear that long-term predictions were extremely challenging to address. In addition, it is still unclear which metric shall be used for a global description of the evolution of the outbreaks. Yet a robust modelling of pandemic dynamics and a consistent choice of the transmission metric is crucial for an in-depth understanding of the macroscopic phenomenology and better-informed mitigation strategies. In this study, we propose a Markovian stochastic framework designed for describing the evolution of entropy during the COVID-19 pandemic together with the instantaneous reproductive ratio. Then, we introduce and use entropy-based metrics of global transmission to measure the impact and the temporal evolution of a pandemic event. In the formulation of the model, the temporal evolution of the outbreak is modelled by an equation governing the probability distribution that describes a nonlinear Markov process of a statistically averaged individual, leading to a clear physical interpretation. The time-dependent parameters are formulated by adaptive basis functions, leading to a parsimonious representation. In addition, we provide a full Bayesian inversion scheme for calibration together with a coherent strategy to address data unreliability. The time evolution of the entropy rate, the absolute change in the system entropy, and the instantaneous reproductive ratio are natural and transparent outputs of this framework. The framework has the appealing property of being applicable to any compartmental epidemic model. As an illustration, we apply the proposed approach to a simple modification of the susceptible–exposed–infected–removed model. Applying the model to the Hubei region, South Korean, Italian, Spanish, German, and French COVID-19 datasets, we discover significant difference in the absolute change of entropy but highly regular trends for both the entropy evolution and the instantaneous reproductive ratio.

Optimal Stopping of McKean--Vlasov Diffusions via Regression on Particle Systems

In this paper we study optimal stopping problems for nonlinear Markov processes driven by a McKean--Vlasov SDE and aim at solving them numerically by Monte Carlo. To this end we propose a novel reg...

Insensitivity of the mean field limit of loss systems under SQ(d) routeing

AbstractIn this paper, we study a large multi-server loss model under the SQ(d) routeing scheme when the service time distributions are general with finite mean. Previous works have addressed the exponential service time case when the number of servers goes to infinity, giving rise to a mean field model. The fixed point of the limiting mean field equations (MFEs) was seen to be insensitive to the service time distribution in simulations, but no proof was available. While insensitivity is well known for loss systems, the models, even with state-dependent inputs, belong to the class of linear Markov models. In the context of SQ(d) routeing, the resulting model belongs to the class of nonlinear Markov processes (processes whose generator itself depends on the distribution) for which traditional arguments do not directly apply. Showing insensitivity to the general service time distributions has thus remained an open problem. Obtaining the MFEs in this case poses a challenge due to the resulting Markov description of the system being in positive orthant as opposed to a finite chain in the exponential case. In this paper, we first obtain the MFEs and then show that the MFEs have a unique fixed point that coincides with the fixed point in the exponential case, thus establishing insensitivity. The approach is via a measure-valued Markov process representation and the martingale problem to establish the mean field limit.

Mean-field Macro Computation in Large-scale Cloud Service Systems with Resource Management and Job Scheduling

Service computing is an emerging and distributed computing mode in cloud service systems, and has become an interesting research direction for both academia and industry. Note that the cloud service systems always display new characteristics, such as stochasticity, large scale, loose coupling, concurrency, non-homogeneity and heterogeneity, thus their load balancing investigation has been more interesting, difficult and challenging until now. By using resource management and job scheduling, this paper proposes an integrated, real-time and dynamic control mechanism for large-scale cloud service systems and their load balancing through combining supermarket models with not only work stealing models but also scheduling of public reserved resource. To this end, this paper provides a novel stochastic model with weak interactions by means of nonlinear Markov processes. To overcome theoretical difficulties growing out of the state explosion in high-dimensional stochastic systems, this paper applies the mean-field theory to develop a macro computational technique in terms of an infinite-dimensional system of mean-field equations. Furthermore, this paper proves the asymptotic independence of the large-scale cloud service system, and show how to compute the fixed point by virtue of an infinite-dimensional system of nonlinear equations. Based on the fixed point, this paper provides effective numerical computation for performance analysis of this system under a high approximate precision. Therefore, we hope that the methodology and results given in this paper can be applicable to the study of more general large-scale cloud service systems.

Metastability of Queuing Networks with Mobile Servers

We study symmetric queuing networks with moving servers and FIFO service discipline. The mean-field limit dynamics demonstrates unexpected behavior which we attribute to the meta-stability phenomenon. Large enough finite symmetric networks on regular graphs are proved to be transient for arbitrarily small inflow rates. However, the limiting non-linear Markov process possesses at least two stationary solutions. The proof of transience is based on martingale techniques.

Deterministic Limit of Mean Field Games Associated with Nonlinear Markov Processes

The paper is concerned with the deterministic limit of mean field games with a nonlocal coupling. It is assumed that the dynamics of mean field games are given by nonlinear Markov processes. This type of games includes stochastic mean field games as well as mean field games with finite state space. We consider the limiting deterministic mean field game within the framework of minimax approach. The concept of minimax solutions is close to the probabilistic formulation. In this case the Hamilton–Jacobi equation is considered in the minimax/viscosity sense, whereas the flow of probabilities is determined by the probability on the set of solutions of the differential inclusion associated with the Hamilton–Jacobi equation such that those solutions are viable in the graph of the minimax solution. The main result of the paper is the convergence (up to subsequence) of the solutions of the mean field games to the minimax solution of a deterministic mean field game in the case when the underlying dynamics converge to the deterministic evolution.

Note on Surjective Polynomial Operators

A linear Markov chain is a discrete time stochastic process whose transitions depend only on the current state of the process. A nonlinear Markov chain is a discrete time stochastic process whose transitions may depend on both the current state and the current distribution of the process. These processes arise naturally in the study of the limit behavior of a large number of weakly interacting Markov processes. The nonlinear Markov processes were introduced by McKean and have been extensively studied in the context of nonlinear Chapman-Kolmogorov equations as well as nonlinear Fokker-Planck equations. The nonlinear Markov chain over a finite state space can be identified by a continuous mapping (a nonlinear Markov operator) defined on a set of all probability distributions (which is a simplex) of the finite state space and by a family of transition matrices depending on occupation probability distributions of states. Particularly, a linear Markov operator is a linear operator associated with a square stochastic matrix. It is well-known that a linear Markov operator is a surjection of the simplex if and only if it is a bijection. The similar problem was open for a nonlinear Markov operator associated with a stochastic hyper-matrix. We solve it in this paper. Namely, we show that a nonlinear Markov operator associated with a stochastic hyper-matrix is a surjection of the simplex if and only if it is a permutation of the Lotka-Volterra operator.

Restoration of Static JPEG Images and RGB Video Frames by Means of Nonlinear Filtering in Conditions of Gaussian and Non‐Gaussian Noise

AbstractThe use of nonlinear Markov process filtering makes it possible to restore both video stream frames and static photos at the stage of preprocessing. The present paper reflects the results of research in comparison of these types image filtering quality by means of special algorithm when Gaussian or non‐Gaussian noises acting. Examples of filter operation at different values of signal‐to‐noise ratio are presented. A comparative analysis has been performed, and the best filtered kind of noise has been defined. It has been shown the quality of developed algorithm is much better than quality of adaptive one for RGB signal filtering at the same a priori information about the signal. Also, an advantage over median filter takes a place when both fluctuation and pulse noise filtering.