non-Markovian Model Research Articles

Asymptotic solutions to nonlinear Hawkes processes: A systematic classification of the steady-state solutions

Hawkes point processes are first-order non-Markovian stochastic models of intermittent bursty dynamics with applications to physical, seismic, epidemic, biological, financial, and social systems. While accounting for positive feedback loops that may lead to critical phenomena in complex systems, the standard linear Hawkes process only describes excitative phenomena. To describe the co-existence of excitatory and inhibitory effects (or negative feedbacks), extensions involving nonlinear dependences of the intensity as a function of past activity are needed. However, such nonlinear Hawkes processes have been found hitherto to be analytically intractable due to the interplay between their non-Markovian and nonlinear characteristics, with no analytical solutions available. Here, we present various exact and robust asymptotic solutions to nonlinear Hawkes processes using the field master equation approach. We report explicit power law formulas for the steady state intensity distributions $P_{\mathrm{ss}}(\lambda)\propto \lambda^{-1-a}$, where the tail exponent $a$ is expressed analytically as a function of parameters of the nonlinear Hawkes models. We present three robust interesting characteristics of the nonlinear Hawkes process: (i) for one-sided positive marks (i.e., in the absence of inhibitory effects), the nonlinear Hawkes process can exhibit any power law relation either as intermediate asymptotics ($a\leq 0$) or as true asymptotics ($a>0$) by appropriate model selection; (ii) for distribution of marks with zero mean (i.e., for balanced excitatory and inhibitory effects), the Zipf law ($a\approx 1$) is universally observed for a wide class of nonlinear Hawkes processes with fast-accelerating intensity map; (iii) for marks with a negative mean, the asymptotic power law tail becomes lighter with an exponent increasing above 1 ($a>1$) as the mean mark becomes more negative.

Renewal equationsfor single-particle diffusion through a semipermeable interface.

Diffusion through semipermeable interfaces has a wide range of applications, ranging from molecular transport through biological membranes to reverse osmosis for water purification using artificial membranes. At the single-particle level, one-dimensional diffusion through a barrier with constant permeability κ_{0} can be modeled in terms of so-called snapping out Brownian motion (BM). The latter sews together successive rounds of partially reflected BMs that are restricted to either the left or right of the barrier. Each round is killed (absorbed) at the barrier when its Brownian local time exceeds an exponential random variable parameterized by κ_{0}. A new round is then immediately started in either direction with equal probability. It has recently been shown that the probability density for snapping out BM satisfies a renewal equationthat relates the full density to the probability densities of partially reflected BM on either side of the barrier. Moreover, generalized versions of the renewal equationcan be constructed that incorporate non-Markovian, encounter-based models of absorption. In this paper we extend the renewal theory of snapping out BM to single-particle diffusion in bounded domains and higher spatial dimensions. In each case we show how the solution of the renewal equationsatisfies the classical diffusion equationwith a permeable boundary condition at the interface. That is, the probability flux across the interface is continuous and proportional to the difference in densities on either side of the interface. We also consider an example of an asymmetric interface in which the directional switching after each absorption event is biased. Finally, we show how to incorporate an encounter-based model of absorption for single-particle diffusion through a spherically symmetric interface. We find that, even when the same non-Markovian model of absorption applies on either side of the interface, the resulting permeability is an asymmetric time-dependent function with memory. Moreover, the permeability functions tend to be heavy tailed.

Open-loop equilibrium strategy for mean-variance Portfolio selection with investment constraints in a non-Markovian regime-switching jump-diffusion model

<p style='text-indent:20px;'>This paper is devoted to study the open-loop equilibrium strategy for a mean-variance portfolio problem with investment constraints in a non-Markovian regime-switching jump-diffusion model. Specially, the investment strategies are constrained in a closed convex cone and all coefficients in the model are stochastic processes adapted to the filtration generated by a Markov chain. First, we provide a necessary and sufficient condition for an equilibrium strategy, which involves a system of forward and backward stochastic differential equations (FBSDEs, for short). Second, by solving these FBSDEs, we obtain a feedback representation of the equilibrium strategy. Third, we prove a theorem ensuring the almost everywhere uniqueness of the equilibrium solution. Finally, the results are applied to solve an example of the Markovian regime-switching model.</p>

Optimal investment strategies for asset-liability management with affine diffusion factor processes and HARA preferences

<p style='text-indent:20px;'>This paper investigates an optimal asset-liability management problem within the expected utility maximization framework. The general hyperbolic absolute risk aversion (HARA) utility is adopted to describe the risk preference of the asset-liability manager. The financial market comprises a risk-free asset and a risky asset. The market price of risk depends on an affine diffusion factor process, which includes, but is not limited to, the constant elasticity of variance (CEV), Stein-Stein, Schöbel and Zhu, Heston, 3/2, 4/2 models, and some non-Markovian models, as exceptional examples. The accumulative liability process is featured by a generalized drifted Brownian motion with possibly unbounded and non-Markovian drift and diffusion coefficients. Due to the sophisticated structure of HARA utility and the non-Markovian framework of the incomplete financial market, a backward stochastic differential equation (BSDE) approach is adopted. By solving a recursively coupled BSDE system, closed-form expressions for both the optimal investment strategy and optimal value function are derived. Moreover, explicit solutions to some particular cases of our model are provided. Finally, numerical examples are presented to illustrate the effect of model parameters on the optimal investment strategies in several particular cases.</p>

Analysis on a non-Markovian batch arrival queuing model with phases of service and multi vacations in cloud computing services

Analysis on a non-Markovian batch arrival queuing model with phases of service and multi vacations in cloud computing services

Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systems

Recently, two approaches, fine-tuning large pre-trained language models and variational training, have attracted significant interests, separately, for semi-supervised end-to-end task-oriented dialog (TOD) systems. In this paper, we propose Variational Latent-State GPT model (VLS-GPT), which is the first to combine the strengths of the two approaches. Among many options of models, we propose the generative model and the inference model for variational learning of the end-to-end TOD system, both as auto-regressive language models based on GPT-2, which can be further trained over a mix of labeled and unlabeled dialog data in a semi-supervised manner. Variational training of VLS-GPT is both statistically and computationally more challenging than previous variational learning works for sequential latent variable models, which use turn-level first-order Markovian. The inference model in VLS-GPT is non-Markovian due to the use of the Transformer architecture. In this work, we establish Recursive Monte Carlo Approximation (RMCA) to the variational objective with non-Markovian inference model and prove its unbiasedness. Further, we develop the computational strategy of sampling-then-forward-computation to realize RMCA, which successfully overcomes the memory explosion issue of using GPT in variational learning and speeds up training. Semi-supervised TOD experiments are conducted on two benchmark multi-domain datasets of different languages - MultiWOZ2.1 and CrossWOZ. VLS-GPT is shown to significantly outperform both supervised-only and semi-supervised self-training baselines.

A probabilistic model of diffusion through a semi-permeable barrier

Diffusion through semi-permeable structures arises in a wide range of processes in the physical and life sciences. Examples at the microscopic level range from artificial membranes for reverse osmosis to lipid bilayers regulating molecular transport in biological cells to chemical and electrical gap junctions. There are also macroscopic analogues such as animal migration in heterogeneous landscapes. It has recently been shown that one-dimensional diffusion through a barrier with constant permeability κ 0 is equivalent to snapping out Brownian motion (BM). The latter sews together successive rounds of partially reflecting BMs that are restricted to either the left or the right of the barrier. Each round is killed when its Brownian local time exceeds an exponential random variable parameterized by κ 0 . A new round is then immediately started in either direction with equal probability. In this article, we use a combination of renewal theory, Laplace transforms and Green’s function methods to show how an extended version of snapping out BM provides a general probabilistic framework for modelling diffusion through a semi-permeable barrier. This includes modifications of the diffusion process away from the barrier (e.g. stochastic resetting) and non-Markovian models of membrane absorption that kill each round of partially reflected BM. The latter leads to time-dependent permeabilities.

Non-Markovian dynamics under time-translation symmetry

A dynamical symmetry is employed to determine the structure of the quantum non-Markovian time-local master equation. Such a structure is composed from two components: scalar kinetic coefficients and the standard quantum Markovian operator form. The kinetic coefficients are generally time-dependent and incorporate information on the kinematics and memory effects, while the operators manifest the dynamical symmetry. Specifically, we focus on time-translation symmetric dynamics, where the Lindblad jump operators constitute the eigenoperators of the free dynamics. This symmetry is motivated by thermodynamic microscopic considerations, where strict energy conservation between system and environment imposes the time-translation symmetry. The construction is generalized to other symmetries, and to driven quantum systems. The formalism is illustrated by three exactly solvable non-Markovian models, where the exact reduced description exhibits a dynamical symmetric structure. The formal structure of the master equation leads to a first principle calculation of the exact kinetic coefficients. This opens the possibility to simulate in a modular fashion non-Markovian dynamics.

Resonance behavior for the dynamical friction of a system in a trapping potential

The kinetic energy fluctuation metric ${\mathrm{\ensuremath{\Omega}}}_{\mathbf{K}}(t)$ is modified to extract the dynamical friction in non-Markovian scenarios to circumvent Markovian scenarios. Dynamical friction is calculated from the ratio ${\mathrm{\ensuremath{\Omega}}}_{\mathbf{K}}(0)/{\mathrm{\ensuremath{\Omega}}}_{\mathbf{K}}(t)$, which obeys a universal scaling law. Both analytical and numerical results of a minimal non-Markovian model substantiate the validity of the modified kinetic energy fluctuation metric. We reveal that previous studies on the rates of relaxation in harmonically bounded proteins, which are somewhat larger than those resulting from the experimental fit, may have neglected non-Markovian effects. When applying the metric to a system subjected to thermal broadband colored noise in the harmonic potential, resonance phenomenon occurs. Specifically, dynamical friction varies nonmonotonically with the frequency of the harmonic potential. The optimal frequency inducing the strongest dynamical friction matches well with the peak frequency of power spectrum of thermal broadband colored noise. The system then approaches to equilibrium rapidly. For an acoustic phonon spectrum, the resonance for dynamical friction provides further insight as to why complete thermalization only occurs when the particle frequency of the system is within a certain range of the environment particle frequencies. This is because the closer the potential frequency is to the central value of the range of frequencies, the stronger is the dynamical friction induced.

Quantitative assessment of the effects of resource optimization and ICU admission policy on COVID-19 mortalities

It is evident that increasing the intensive-care-unit (ICU) capacity and giving priority to admitting and treating patients will reduce the number of COVID-19 deaths, but the quantitative assessment of these measures has remained inadequate. We develop a comprehensive, non-Markovian state transition model, which is validated through the accurate prediction of the daily death toll for two epicenters: Wuhan, China and Lombardy, Italy. The model enables prediction of COVID-19 deaths in various scenarios. For example, if appropriate treatment priorities had been used, the death toll in Wuhan and Lombardy would have been reduced by about 10% and 7%, respectively. The strategy depends on the epidemic scale and is more effective in countries with a younger population structure. Analyses of data from China, South Korea, Italy, and Spain suggest that countries with less per capita ICU medical resources should implement this strategy in the early stage of the pandemic to reduce mortalities. We emphasize that the results of this paper should be interpreted purely from a scientific and a quantitative-analysis point of view. No ethical implications are intended and meaningful.

Unexpected advantages of exploitation for target searches in complex networks.

Exploitation universally emerges in various decision-making contexts, e.g., animals foraging, web surfing, the evolution of scientists' research topics, and our daily lives. Despite its ubiquity, exploitation, which refers to the behavior of revisiting previous experiences, has often been considered to delay the search process of finding a target. In this paper, we investigate how exploitation affects search performance by applying a non-Markovian random walk model, where a walker randomly revisits a previously visited node using long-term memory. We analytically study two broad forms of network structures, namely, (i) clique-like networks and (ii) lollipop-like networks and find that exploitation can significantly improve search performance in lollipop-like networks, whereas it hinders target search in clique-like networks. Moreover, we numerically verify that exploitation can reduce the time needed to fully explore the underlying networks using 550 diverse real-world networks. Based on the analytic result, we define the lollipop-likeness of a network and observe a positive relationship between the advantage of exploitation and lollipop-likeness.

Persistent Correlation in Cellular Noise Determines Longevity of Viral Infections.

The slowly decaying viral dynamics, even after 2-3 weeks from diagnosis, is one of the characteristics of COVID-19 infection that is still unexplored in theoretical and experimental studies. This long-lived characteristic of viral infections in the framework of inherent variations or noise present at the cellular level is often overlooked. Therefore, in this work, we aim to understand the effect of these variations by proposing a stochastic non-Markovian model that not only captures the coupled dynamics between the immune cells and the virus but also enables the study of the effect of fluctuations. Numerical simulations of our model reveal that the long-range temporal correlations in fluctuations dictate the long-lived dynamics of a viral infection and, in turn, also affect the rates of immune response. Furthermore, predictions of our model system are in agreement with the experimental viral load data of COVID-19 patients from various countries.

Non-Markovian SIR epidemic spreading model of COVID-19

We introduce non-Markovian SIR epidemic spreading model inspired by the characteristics of the COVID-19, by considering discrete- and continuous-time versions. The distributions of infection intensity and recovery period may take an arbitrary form. By taking corresponding choice of these functions, it is shown that the model reduces to the classical Markovian case. The epidemic threshold is analytically determined for arbitrary functions of infectivity and recovery and verified numerically. The relevance of the model is shown by modeling the first wave of the epidemic in Italy, Spain and the UK, in the spring, 2020.

Functional limit theorems for non-Markovian epidemic models

We study non-Markovian stochastic epidemic models (SIS, SIR, SIRS, and SEIR), in which the infectious (and latent/exposing, immune) periods have a general distribution. We provide a representation of the evolution dynamics using the time epochs of infection (and latency/exposure, immunity). Taking the limit as the size of the population tends to infinity, we prove both a functional law of large number (FLLN) and a functional central limit theorem (FCLT) for the processes of interest in these models. In the FLLN, the limits are a unique solution to a system of deterministic Volterra integral equations, while in the FCLT, the limit processes are multidimensional Gaussian solutions of linear Volterra stochastic integral equations. In the proof of the FCLT, we provide an important Poisson random measures representation of the diffusion-scaled processes converging to Gaussian components driving the limit process.

Dynamic survival analysis for non-Markovian epidemic models.

We present a new method for analysing stochastic epidemic models under minimal assumptions. The method, dubbed dynamic survival analysis (DSA), is based on a simple yet powerful observation, namely that population-level mean-field trajectories described by a system of partial differential equations may also approximate individual-level times of infection and recovery. This idea gives rise to a certain non-Markovian agent-based model and provides an agent-level likelihood function for a random sample of infection and/or recovery times. Extensive numerical analyses on both synthetic and real epidemic data from foot-and-mouth disease in the UK (2001) and COVID-19 in India (2020) show good accuracy and confirm the method’s versatility in likelihood-based parameter estimation. The accompanying software package gives prospective users a practical tool for modelling, analysing and interpreting epidemic data with the help of the DSA approach.

Optimal DC pension investment with square-root factor processes under stochastic income and inflation risks

ABSTRACT This paper studies optimally defined contribution (DC) pension investment problems under the expected utility maximization framework with stochastic income and inflation risks. The member has access to a financial market consisting of a risk-free asset (money account), an inflation-indexed bond, and a stock. The market price of volatility risk is assumed to depend on an affine-form, Markovian, square-root factor process, while the return rate and the volatility of the stock are possibly given by general non-Markovian, unbounded stochastic processes. This financial framework recovers the Black–Scholes model, constant elasticity of variance (CEV) model, Heston model, 3/2 model, 4/2 model, and some non-Markovian models as exceptional cases. To tackle the potentially non-Markovian structures, we adopt a backward stochastic differential equation (BSDE) approach. By solving the associated BSDEs explicitly, closed-form expressions for the optimal investment strategies and optimal value functions are obtained for the power, logarithmic, and exponential utility functions. Moreover, explicit solutions to some special cases of our portfolio model are provided. Finally, numerical examples are provided to illustrate the effects of model parameters on the optimal investment strategies under the 4/2 model.

Kinetics of Hawking-Page phase transition with the non-Markovian effects

Based on the free energy landscape description of Hawking-Page phase transition, the transition process from the Schwarzschild-anti-de Sitter black hole to the thermal anti-de Sitter space are considered to be stochastic under the thermal fluctuations. If the correlation time of the effective thermal bath is comparable or even longer than the oscillating time of the spacetime state in the potential well on the free energy landscape, the non-Markovian model of the black hole phase transition is required to study the kinetics of the transition processes. The non-Markovian or memory effect is represented by the time dependent friction kernel and the kinetics is then governed by the generalized Langevin equation complemented by the free energy potential. As the concrete examples, we study the effects of the exponentially decay friction kernel and the oscillatory friction kernel on the kinetics of Hawking-Page phase transition. For the exponentially decayed friction, the non-Markovian effects promote the transition process, and for the oscillatory friction, increasing the oscillating frequency also speeds up the transition process.

Endemic state equivalence between non-Markovian SEIS and Markovian SIS model in complex networks

In the light of several major epidemic events that emerged in the past two decades, and emphasized by the COVID-19 pandemics, the non-Markovian spreading models occurring on complex networks gained significant attention from the scientific community. Following this interest, in this article, we explore the relations that exist between the mean-field approximated non-Markovian SEIS (Susceptible–Exposed–Infectious–Susceptible) and the classical Markovian SIS, as basic reoccurring virus spreading models in complex networks. We investigate the similarities and seek for equivalences both for the discrete-time and the continuous-time forms. First, we formally introduce the continuous-time non-Markovian SEIS model, and derive the epidemic threshold in a strict mathematical procedure. Then we present the main result of the paper that, providing certain relations between process parameters hold, the stationary-state solutions of the status probabilities in the non-Markovian SEIS may be found from the stationary state probabilities of the Markovian SIS model. This result has a two-fold significance. First, it simplifies the computational complexity of the non-Markovian model in practical applications, where only the stationary distributions of the state probabilities are required. Next, it defines the epidemic threshold of the non-Markovian SEIS model, without the necessity of a thrall mathematical analysis. We present this result both in analytical form, and confirm the result through numerical simulations. Furthermore, as of secondary importance, in an analytical procedure we show that each Markovian SIS may be represented as non-Markovian SEIS model.

Phase Transition in a Non-Markovian Animal Exploration Model with Preferential Returns.

We study a non-Markovian and nonstationary model of animal mobility incorporating both exploration and memory in the form of preferential returns. Exact results for the probability of visiting a given number of sites are derived and a practical WKB approximation to treat the nonstationary problem is developed. Amean-field version of this model, first suggested by Song etal., [Modelling the scaling properties of human mobility, Nat. Phys. 6, 818 (2010)NPAHAX1745-247310.1038/nphys1760] was shown to well describe human movement data. We show that our generalized model adequately describes empirical movement data of Egyptian fruit bats (Rousettus aegyptiacus) when accounting for interindividual variation in the population. We also study the probability of visiting any site a given number of times and derive a mean-field equation. Our analysis yields a remarkable phase transition occurring at preferential returns which scale linearly with past visits. Following empirical evidence, we suggest that this phase transition reflects a trade-off between extensive and intensive foraging modes.

Optimal switching paths of a non-Markovian model of gene bursty expression

Gene expression usually involves multiple biochemical reactions, resulting into the probability of waiting time between consecutive reaction steps is a non-exponential probability distribution, and creating molecular memory. Besides, some experiments indicate that burst of gene expression and positive feedback loops play important roles in gene expression and phenotypic variation. However, how molecular memory and the burst together affect metastable states in gene expression is still open, as far as our knowledge. In this paper, we build a generalized chemical master equation of gene bursty expression with a self-regulatory positive feedback. Using the Wentzel-Kramers-Brillouin (WKB) method, we study optimal paths of switching between bistable states of gene expression under the combination of molecular memory and the burst. Through theoretical analysis and numerical simulations, we find that the burst and molecular memory can change the metastable state structure of the mean-field system of our model, and that the burst can prolong optimal paths of switching from the high state of gene expression to the low state, and shorten optimal paths of switching from the low state to the high state, but molecular memory has the opposite effect on the optimal switching paths. This result is the further improvement on our previous work (Yin et al., 2021).