Time-homogeneous Markov Process Research Articles

Feynman-Kac formula for general diffusion equations driven by TFBM with Hurst index H ∈ (0,1)

We consider the general diffusion equation driven by tempered fractional Brownian motion (TFBM)∂u(t,x)∂t=Lu(t,x)+u(t,x)dBH,λ(t)dt,(t,x)∈R+×Rd, where L is the infinitesimal generator of some time-homogeneous Markov process {Xt}t≥0 and {BH,λ(t)}t∈R is a tempered fractional Brownian motion with Hurst index H∈(0,12)∪(12,1) and tempering parameter λ>0 which is independent of {Xt}t≥0. Based on approximating TFBM with a family of Gaussian processes possessing absolutely continuous sample paths, a unified framework of the Feynman-Kac formulau(t,x)=Ex[f(Xt)]eBH,λ(t) is established for the general stochastic diffusion equation driven by TFBM with the initial value f, Hurst index H∈(0,12)∪(12,1) and tempering parameter λ>0. The difficulty is that the Hurst parameter H can be allowed to be less than 1/2. The idea is to explore the above simplest form, to utilize the techniques of Malliavin calculus. By using the properties of TFBM and especially that of the modified Bessel function of the second kind, we prove that the process defined by the Feynman-Kac formula is the mild and weak solutions of the general diffusion equation driven by TFBM. From the Feynman-Kac formula, we exhibit the smoothness of the density of the solution and the sharp Hölder regularity. We further show that limt→∞⁡ln⁡‖u(t,⋅)‖pg(t)=0 in the almost surely sense where p∈R and the function g:R+→R+ grows at least as fast as a linear function. The Feynman-Kac formula for the stochastic general diffusion equation in the Skorohod sense is also achieved by exploring the relationship between the Stratonovich and Skorohod integrals.

Speed and duration of drawdown under general Markov models

We propose an efficient computational method based on continuous-time Markov chain (CTMC) approximation to compute the distributions of the speed and duration of drawdown for general one-dimensional (1D) time-homogeneous Markov processes. We derive linear systems for the Laplace transforms of drawdown quantities and show how to solve them efficiently by recursion. In addition, we prove the convergence of our method and obtain a sharp estimate of the convergence rate under some assumptions. As applications, we consider pricing two financial options that provide protection against drawdown risk. Finally, we demonstrate the accuracy and efficiency of our method through extensive numerical experiments.

Globally optimal interferometry with lossy twin Fock probes

Parity or quadratic spin (e.g., Jz2) readouts of a Mach–Zehnder (MZ) interferometer probed with a twin Fock (TF) input state allow saturating the optimal sensitivity attainable among all mode-separable states with a fixed total number of particles but only when the interferometer phase θ is near zero. When more general Dicke state probes are used, the parity readout saturates the quantum Fisher information (QFI) at θ = 0, whereas better-than-standard quantum limit performance of the Jz2 readout is restricted to an o(N) occupation imbalance. We show that a method of moments readout of two quadratic spin observables Jz2 and J+2+J−2 is globally optimal for Dicke state probes; i.e., the error saturates the QFI for all θ. In the lossy setting, we derive the time-inhomogeneous Markov process describing the effect of particle loss on TF states, showing that the method of moments readout of four at-most-quadratic spin observables is sufficient for globally optimal estimation of θ when two or more particles are lost. The analysis culminates in a numerical calculation of the QFI matrix for distributed MZ interferometry on the four-mode state |N4,N4,N4,N4〉 and its lossy counterparts, showing that an advantage for the estimation of any linear function of the local MZ phases θ1 and θ2 (compared to independent probing of the MZ phases by two copies of |N4,N4〉) appears when more than one particle is lost.

Compound Poisson disorder problem with uniformly distributed disorder time

Suppose that the arrival rate and the jump distribution of a compound Poisson process change suddenly at an unknown and unobservable time. We want to detect the change as quickly as possible to take counteractions, e.g., to assure top quality of products in a production system, or to stop credit card fraud in a banking system. If we have no prior information about future disorder time, then we typically assume that the disorder is equally likely to happen any time – or has uniform distribution – over a long but finite time horizon. We solve this so-called compound Poisson disorder problem for the practically important case of unknown, unobserved, but uniformly distributed disorder time. The solution hinges on the complete separation of information flow from the hard time horizon constraint, by describing the former with an autonomous time-homogeneous one-dimensional Markov process in terms of which the detection problem translates into a finite horizon optimal stopping problem. For any given finite horizon, the solution is two-dimensional. For cases where the horizon is large and one is unwilling to set a fixed value for it, we give a one-dimensional approximation. Also, we discuss an extension where the disorder may not happen on the given interval with a positive probability. In this extended model, if no detection decision is made by the end of the horizon, then a second level hypothesis testing problem is solved to determine the local parameters of the observed process.

A general approach for lookback option pricing under Markov models

We propose a computationally efficient method for pricing various types of lookback options under Markov models. We utilize the model-free representations of lookback option prices as integrals of first passage probabilities. We combine efficient numerical quadrature with continuous-time Markov chain approximation for the first passage problem to price lookbacks. Our method is applicable to a variety of models, including one-dimensional time-homogeneous and time-inhomogeneous Markov processes, regime-switching models and stochastic local volatility models. We demonstrate the efficiency of our method through various numerical examples.

Stationary distribution of the Milstein scheme for stochastic differential delay equations with first-order convergence

This paper focuses on the stationary distribution of the Milstein scheme for stochastic differential delay equations. The numerical segment process is constructed, which is proved to be a time homogeneous Markov process. We show that this numerical segment process admits a unique numerical stationary distribution. Then we reveal that the distribution of numerical segment process converges exponentially to the underlying one in the Wasserstein metric. Moreover, the first-order convergence of numerical stationary distribution to exact stationary distribution is presented. Finally, abundant numerical experiments confirm the reliability of theoretical findings.

Best Arm Identification in Restless Markov Multi-Armed Bandits

We study the problem of identifying the best arm in a multi-armed bandit environment when each arm is a time-homogeneous and ergodic discrete-time Markov process on a common, finite state space. The state evolution on each arm is governed by the arm’s transition probability matrix (TPM). A decision entity that knows the set of arm TPMs but not the exact mapping of the TPMs to the arms, wishes to find the index of the best arm as quickly as possible, subject to an upper bound on the error probability. The decision entity selects one arm at a time sequentially, and all the unselected arms continue to undergo state evolution ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">restless</i> arms). For this problem, we derive the first-known problem instance-dependent asymptotic lower bound on the growth rate of the expected time required to find the index of the best arm, where the asymptotics is as the error probability vanishes. Further, we propose a sequential policy that, for an input parameter <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> , forcibly selects an arm that has not been selected for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> consecutive time instants. We show that this policy achieves an upper bound that depends on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> and is monotonically non-increasing as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> → ∞. The question of whether, in general, the limiting value of the upper bound as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> → ∞ matches with the lower bound, remains open. We identify a special case in which the upper and the lower bounds match. Prior works on best arm identification have dealt with (a) independent and identically distributed observations from the arms, and (b) rested Markov arms, whereas our work deals with the more difficult setting of restless Markov arms.

An ergodic theorem for asymptotically periodic time-inhomogeneous Markov processes, with application to quasi-stationarity with moving boundaries

AbstractThis paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded function f, the time average $\frac{1}{t} \int_0^t f(X_s)ds$ converges in $\mathbb{L}^2$ towards a limiting distribution, starting from any initial distribution for the process $(X_t)_{t \geq 0}$ . This convergence can be improved to an almost sure convergence under an additional assumption on the initial measure. This result is then applied to show the existence of a quasi-ergodic distribution for processes absorbed by an asymptotically periodic moving boundary, satisfying a conditional Doeblin condition.

Resilience modeling for multi-state systems based on Markov processes

Modern systems are increasingly under the threaten of disruptive events like earthquakes, floods and storms. Under real life scenarios, multi-state models are often used to describe the behaviors of the system exposed to disruptive events. This article develops a comprehensive resilience modeling and quantifying framework for a multi-state system in which the evolution of the performance level over time is described by a time-homogeneous Markov process. In order to characterize the different dimensions of the system resilience, four types of resilience metrics are proposed to describe the resistant, absorption, recovery, and overall resilience, where each type is divided into an inherent resilience metric and an acquired resilience metric. The theory of aggregated stochastic processes is applied to derive explicit formulas for the four types of resilience metrics. Meanwhile, simulation-based algorithms are proposed to verify the correctness of analytical formulas. They are first exploited to the resilience analysis of a nuclear power plant under the threat of earthquakes, and then used in a numerical example to illustrate the applicability of the proposed method in dealing with the state space explosion problem. The results show that the developed resilience modeling and quantifying framework is able to comprehensively describe the resilience of multi-state systems threatened by disruptive events, and further, some practical suggestions are given to the designers and operators of the system based on the results.

Time reversal of spinal processes for linear and non-linear branching processes near stationarity

We consider a stochastic individual-based population model with competition, trait-structure affecting reproduction and survival, and changing environment. The changes of traits are described by jump processes, and the dynamics can be approximated in large population by a non-linear PDE with a non-local mutation operator. Using the fact that this PDE admits a non-trivial stationary solution, we can approximate the non-linear stochastic population process by a linear birth-death process where the interactions are frozen, as long as the population remains close to this equilibrium. This allows us to derive, when the population is large, the equation satisfied by the ancestral lineage of an individual uniformly sampled at a fixed time T, which is the path constituted of the traits of the ancestors of this individual in past times t≤T. This process is a time inhomogeneous Markov process, but we show that the time reversal of this process possesses a very simple structure (e.g. time-homogeneous and independent of T). This extends recent results where the authors studied a similar model with a Laplacian operator but where the methods essentially relied on the Gaussian nature of the mutations.

On the local times of noise reinforced Bessel processes

We investigate the effects of noise reinforcement on a Bessel process of dimension $d\in(0,2)$, and more specifically on the asymptotic behavior of its additive functionals. This leads us to introduce a local time process and its inverse. We identify the latter as an increasing self-similar (time-homogeneous) Markov process, and from this, several explicit results can be deduced.

Multistate models for the natural history of cancer progression.

Multistate models can be effectively used to characterise the natural history of cancer. Inference from such models has previously been useful for setting screening policies. We introduce the basic elements of multistate models and the challenges of applying these models to cancer data. Through simulation studies, we examine (1) the impact of assuming time-homogeneous Markov transition intensities when the intensities depend on the time since entry to the current state (i.e., the process is time-inhomogenous semi-Markov) and (2) the effect on precancer risk estimation when observation times depend on an unmodelled intermediate disease state. In the settings we examined, we found that misspecifying a time-inhomogenous semi-Markov process as a time-homogeneous Markov process resulted in biased estimates of the mean sojourn times. When screen-detection of the intermediate disease leads to more frequent future screening assessments, there was minimal bias induced compared to when screen-detection of the intermediate disease leads to less frequent screening. Multistate models are useful for estimating parameters governing the process dynamics in cancer such as transition rates, sojourn time distributions, and absolute and relative risks. As with most statistical models, to avoid incorrect inference, care should be given to use the appropriate specifications and assumptions.

General drawdown of general tax model in a time-homogeneous Markov framework

AbstractDrawdown/regret times feature prominently in optimal stopping problems, in statistics (CUSUM procedure), and in mathematical finance (Russian options). Recently it was discovered that a first passage theory with more general drawdown times, which generalize classic ruin times, may be explicitly developed for spectrally negative Lévy processes [9, 20]. In this paper we further examine the general drawdown-related quantities in the (upward skip-free) time-homogeneous Markov process, and then in its (general) tax process by noticing the pathwise connection between general drawdown and the tax process.

A Research on the Different Financing Models of Long-term Care Insurance in China

Establishing a long-term care insurance (LTCI) system covering the whole people and balancing urban and rural areas is an important strategic plan for China to deal with the aging and disability of the population. Since the establishment of the LTCI system at the national level in 2016, China has 49 pilot areas for LTCI by 2020. Each pilot area is seriously fragmented, and the financing model of LTCI has not yet been finalized. So what is the financing model of LTCI that suits China’s national conditions? In addition, different financing models have their own advantages and disadvantages. The optimal financing model may depend on economic, social and demographic factors. Since the fund funded system does not have the function of mutual assistance in social insurance, this paper limits the financing model of LTCI between the pay-as-you-go system and the mixed system. Which one is optimal for the long-term sustainable development of China’s LTCI? Starting from the evolutionary law of the health status of elderly individuals, this paper explores the operation of China’s LTCI system under different financing models.This paper uses multi-period micro-tracking survey data, through the continuous time homogeneous Markov process and the generalized linear model, to describe the movement law of the health status transition of the elderly in China, and combines the seven census data and the age shift algorithm to calculate the size and structure of the disabled elderly population from 2020 to 2040. Further, based on the social insurance actuarial balance theory, a dynamic actuarial model under the pay-as-you-go system and three mixed systems is constructed to simulate the equilibrium premium rate and payment amount of LTCI for urban employees and urban and rural residents in the next 20 years. The study finds that, in terms of the size of disabled population, the female elderly are the main body of disabled population; the scale of disabled population in urban areas is significantly higher than that in rural areas. The main reason is that rural-urban migration has made urban elderly population larger, and the urban disability rate is higher than that in rural areas. In addition, in terms of funding pressure, the study shows that at any time and under any financing model, the relative payment pressure of LTCI for urban employees is significantly lower than that of urban and rural residents, but the absolute payment pressure is slightly higher than that of urban and rural residents. Finally, in terms of the selection of financing models, this paper believes that LTCI for urban employees should adopt a mixed system, and LTCI for urban and rural residents should adopt a pay-as-you-go system.The contribution of this paper is mainly reflected in two aspects: First, at the level of empirical research, it combines macro and micro data to describe the evolution of the health status of the elderly from the micro level, predicts the size of disabled population in China from the macro level, and finally establishes a LTCI system that covers the entire population. This not only deepens the research on individual health, but also enriches the research results in the social insurance actuarial field. Second, at the policy level, based on the results of the research, it provides an optimal financing model for LTCI for urban employees and LTCI for urban and rural residents. This optimal model can alleviate the payment burden of financing entities and enhance the stability of the LTCI system. It has important practical significance for the work focus and further promotion of China’s LTCI system.

State space splitting of a finite markov process and some discussions on related counting processes

In this paper, state space of a time-homogeneous Markov process is split into several ordered subspaces. Then, there are three kinds of transitions between states—transitions from a higher-order subspace to a lower-order subspace, transitions within the same subspace, and transitions from a lower-order subspace to a higher-order subspace. Considering time interval omission problem and first passage time considered, we define some related counting processes for the Markov process and discuss their associated probabilities, expectations and generating functions by using Laplace transform. The main results are presented in matrix forms. The relationships among counting processes and their special cases are also discussed briefly. Some simple numerical examples are presented to illustrate the established results. These results will be useful in different problems arising in reliability, economics and social science fields.

Optimal Control and Stabilization for Networked Systems With Input Delay and Markovian Packet Losses

The linear quadratic regulation (LQR) problem for discrete-time networked control systems (NCSs) is investigated in this article. The difference from most previous works is that input delay and packet losses occur simultaneously in the communication channel connecting the controller to the actuator. Moreover, the data packet dropout is modeled as a time-homogeneous Markov process which poses challenges due to the temporal correlation. The contributions of this article are twofold. First, by applying the maximum principle involving Markov jumps and delay, the linear quadratic optimal control problem in finite horizon is solved and the solution is given in terms of a forward and a backward stochastic difference equations (FBSDEs-M). Second, under a basic assumption, the infinite horizon optimal control problem is solved and the necessary and sufficient condition for mean square stabilization is given in terms of the solutions to coupled algebraic Riccati-type equations with Markov jumps. The presented results are new to the best of our knowledge since there is no existing work that tackles delay and Markovian packet dropouts simultaneously. It is a generalization of the previous work in which the packet dropouts is modeled as an independent identically distributed Bernoulli process.

THE APPLICABILITY OF FINITE HOMOGENEOUS MARKOV PROCESSES IN THE STUDY OF CONSUMER LOYALTY

Markov Processes are sequential events that are related to each other stochastically. Such events, also known as states, may be such that the probability of an event occurring depends only on the previous event and not on any event prior to that. This is known as the memoryless property of Markov Processes. Certain dynamic market conditions especially with respect to the Fast-Moving Consumer Goods Sector, the Telecommunications Sector may enable the use of finite and time-homogeneous Markov Processes to study the brand switching tendency of the consumer, and thus predict consumer loyalty. In this conceptual paper, we shall study how to predict brand switching tendencies using finite, time-homogeneous Markov Processes. KEYWORDS: Markov Process, Brand Switching, Time-Homogeneous, Consumer Loyalty, Transition Matrix

Periodic Solutions of Stochastic Differential Equations Driven by Lévy Noises

In this paper, we first show the well-posedness of SDEs driven by Lévy noises under mild conditions. Then, we consider the existence and uniqueness of periodic solutions of the SDEs. To establish the ergodicity and uniqueness of periodic solutions, we investigate the strong Feller property and the irreducibility of the corresponding time-inhomogeneous semigroups when both small and large jumps are allowed in the equations. Doob’s celebrated theorem on the uniqueness of invariant measures for time-homogeneous Markov processes has been generalized to obtain the uniqueness of periodic measures for time-inhomogeneous Markov processes. Some examples are presented to illustrate our results.

Estimating a density, a hazard rate, and a transition intensity via the ρ-estimation method

We propose a unified study of three statistical settings by widening the ρ-estimation method developed in [BBS17]. More specifically, we aim at estimating a density, a hazard rate (from censored data), and a transition intensity of a time inhomogeneous Markov process. We relate the performance of ρ-estimators to deviations of a (possibly unbounded) empirical process. We deduce non-asymptotic risk bounds for an Hellinger-type loss on possibly random models. When the models are convex, maximum likelihood estimators coincide with ρ-estimators, and satisfy therefore our risk bounds. However, our results also apply to some models where the maximum likelihood method does not work. Besides, the robustness properties of ρ-estimators are not, in general, shared by maximum likelihood estimators. Subsequently, we present an alternative procedure to ρ-estimation, more numerically friendly, that yields a piecewise polynomial estimator. We prove theoretical results and carry out some numerical simulations that show the benefits of our approach compared with a more classical one based on maximum likelihood.

A General Method for Analysis and Valuation of Drawdown Risk under Markov Models

Drawdown risk is a major concern in financial markets. We develop a novel algorithm to solve the first passage problem of the drawdown process of general one-dimensional time-homogeneous Markov processes. We compute its Laplace transform based on continuous time Markov chain (CTMC) approximation and numerically invert the Laplace transform to obtain the first passage probabilities and the distribution of the maximum drawdown. We prove convergence of our algorithm for general Markovian models and provide sharp estimate of the convergence rate for a general class of jump-diffusion models. We apply the algorithm to calculate the Calmar ratio for investment analysis, price maximum drawdown derivatives and hedge the risk of selling such derivatives with a highly volatile asset as the underlying. Various numerical experiments document the computational efficiency of our method. We also develop extensions to solve the drawdown problem in models with time dependence or stochastic volatility or regime switching and for portfolio drawdown analysis.