General Markov Processes Research Articles

Closed-form approximations of moments and densities of continuous–time Markov models

This paper develops power series expansions of a general class of moment functions, including transition densities and option prices, of continuous-time Markov processes, including jump–diffusions. The proposed expansions extend the ones in Kristensen and Mele (2011) to cover general Markov processes, and nest transition density and option price expansions recently developed in the literature, thereby connecting seemingly different ideas in a unified framework. We show how the general expansion can be implemented for fully general jump–diffusion models. We provide a new theory for the validity of the expansions which shows that series expansions are not guaranteed to converge as more terms are added in general once the time span of interest gets larger than some model–specific threshold. Thus, these methods should be used with caution when applied to problems with a larger time span of interest, such as long-term options or data observed at a low frequency. At the same time, the numerical studies in this paper demonstrate good performance of the proposed implementation in practice when applied to pricing options with time to maturity below three months. Thus, our expansions are particularly well suited for pricing ultra-short-term (such as “zero–day”) options.

On Doob h-transformations for finite-time quantum state reduction

The paper develops a finite-time quantum state reduction framework via the use of Lévy random bridges (LRBs) that can be understood as Doob h-transformations on Lévy processes. Building upon the non-anticipative semimartingale representation of LRBs, we propose a family of energy-driven stochastic Schrödinger equations that go beyond the purely-continuous Brownian motion setup, and enter the scope of quantum systems involving discontinuities and heavy-tails. In doing so, we allow collapse dynamics to be governed intrinsically by the Markovian statistics of LRBs. The framework can host progressively convoluted stochastic state reduction dynamics in a tractable way and encapsulates jump-diffusion Schrödinger evolutions by use of the Lévy-Itô decomposition. Our construct motivates the usage of more general Markov processes and Doob h-transformations in producing consistent wave function collapse dynamics in finite-time.

Mixing times and hitting times for general Markov processes

The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi [PS15] and Oliveira [Oli12], the authors show that the mixing times and “worst-case” hitting times of reversible Markov chains on finite state spaces are equal up to some universal multiplicative constant. We use tools from nonstandard analysis to extend this result to reversible Markov chains on general state spaces that satisfy the strong Feller property. Finally, we show that this asymptotic equivalence can be used to find bounds on the mixing times of a large class of Markov chains used in MCMC, such as typical Gibbs samplers and Metropolis–Hastings chains, even though they usually do not satisfy the strong Feller property.

Classical uncertainty relations and entropy production in non-equilibrium statistical mechanics

We analyze Fürth’s 1933 classical uncertainty relations in the modern language of stochastic differential equations. Our interest is motivated by their application to non-equilibrium classical statistical mechanics. We show that Fürth’s uncertainty relations are a property inherent in martingales within the framework of a diffusion process. This result implies a lower bound on the fluctuations in current velocities of entropic quantifiers associated with transitions in stochastic thermodynamics. In cases of particular interest, we recover a well-known inequality for optimal mass transport relating the mean kinetic energy of the current velocity and the squared quadratic Wasserstein distance between the probability distributions of the entropy. We take advantage in particular of an unpublished suggestion by Krzysztof Gawȩdzki to derive a lower bound to the entropy production by a transition described by a Langevin–Kramers process in terms of the squared quadratic Wasserstein distance between the initial and final states of the transition. Finally, we illustrate how Fürth’s relations admit a straightforward extension to piecewise deterministic processes. We show that the results presented in this paper pertain to the characteristics exhibited by general Markov processes.

On quasi-stationaries for symmetric Markov processes

This paper intends to give some mild sufficient conditions for the existence and uniqueness of quasi-stationary distributions (QSDs) for general symmetric Markov processes. Under the same conditions, it is also proved that: the unique QSD attracts exponentially all initial distributions supported in the allowed states; the considered process admits a quasi-ergodic stationary distribution (QESD); the Lp-spectral bounds of its associated semigroup are independent of 1≤p≤∞. Finally, we present three typical examples to illustrate these results.

A general method for analysis and valuation of drawdown risk

Drawdown risk is a major concern in financial markets. We develop a novel method to solve the first passage problem of the drawdown process for general one-dimensional Markov processes (including time-inhomogeneous ones) as well as regime-switching and stochastic volatility models. We compute its Laplace transform based on continuous-time Markov chain (CTMC) approximation and invert the Laplace transform numerically to obtain the first passage probabilities and the distribution of the maximum drawdown. We prove convergence of our method for general Markov models and provide sharp estimate of the convergence rate for a general class of jump-diffusion models. We apply our method to price and hedge maximum drawdown options and demonstrate its accuracy and efficiency through various numerical experiments. In addition, we can apply our method to calculate the Calmar ratio for investment analysis, and quantify the contributions of assets to the drawdown risk of a portfolio when the assets follow multivariate exponential Lévy models.

Occupation time statistics of the fractional Brownian motion in a finite domain.

We study statistics of occupation times for a fractional Brownian motion (fBm), which is a typical model of a non-Markov process. Due to the non-Markovian nature, recurrence times to the origin depend on the history. Numerical simulations indicate that dependence on the sum of successive recurrence times becomes weak. As a result, the distribution of the occupation time in a finite domain follows the Mittag-Leffler distribution when the Hurst exponent of the fBm is close to 1/2. We show this distributional behavior of a time-averaged observable by renewal theory. This result is an extension of the distributional limit theorem known as the Darling-Kac theorem in general Markov processes to non-Markov processes.

Dirichlet eigenvalues and exit time moments for symmetric Markov processes

We give some relationships between the first Dirichlet eigenvalues and the exit time moments for the general symmetric Markov processes. As applications, we present some examples, including symmetric diffusions and α-stable processes, and provide the estimates of their first Dirichlet eigenvalues and the exit time moments.

Variational Principles for Asymptotic Variance of General Markov Processes

A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get an upper bound of the mean exit time of reversible Markov processes, and some comparison theorems between the reversible and non-reversible diffusion processes.

Universal minimal cost of coherent biochemical oscillations.

Biochemical clocks are essential for virtually all living systems. A biochemical clock that is isolated from an external periodic signal and subjected to fluctuations can oscillate coherently only for a finite number of oscillations. Furthermore, such an autonomous clock can oscillate only if it consumes free energy. What is the minimum amount of free-energy consumption required for a certain number of coherent oscillations? We conjecture a universal bound that answers this question. A system that oscillates coherently for N oscillations has a minimal free-energy cost per oscillation of 4π^{2}Nk_{B}T. Our bound is valid for general finite Markov processes, is conjectured based on extensive numerical evidence, is illustrated with numerical simulations of a known model for a biochemical oscillator, and applies to existing experimental data.

Analysis of k-Out-of-N-Systems with Different Units under Simultaneous Failures: A Matrix-Analytic Approach

An N-system with different units submitted to shock and wear is studied. The shocks cause damage and, eventually, simultaneous failures of several units. The units can also fail due to internal failures. At random times, the system is inspected, and the down units are simultaneously replaced by identical ones. The arrival of shocks is governed by a Markovian arrival process. The operational times and the interarrival times between inspections follow phase-type distributions. The generator of the multidimensional Markov process modeling the system is constructed. This is performed introducing indicator functions for the different transition rates among the units using the algorithm of Kronecker. This is a general Markov process that can be applied for modeling different reliability systems depending on the structure of the units and how the systems operate. The general model is applied to the study of k-out-of-N systems, calculating the main performance measures. A practical example is presented showing the approximation of the model to a system with units following different Weibull distributions.

General optimal stopping with linear cost

This article treats both discrete time and continuous time stopping problems for general Markov processes on the real line with general linear costs as they naturally arise in many problems in sequential decision making. Using an auxiliary function of maximum representation type, conditions are given to guarantee the optimal stopping time to be of threshold type. The optimal threshold is then characterized as the root of that function. For random walks, our results condense in the fact that all combinations of concave increasing payoff functions and convex cost functions lead to a one-sided solution. For Lévy processes, an explicit way to obtain the auxiliary function and the threshold is given by use of the ladder height processes. Lastly, the connection from discrete and continuous problems and possible approximation of the latter via the former is discussed and the results are applied to sequential tests of power one.

Statistical Causality for Multivariate Nonlinear Time Series via Gaussian Process Models

The ability to test for statistical causality in linear and nonlinear contexts, in stationary or non-stationary settings, and to identify whether statistical causality influences trend of volatility forms a particularly important class of problems to explore in multi-modal and multivariate processes. In this paper, we develop novel testing frameworks for statistical causality in general classes of multivariate nonlinear time series models. Our framework accommodates flexible features where causality may be present in either: trend, volatility or both structural components of the general multivariate Markov processes under study. In addition, we accommodate the added possibilities of flexible structural features such as long memory and persistence in the multivariate processes when applying our semi-parametric approach to causality detection. We design a calibration procedure and formal testing procedure to detect these relationships through classes of Gaussian process models. We provide a generic framework which can be applied to a wide range of problems, including partially observed generalised diffusions or general multivariate linear or nonlinear time series models. We demonstrate several illustrative examples of features that are easily testable under our framework to study the properties of the inference procedure developed including the power of the test, sensitivity and robustness. We then illustrate our method on an interesting real data example from commodity modelling.

Dirichlet Eigenvalues and Exit Time Moments for Symmetric Markov Processes

We give some relationships between the first Dirichlet eigenvalues and the exit time moments for the general symmetric Markov processes. As applications, we present some examples, including symmetric diffusions and $\alpha$-stable processes, and provide the estimates of their first Dirichlet eigenvalues and the exit time moments.

Ergodicity of Markov Processes via Nonstandard Analysis

The Markov chain ergodic theorem is well-understood if either the time-line or the state space is discrete. However, there does not exist a very clear result for general state space continuous-time Markov processes. Using methods from mathematical logic and nonstandard analysis, we introduce a class of hyperfinite Markov processes-namely, general Markov processes which behave like finite state space discrete-time Markov processes. We show that, under moderate conditions, the transition probability of hyperfinite Markov processes align with the transition probability of standard Markov processes. The Markov chain ergodic theorem for hyperfinite Markov processes will then imply the Markov chain ergodic theorem for general state space continuous-time Markov processes.

Approximation of Space-Time Fractional Equations

The aim of this paper is to provide approximation results for space-time non-local equations with general non-local (and fractional) operators in space and time. We consider a general Markov process time changed with general subordinators or inverses to general subordinators. Our analysis is based on Bernstein symbols and Dirichlet forms, where the symbols characterize the time changes, and the Dirichlet forms characterize the Markov processes.

Closed-Form Solutions for the Probability Distribution of Time-Variant Maximal Value Processes for Some Classes of Markov Processes

The time-variant maximal value process (MVP) of a Markov process has significant applications in various science and engineering fields. In the present paper, the closed-form solutions for the probability distribution of the time-variant MVP for some classes of Markov process are studied. For general continuous Markov processes, a unified Volterra integral equation governing the evolution of the cumulative distribution functions (CDFs) of the time-variant MVP of a Markov process is established for the first time. Closed-form or numerical solutions for MVP of some special continuous Markov processes are derived according to this equation. For the compound Poisson process, which is a discontinuous Markov process, the closed-form solution of concentrated probability of the time-variant MVP at zero point is given analytically. Finally, several examples are illustrated as case studies of these theoretical results, demonstrating the effectiveness of the results. Though the analytical results are now only applicable to one-dimensional Markov process, it provides at least some benchmark results for the checking of future possible analytical or numerical methods for the probability density function (PDF) of MVP of more general and high-dimensional Markov process. Further, it provides insights that might stimulate more sophisticated results in the future.

The martingale comparison method for Markov processes

AbstractComparison results for Markov processes with respect to function-class-induced (integral) stochastic orders have a long history. The most general results so far for this problem have been obtained based on the theory of evolution systems on Banach spaces. In this paper we transfer the martingale comparison method, known for the comparison of semimartingales to Markovian semimartingales, to general Markov processes. The basic step of this martingale approach is the derivation of the supermartingale property of the linking process, giving a link between the processes to be compared. This property is achieved using the characterization of Markov processes by the associated martingale problem in an essential way. As a result, the martingale comparison method gives a comparison result for Markov processes under a general alternative but related set of regularity conditions compared to the evolution system approach.

ANALYSIS OF STOCHASTIC PROCESS TO MODEL SAFETY RISK IN CONSTRUCTION INDUSTRY

There are many factors leading to construction safety accident. The rule presented under the influence of these factors should be a statistical random rule. To reveal those random rules and study the probability prediction method of construction safety accident, according to stochastic process theory, general stochastic process, Markov process and normal process are respectively used to simulate the risk-accident process in this paper. First, in the general-random-process-based analysis the probability of accidents in a period of time is calculated. Then, the Markov property of the construction safety risk evolution process is illustrated, and the analytical expression of probability density function of first-passage time of Markov-based risk-accident process is derived to calculate the construction safety probability. In the normal-process-based analysis, the construction safety probability formulas in cases of stationary normal risk process and non-stationary normal risk process with zero mean value are derived respectively. Finally, the number of accidents that may occur on construction site in a period is studied macroscopically based on Poisson process, and the probability distribution of time interval between adjacent accidents and the time of the nth accident are calculated respectively. The results provide useful reference for the prediction and management of construction accidents.

Lyapunov criteria for uniform convergence of conditional distributions of absorbed Markov processes

We study the uniform convergence to quasi-stationarity of multidimensional processes absorbed when one of the coordinates vanishes. Our results cover competitive or weakly cooperative Lotka–Volterra birth and death processes and Feller diffusions with competitive Lotka–Volterra interaction. To this aim, we develop an original non-linear Lyapunov criterion involving two functions, which applies to general Markov processes.