Continuous Markov Process Research Articles

Potentials of continuous Markov processes and random perturbations

With a scalar potential and a bivector potential, the vector field associated with the drift of a diffusion is decomposed into a generalized gradient field, a field perpendicular to the gradient, and a divergence-free field. We give such decomposition a probabilistic interpretation by introducing cycle velocity from a bivectorial formalism of nonequilibrium thermodynamics. New understandings on the mean rates of thermodynamic quantities are presented. Deterministic dynamical system is further proven to admit a generalized gradient form with the emerged potential as the Lyapunov function by the method of random perturbations.

Closed-Form Solutions for the Probability Distribution of Time-Variant Maximal Value Processes for Some Classes of Markov Processes

The time-variant maximal value process (MVP) of a Markov process has significant applications in various science and engineering fields. In the present paper, the closed-form solutions for the probability distribution of the time-variant MVP for some classes of Markov process are studied. For general continuous Markov processes, a unified Volterra integral equation governing the evolution of the cumulative distribution functions (CDFs) of the time-variant MVP of a Markov process is established for the first time. Closed-form or numerical solutions for MVP of some special continuous Markov processes are derived according to this equation. For the compound Poisson process, which is a discontinuous Markov process, the closed-form solution of concentrated probability of the time-variant MVP at zero point is given analytically. Finally, several examples are illustrated as case studies of these theoretical results, demonstrating the effectiveness of the results. Though the analytical results are now only applicable to one-dimensional Markov process, it provides at least some benchmark results for the checking of future possible analytical or numerical methods for the probability density function (PDF) of MVP of more general and high-dimensional Markov process. Further, it provides insights that might stimulate more sophisticated results in the future.

Dynamic reduction of time and cost uncertainties in tunneling projects

The cost and time estimation play a vital role in successfully completing a tunnel construction, however in most cases the estimation falls short compared to the actual time and cost that a tunnel construction consumes due to a number of geological and geotechnical reasons. This creates uncertainties in tunnel construction in terms time and cost. In this article, the effects of geological/geotechnical uncertainties in tunnel construction time and costs are being minimized through continuous updating techniques. The forecasting of geological conditions is based on continuous space-discrete state Markov process. Whereas, predicting the construction time and costs are based on Monte-Carlo (MC) simulation. To verify the tool applicability, it has been applied to Hamru tunnel in Iran. Also, to assess the updating effect on obtained outcomes during construction, the tool has been updated two times. In the first update, the total uncertainty in construction time and costs (measured by standard deviation) has been reduced by about 45% and 52%, respectively, followed by a reduction by about 66% and 61% in the second update. Finally, we can conclude that, the presented methodology is a helpful tool for tunnel project managers to identify risks and the possibility of deviations in their construction time and costs.

Optimal Equilibria for Multidimensional Time-Inconsistent Stopping Problems

We study an optimal stopping problem under non-exponential discounting, where the state process is a multi-dimensional continuous strong Markov process. The discount function is taken to be log sub-additive, capturing decreasing impatience in behavioral economics. On strength of probabilistic potential theory, we establish the existence of an optimal equilibrium among a sufficiently large collection of equilibria, consisting of finely closed equilibria satisfying a boundary condition. This generalizes the existence of optimal equilibria for one-dimensional stopping problems in prior literature.

МАТЕМАТИЧЕСКАЯ МОДЕЛЬ ИНФОРМАЦИОННОЙ ПОДДЕРЖКИ ОПЕРАТИВНЫХ ГРУПП ПОЖАРНО-СПАСАТЕЛЬНОГО ГАРНИЗОНА

Support for management decision-making in emergencies (firefighting), in uncertain conditions of its occurrence and uncertainty of its development is necessary for the organization of optimal distribution of the forces and resources of operational units involved in firefighting, and the knowledge of the Manager. The process of our research is limited only to the case of fire. Taking into account the fact that the data in the fire-extinguishing card does not always coincide with the state of the object where the fire occurred and the unpredictability of the direction of fire propagation, the decision-maker in this situation will organize work to obtain reliable information. These features allow us to consider the object where the fire occurred as a complex system. In this case, the main difficulty of information and analytical support for operational groups, as a management entity, is that at the first stage, the decision-maker in the operational group, without having complete reliable information, organizes its clarification. With the correct organization of their work, the task force ensures maximum efficiency of using the units of the Ministry of emergencies of Russia, when solving their combat tasks. At the same time, it can collect information about an emergency when it receives a notification of its occurrence, as well as on the way of the operational units to the specified area. The issues that determine the development of documents in different periods of the operational situation are considered, and the actions of the decision-maker in the operational group during fire extinguishing are highlighted separately. In this mathematical model, in which the transition of the system from the state of “fire occurrence” to the state of “fire extinguished” is modeled, the properties of a continuous Markov process are used. The solution of Kolmogorov’s system of differential equations determines the probability that characterizes the competence of the decision-maker in the task force.

Multiple barrier-crossings of an Ornstein-Uhlenbeck diffusion in consecutive periods

We investigate the joint distribution and the multivariate survival functions for the maxima of an Ornstein-Uhlenbeck (OU) process in consecutive time-intervals. A PDE method, alongside an eigenfunction expansion, is adopted with which we first calculate the distribution and the survival functions for the maximum of a homogeneous OU-process in a single interval. By a deterministic time-change and a parameter translation, this result can be extended to an inhomogeneous OU-process. Next, we derive a general formula for the joint distribution and the survival functions for the maxima of a continuous Markov process in consecutive periods. With these results, one can obtain semi-analytical expressions for the joint distribution and the multivariate survival functions for the maxima of an OU-process, with piecewise constant parameter functions, in consecutive time periods. The joint distribution and the survival functions can be evaluated numerically by an iterated quadrature scheme, which can be implemented efficiently by matrix multiplications. Moreover, we show that the computation can be further simplified to the product of single quadratures by imposing a mild condition. Such results may be used for the modeling of heatwaves and related risk management challenges.

Wasserstein convergence rates for random bit approximations of continuous Markov processes

We determine the convergence speed of a numerical scheme for approximating one-dimensional continuous strong Markov processes. The scheme is based on the construction of certain Markov chains whose laws can be embedded into the process with a sequence of stopping times. Under a mild condition on the process' speed measure we prove that the approximating Markov chains converge at fixed times at the rate of 1/4 with respect to every p-th Wasserstein distance. For the convergence of paths, we prove any rate strictly smaller than 1/4. Our results apply, in particular, to processes with irregular behavior such as solutions of SDEs with irregular coefficients and processes with sticky points.

Методы обоснования количественного состава и оценки значений показателей надежности технических объектов вычислительной сети летательных аппаратов

When designing complex organizational and technical systems, one of the main quality criteria is to meet the requirements for indicators characterizing the operational and technical level of the system. The task of substantiating these requirements has not been systematically approached. Requirements, most often, are presented empirically or using the apparatus of expert analysis. This is due to problems in the field of formalization of processes occurring in complex technical systems, difficulties arising in the construction of mathematical models and the development of criteria that allow optimizing information, technical, software, linguistic and other structures of ACS. The problem of substantiating the requirements should be solved using a systematic approach, and the whole set of properties, integral and partial indicators of the system should be taken into account. The study focuses on the approach to solving the problem of substantiating the requirements for reliability indicators of technical means of an organizational and technical system using the apparatus of continuous Markov processes. Time between failures and recovery time were used as reliability indicators. As a result, we found an analytical solution to the problem of substantiating the requirements for the reliability indicators of technical means and the characteristics of their performance, as well as the number of computer network workstations, necessary to meet the integral requirements for the ACS.

On the Correlation Function of an Arbitrary Distributed Continuous Markov Process

Continuous Markov processes widely used as a tool for modeling random phenomena in numerous applications, can be defined as solutions of generally nonlinear stochastic differential equations (SDEs) with certain drift and diffusion coefficients which together governs the process’ probability density and correlation functions. Usually it is assumed that the diffusion coefficient does not depend on the process' current value. Sometimes, in particular for presentation of non- Gaussian real processes this assumption becomes undesirable, leads generally to complexity of the correlation function estimation. We consider its analysis for the process with arbitrary pair of the drift and diffusion coefficients providing the given stationary probability distribution of the considered process.

Local densities for a class of degenerate diffusions

Dans cet article on etudie une classe de processus de type Markov fort continus a valeurs dans $\mathbb{R}^{d}$ qui sont engendres, seulement localement, par un operateur ultra-parabolique avec des coefficients reguliers par rapport a la geometrie intrinseque induite par l’operateur lui-meme et non par rapport a la geometrie euclidienne. On obtient deux resultats importants. Le premier est une formule locale d’Ito valable pour des fonctions qui ne sont pas deux fois differentiables dans le sens classique, mais seulement intrinsequement par rapport a un ensemble de champs de vecteurs, lies au generateur, satisfaisant a la condition d’Hormander. La deuxieme contribution, qui s’appuie sur la premiere, est un resultat d’existence et de regularite pour la densite de transition locale.

Approximating exit times of continuous Markov processes

The time at which a one-dimensional continuous strong Markov process attains a boundary point of its state space is a discontinuous path functional and it is, therefore, unclear whether the exit time can be approximated by hitting times of approximations of the process. We prove a functional limit theorem for approximating weakly both the paths of the Markov process and its exit times. In contrast to the functional limit theorem in [ 3 ] for approximating the paths, we impose a stronger assumption here. This is essential, as we present an example showing that the theorem extended with the convergence of the exit times does not hold under the assumption in [ 3 ]. However, the EMCEL scheme introduced in [ 3 ] satisfies the assumption of our theorem, and hence we have a scheme capable of approximating both the process and its exit times for every one-dimensional continuous strong Markov process, even with irregular behavior (e.g., a solution of an SDE with irregular coefficients or a Markov process with sticky features). Moreover, our main result can be used to check for some other schemes whether the exit times converge. As an application we verify that the weak Euler scheme is capable of approximating the absorption time of the CEV diffusion and that the scale-transformed weak Euler scheme for a squared Bessel process is capable of approximating the time when the squared Bessel process hits zero.

The What, When and Where of Limit Order Books

We model the limit order book (LOB) as a continuous Markov process and develop an algebra to describe its dynamics based on the fundamental events of the book: order arrivals and cancellations. We show how all observables (prices, returns, and liquidity measures) are governed by the same variables which also drive arrival and cancellation rates.The sensitivity of our model is evaluated in a simulation study and an empirical analysis. We estimate several linearized model specifications based on the theoretical description of the LOB and conduct in- and out-of-sample forecasts on several frequencies. The in-sample results based on contemporaneous information suggest that our model describes up to 90% of the variation of close-to-close returns, the adjusted $R^2$ still ranges at around 80%. In the more realistic setting where only past information enters the model, we still observe an adjusted $R^2$ in the range of 15%. The direction of the next return can be predicted, out-of-sample, with an accuracy of over 75\% for short time horizons below 10 minutes. Out-of-sample, on average, we obtain $R^2$ values for the Mincer-Zarnowitz regression of around 2-3% and an $RMSPE$ that is 10 times lower than values documented in the literature.

Lyapunov Exponent of Rank-One Matrices: Ergodic Formula and Inapproximability of the Optimal Distribution

The Lyapunov exponent corresponding to a set of square matrices $\mathcal{A} = \{A_1, \dots, A_n \}$ and a probability distribution $p$ over $\{1, \dots, n\}$ is $\lambda(\mathcal{A},p) := \lim_{k \to \infty} \frac{1}{k} \,\mathbb{E} \log \|A_{\sigma_k} \cdots A_{\sigma_2}A_{\sigma_1}\|$, where $\sigma_i$ are i.i.d. according to $p$. This quantity is of fundamental importance to control theory since it determines the asymptotic convergence rate $e^{\lambda(\mathcal{A},p)}$ of the stochastic linear dynamical system $x_{k+1} = A_{\sigma_k} x_k$. This paper investigates the following "design problem": given $\mathcal{A}$, compute the distribution $p$ minimizing $\lambda(\mathcal{A},p)$. Our main result is that it is NP-hard to decide whether there exists a distribution $p$ for which $\lambda(\mathcal{A},p)< 0$, i.e. it is NP-hard to decide whether this dynamical system can be stabilized. This hardness result holds even in the "simple"' case where $\mathcal{A}$ contains only rank-one matrices. Somewhat surprisingly, this is in stark contrast to the Joint Spectral Radius -- the deterministic kindred of the Lyapunov exponent -- for which the analogous optimization problem for rank-one matrices is known to be exactly computable in polynomial time. To prove this hardness result, we first observe via Birkhoff's Ergodic Theorem that the Lyapunov exponent of rank-one matrices admits a simple formula and in fact is a quadratic form in $p$. Hardness of the design problem is shown through a reduction from the Independent Set problem. Along the way, simple examples are given illustrating that $p \mapsto \lambda(\mathcal{A},p)$ is neither convex nor concave in general. We conclude with extensions to continuous distributions, exchangeable processes, Markov processes, and stationary ergodic processes.

Mathematical model filtering canonical parameters of satellite-repeater in orbital motion

For the effective solution of the problem estimation of canonical parameters the satellite-repeater at orbital motion the substantial statement within which the generalized scheme of model filtration of coordinates and a vector of speed movement the satellite-repeater is given is formulated. The system of restrictions and assumptions is introduced. When the state vector is represented by a continuous vector Markov process, the generalized equations of state and observations in kinematic variables are determined. Using the methods of numerical differentiation, schemes for solving ordinary differential equations, optimization, linearization of the original filtration problem in determining the rules for calculating the evolution and measurement matrices is performed. The efficiency of the generated algorithmic solutions is tested on specific examples.

Nonparametric tests for transition probabilities in nonhomogeneous Markov processes

This paper proposes nonparametric two-sample tests for the direct comparison of the probabilities of a particular transition between states of a continuous time nonhomogeneous Markov process with a finite state space. The proposed tests are a linear nonparametric test, an -norm-based test and a Kolmogorov–Smirnov-type test. Significance level assessment is based on rigorous procedures, which are justified through the use of modern empirical process theory. Moreover, the -norm and the Kolmogorov–Smirnov-type tests are shown to be consistent for every fixed alternative hypothesis. The proposed tests are also extended to more complex situations such as cases with incompletely observed absorbing states and non-Markov processes. Simulation studies show that the test statistics perform well even with small sample sizes. Finally, the proposed tests are applied to data on the treatment of early breast cancer from the European Organization for Research and Treatment of Cancer (EORTC) trial 10854, under an illness-death model.

Mean-Square Non-Local Stability of Ship in Storm Conditions of Operation

Abstract The purpose of the paper is to create a method for studying nonlocal stability in the mean and in the mean square of the ship, positioned on the beam of an intensive wind–waves mode, which is based on the use of the correlation theory of random functions close to continuous Markov processes. With the help of this method and the integral formula of event probability, a method for determining the reliability indicator of the ship in respect of the existing wind–waves excitations of the operating area is formed. An example of investigating the nonlinear motion of the ship, determining its local and nonlocal stability in the first approximation of the theory of considered random functions, is given. Such approximation uses correlation theory with models of acting excitations represented by the generalised derivatives of the Wiener process. Moreover, special attention is paid to reflecting the connection of the proposed methods for investigating the ship stability under constantly acting random excitations with the traditional methods of studying ship stability at small and large inclinations. The established connection defines the proposed methods as a development of the traditional methods of ship stability deterministic theory during the transition to its formation in the class of random functions, with the addition to these methods of the missing link of determining the level of reliability of ships towards the acting wind–waves excitations of the operation area.

A Markov Model for Non-lognormal Density Distributions in Compressive Isothermal Turbulence

Abstract Compressive isothermal turbulence is known to have a near lognormal density probability distribution function (PDF) with a width that scales with the sonic Mach number and nature of the turbulent driving (solenoidal versus compressive). However, the physical processes that mold the extreme high and low density structures in a turbulent medium can be different, with the densest structures being composed of strong shocks that evolve on shorter timescales than the low density fluid. The density PDF in a turbulent medium exhibits deviations from lognormal due to shocks, that increases with the sonic Mach number, which is often ignored in analytic models for turbulence and star formation. We develop a simple model for turbulence by treating it as a continuous Markov process, which explains both the density PDF and the transient timescales of structures as a function of density, using a framework developed in Scannapieco &amp; Safarzadeh (2018). Our analytic model depends on only a single parameter, the effective compressive sonic Mach number, and successfully describes the non-lognormal behavior seen in both 1D and 3D simulations of supersonic and subsonic compressive isothermal turbulence. The model quantifies the non-lognormal distribution of density structures in turbulent environments, and has application to star-forming molecular clouds and star formation efficiencies.

Affinity-dependent bound on the spectrum of stochastic matrices

Affinity has proven to be a useful tool for quantifying the non-equilibrium character of time continuous Markov processes since it serves as a measure for the breaking of time reversal symmetry. It has recently been conjectured that the number of coherent oscillations, which is given by the ratio of imaginary and real part of the first non-trivial eigenvalue of the corresponding master matrix, is constrained by the maximum cycle affinity present in the network. In this paper, we conjecture a bound on the whole spectrum of these master matrices that constrains all eigenvalues in a fashion similar to the well known Perron–Frobenius theorem that is valid for any stochastic matrix. As in other studies that are based on affinity-dependent bounds, the limiting process that saturates the bound is given by the asymmetric random walk. For unicyclic networks, we prove that it is not possible to violate the bound by small perturbation of the asymmetric random walk and provide numerical evidence for its validity in randomly generated networks. The results are extended to multicyclic networks, backed up by numerical evidence provided by networks with randomly constructed topology and transition rates.

The Zig-Zag process and super-efficient sampling for Bayesian analysis of big data

Standard MCMC methods can scale poorly to big data settings due to the need to evaluate the likelihood at each iteration. There have been a number of approximate MCMC algorithms that use sub-sampling ideas to reduce this computational burden, but with the drawback that these algorithms no longer target the true posterior distribution. We introduce a new family of Monte Carlo methods based upon a multi-dimensional version of the Zig-Zag process of (Bierkens, Roberts, 2017), a continuous time piecewise deterministic Markov process. While traditional MCMC methods are reversible by construction (a property which is known to inhibit rapid convergence) the Zig-Zag process offers a flexible non-reversible alternative which we observe to often have favourable convergence properties. We show how the Zig-Zag process can be simulated without discretisation error, and give conditions for the process to be ergodic. Most importantly, we introduce a sub-sampling version of the Zig-Zag process that is an example of an {\em exact approximate scheme}, i.e. the resulting approximate process still has the posterior as its stationary distribution. Furthermore, if we use a control-variate idea to reduce the variance of our unbiased estimator, then the Zig-Zag process can be super-efficient: after an initial pre-processing step, essentially independent samples from the posterior distribution are obtained at a computational cost which does not depend on the size of the data.

Multiple Barrier-Crossings of an Ornstein-Uhlenbeck Diffusion in Consecutive Periods

We investigate the joint distribution and the multivariate survival functions for the maxima of an Ornstein-Uhlenbeck (OU) process in consecutive time-intervals. A PDE method, alongside an eigenfunction expansion, is adopted with which we first calculate the distribution and the survival functions for the maximum of a homogeneous OU-process in a single interval. By a deterministic time-change and a parameter translation, this result can be extended to an inhomogeneous OU-process. Next, we derive a general formula for the joint distribution and the survival functions for the maxima of a continuous Markov process in consecutive periods. With these results, one can obtain semi-analytical expressions for the joint distribution and the multi- variate survival functions for the maxima of an OU-process, with piecewise constant parameter functions, in consecutive time periods. The joint distribution and the survival functions can be evaluated numerically by an iterated quadrature scheme, which can be implemented efficiently by matrix multiplications. Moreover, we show that the computation can be further simplified to the product of single quadratures by imposing a mild condition. Such results may be used for the modelling of heatwaves and related risk management challenges.