Linear Markov Processes Research Articles

Stochastic modeling of stationary scalar Gaussian processes in continuous time from autocorrelation data

We consider the problem of constructing a vector-valued linear Markov process in continuous time, such that its first coordinate is in good agreement with given samples of the scalar autocorrelation function of an otherwise unknown stationary Gaussian process. This problem has intimate connections to the computation of a passive reduced model of a deterministic time-invariant linear system from given output data in the time domain. We construct the stochastic model in two steps. First, we employ the AAA algorithm to determine a rational function which interpolates the z-transform of the discrete data on the unit circle and use this function to assign the poles of the transfer function of the reduced model. Second, we choose the associated residues as the minimizers of a linear inequality constrained least squares problem which ensures the positivity of the transfer function’s real part for large frequencies. We apply this method to compute extended Markov models for stochastic processes obtained from generalized Langevin dynamics in statistical physics. Numerical examples demonstrate that the algorithm succeeds in determining passive reduced models and that the associated Markov processes provide an excellent match of the given data.

Emergent facilitation and glassy dynamics in supercooled liquids

In supercooled liquids, dynamical facilitation refers to a phenomenon where microscopic motion begets further motion nearby, resulting in spatially heterogeneous dynamics. This is central to the glassy relaxation dynamics of such liquids, which show super-Arrhenius growth of relaxation timescales with decreasing temperature. Despite the importance of dynamical facilitation, there is no theoretical understanding of how facilitation emerges and impacts relaxation dynamics. Here, we present a theory that explains the microscopic origins of dynamical facilitation. We show that dynamics proceeds by localized bond-exchange events, also known as excitations, resulting in the accumulation of elastic stresses with which new excitations can interact. At low temperatures, these elastic interactions dominate and facilitate the creation of new excitations near prior excitations. Using the theory of linear elasticity and Markov processes, we simulate a model, which reproduces multiple aspects of glassy dynamics observed in experiments and molecular simulations, including the stretched exponential decay of relaxation functions, the super-Arrhenius behavior of relaxation timescales as well as their two-dimensional finite-size effects. The model also predicts the subdiffusive behavior of the mean squared displacement (MSD) on short, intermediate timescales. Furthermore, we derive the phonon contributions to diffusion and relaxation, which when combined with the excitation contributions produce the two-step relaxation processes, and the ballistic-subdiffusive-diffusive crossover MSD behaviors commonly found in supercooled liquids.

The entropy rate of Linear Additive Markov Processes.

This work derives a theoretical value for the entropy of a Linear Additive Markov Process (LAMP), an expressive but simple model able to generate sequences with a given autocorrelation structure. Our research establishes that the theoretical entropy rate of a LAMP model is equivalent to the theoretical entropy rate of the underlying first-order Markov Chain. The LAMP model captures complex relationships and long-range dependencies in data with similar expressibility to a higher-order Markov process. While a higher-order Markov process has a polynomial parameter space, a LAMP model is characterised only by a probability distribution and the transition matrix of an underlying first-order Markov Chain. This surprising result can be explained by the information balance between the additional structure imposed by the next state distribution of the LAMP model, and the additional randomness of each new transition. Understanding the entropy of the LAMP model provides a tool to model complex dependencies in data while retaining useful theoretical results. To emphasise the practical applications, we use the LAMP model to estimate the entropy rate of the LastFM, BrightKite, Wikispeedia and Reuters-21578 datasets. We compare estimates calculated using frequency probability estimates, a first-order Markov model and the LAMP model, also considering two approaches to ensure the transition matrix is irreducible. In most cases the LAMP entropy rates are lower than those of the alternatives, suggesting that LAMP model is better at accommodating structural dependencies in the processes, achieving a more accurate estimate of the true entropy.

Non-reversible processes: GENERIC, hypocoercivity and fluctuations

We consider two approaches to study non-reversible Markov processes, namely the hypocoercivity theory and general equations for non-equilibrium reversible–irreversible coupling; the basic idea behind both of them is to split the process into a reversible component and a non-reversible one, and then quantify the way in which they interact. We compare such theories and provide explicit formulas to pass from one formulation to the other; as a bi-product we give a simple proof of the link between reversibility of the dynamics and gradient flow structure of the associated Fokker–Planck equation. We do this both for linear Markov processes and for a class of nonlinear Markov process as well. We then characterise the structure of the large deviation functional of generalised-reversible processes; this is a class of non-reversible processes of large relevance in applications. Finally, we show how our results apply to two classes of Markov processes, namely non-reversible diffusion processes and a class of piecewise deterministic Markov processes (PDMPs), which have recently attracted the attention of the statistical sampling community. In particular, for the PDMPs we consider we prove entropy decay.

Reducing exit-times of diffusions with repulsive interactions

In this work we prove a Kramers’ type law for the low-temperature behavior of the exittimes from a metastable state for a class of self-interacting nonlinear diffusion processes. Contrary to previous works, the interaction is not assumed to be convex, which means that this result covers cases where the exit-time for the interacting process is smaller than the exit-time for the associated non-interacting process. The technique of the proof is based on the fact that, under an appropriate contraction condition, the interacting process is conveniently coupled with a non-interacting (linear) Markov process where the interacting law is replaced by a constant Dirac mass at the fixed point of the deterministic zero-temperature process.

Determination of training limiting levels for specialists of radio engineering complexes during simulation training using the Z-transformation linear Markov process method

Determination of training limiting levels for specialists of radio engineering complexes during simulation training using the Z-transformation linear Markov process method

Understanding causation via correlations and linear response theory

In spite of the (correct) common-wisdom statement correlation does not imply causation, a proper employ of time correlations and of fluctuation-response theory allows to understand the causal relations between the variables of a multi-dimensional linear Markov process. It is shown that the fluctuation-response formalism can be used both to find the direct causal links between the variables of a system and to introduce a degree of causation, cumulative in time, whose physical interpretation is straightforward. Although for generic non-linear dynamics there is no simple exact relationship between correlations and response functions, the described protocol can still give a useful proxy also in presence of weak nonlinear terms.

Modifying an approximation process using non-Newtonian calculus

In the present note we modify a linear positive Markov process of discrete type by using so called multiplicative calculus. In this framework, a convergence property and the error of approximation are established.

Development of a method for determination of ship’s loading time distribution under іrregular cargo arrival

A method for determining the time distribution function of ship berthing under loading under the direct option of loading operations (i. e., without the warehouse) in conditions of irregular cargo delivery by rail is developed. In order to take into account uncertainty and risk factors (random arrivals of loaded cars at the terminal and loading volume of cars), it is proposed to use linear Markov processes that describe the dynamics of train arrival and cargo loading on the ship. It is assumed that the intervals between adjacent arrivals of loaded cars at the terminal are exponential random variables. Cargo transshipment from cars on the ship is carried out at a constant rate. The cases when the volume of cargo in cars is a random or constant variable are considered in detail. To find the probability densities and state probabilities of the corresponding Markov process, a system of linear differential equations and initial conditions is derived. A solution to this system of equations in terms of the Laplace transform is found, in particular, the distribution function of the ship berthing time, taking into account possible interruptions while waiting for cargo delivery by cars. For the case of a constant volume of cargo on cars, the corresponding distribution function of the ship berthing time and its asymptotics for a large deadweight tonnage are also found using the central limit theorem. Based on the results obtained, the problem of finding a criterion of expediency of insuring the risk of exceeding the laytime (contractual) of the ship is formulated. It is proved that the results obtained are important for the theory, as well as for the practice of the port operator and shipping companies, since they can reduce the risk of exceeding the ship berthing time under loading operations. A numerical illustration of the proposed method is given

The Nyquist Issue in Linear Inverse Modeling

AbstractLinear inverse modeling (LIM) is a statistical technique based on covariance statistics that estimates the best-fit linear Markov process to a multivariate time series. An integral, often-ignored part of the technique is a test of whether or not the linear assumptions are valid. One test for linearity is the so-called tau test. While this test can be trusted when it passes, it sometimes fails when it ought to pass. In this article, we discuss one of the reasons for spurious failure, the “Nyquist issue,” which occurs when the lagged covariance matrix used in the analysis is numerically performed at a lag greater than or nearly equal to half the period of a natural mode of variability represented in the time series. As an illustration relevant to a system with many degrees of freedom, but simple enough to solve analytically, we consider a four-dimensional system consisting of two modal pairs. Within this framework, we suggest one solution that can be applied if the time series are long enough. It is hoped that awareness of this issue can prevent misinterpretation of LIM results.

Increasing risk: Dynamic mean-preserving spreads

We extend the celebrated Rothschild and Stiglitz (1970) definition of Mean-Preserving Spreads to a dynamic framework. We adapt the original integral conditions to transition probability densities, and give sufficient conditions for their satisfaction. We then focus on a class of nonlinear scalar diffusion processes, the super-diffusive ballistic process, and prove that it satisfies the integral conditions. We further prove that this class is unique among Brownian bridges. This class of processes can be generated by a random superposition of linear Markov processes with constant drifts. This exceptionally simple representation enables us to systematically revisit, by means of the properties of dynamic mean-preserving spreads, workhorse economic models originally based on White Gaussian Noise. A selection of four examples is presented and explicitly solved.

Controllability Issues for Randomly Switching Piecewise Linear Markov Processes

We investigate some stability properties of approximate null-controllability (A0C). We show that A0C is inherited when increasing the number of observed jumps. However, in all generality, there is no direct connection between A0C with an initial mode and the A0C starting from directly accessible modes. We have also given a sufficient condition under which checking A0C implies AC and discuss some examples. Unfortunately, such conditions are too strong and do not cover all AC systems (see examples). In a second part, we give a Riccati technique-based viability kernel characterisation of A0C. Some conditions are inferred for AC.

On discrete-time stable multiple Markov processes

ABSTRACTIn this paper, we provide necessary conditions for discrete-time symmetric α-stable processes to be linear 2-ple Markov. The aim of this paper is to extend the results given by Adler et al. (1990) to general multiple Markov processes, called linear multiple Markov processes. A necessary and sufficient condition based on the covariation for SαS processes to be linear multiple Markov is provided. A complete description of this class of covariation functions including the stationary case is given. We show that the sum of two independent time-changed Levy motions is linear multiple Markov of order 2. The SαS conditional distributions are also studied, and the conditional characteristic functions of linear 2-ple Markov SαS processes are formed. Finally, we study two famous classes of SαS processes, namely, sub-Gaussian and harmonizable processes, and discuss their Markov properties.

Need for Caution in Interpreting Extreme Weather Statistics

AbstractGiven the reality of anthropogenic global warming, it is tempting to seek an anthropogenic component in any recent change in the statistics of extreme weather. This paper cautions that such efforts may, however, lead to wrong conclusions if the distinctively skewed and heavy-tailed aspects of the probability distributions of daily weather anomalies are ignored or misrepresented. Departures of several standard deviations from the mean, although rare, are far more common in such a distinctively non-Gaussian world than they are in a Gaussian world. This further complicates the problem of detecting changes in tail probabilities from historical records of limited length and accuracy.A possible solution is to exploit the fact that the salient non-Gaussian features of the observed distributions are captured by so-called stochastically generated skewed (SGS) distributions that include Gaussian distributions as special cases. SGS distributions are associated with damped linear Markov processes perturbed by asymmetric stochastic noise and as such represent the simplest physically based prototypes of the observed distributions. The tails of SGS distributions can also be directly linked to generalized extreme value (GEV) and generalized Pareto (GP) distributions. The Markov process model can be used to provide rigorous confidence intervals and to investigate temporal persistence statistics. The procedure is illustrated for assessing changes in the observed distributions of daily wintertime indices of large-scale atmospheric variability in the North Atlantic and North Pacific sectors over the period 1872–2011. No significant changes in these indices are found from the first to the second half of the period.

Algebraic invariance conditions in the study of approximate (null-)controllability of Markov switch processes

We aim at studying approximate null-controllability properties of a particular class of piecewise linear Markov processes (Markovian switch systems). The criteria are given in terms of algebraic invariance and are easily computable. We propose several necessary conditions and a sufficient one. The hierarchy between these conditions is studied via suitable counterexamples. Equivalence criteria are given in abstract form for general dynamics and algebraic form for systems with constant coefficients or continuous switching. The problem is motivated by the study of lysis phenomena in biological organisms and price prediction on spike-driven commodities.

Generalized Mass Action Law and Thermodynamics of Nonlinear Markov Processes

The nonlinear Markov processes are measure-valued dynamical systems which preserve positivity. They can be represented as the law of large numbers limits of general Markov models of interacting particles. In physics, the kinetic equations allow Lyapunov functionals (entropy, free energy, etc.). This may be considered as a sort of inheritance of the Lyapunov functionals from the microscopic master equations. We study nonlinear Markov processes that inherit thermodynamic properties from the microscopic linear Markov processes. We develop the thermodynamics of nonlinear Markov processes and analyze the asymptotic assumption, which are sufficient for this inheritance.

Multiple model moving horizon estimation approach to prognostics in coupled systems

This article presents a novel MM-MHE algorithm for online prediction of the component survival functions based on their usage profiles. The framework employs Cox PHM based on offline and online data for the RUL prediction. The proposed approach has been validated by way of application to data derived from an automotive ETC system simulator. The MM-MHE algorithm shows excellent performance (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and MSE) in the presence of significant measurement noise over all windows and converges to the correct cluster number. The future work includes application of this approach to continuous PID and to account for the uncertainty in RUL estimation. In the near future, by simple transformations, the authors plan on implementing the MM-MHE algorithm for measurements and states between (-∞,∞) and considering the effects of process noise, hence, modifying the cost function accordingly. A potential extension of the Cox PHM framework for prognosis of coupled systems will be to model the coupled survival dynamics as monotone positive linear systems or monotone Markov processes in which the state matrix is a Metzler matrix (i.e., has nonnegative off-diagonal elements).

Alternative interpretations of power-law distributions found in nature

We investigate two inherently different classes of probability density functions (pdfs) that share the common property of power law tails: the α-stable Lévy process and the linear Markov diffusion process with additive and multiplicative Gaussian noise. Dynamical processes described by these distributions cannot be uniquely identified as belonging to one or the other class either by diverging variance due to power-law tails in the pdf or by the possible existence of skew. However, there are distinguishing features that may be found in sufficiently well sampled time series. We examine these features and discuss how they may guide the development of proper approximations to equations of motion underlying dynamical systems. An additional result of this research was the identification of a variable describing the relative importance of the multiplicative and independent additive noise forcing in our linear Markov process. The distribution of this variable is generally skewed, depending on the level of correlation between the additive and multiplicative noise.

The Lévy–Khintchine type operators with variable Lipschitz continuous coefficients generate linear or nonlinear Markov processes and semigroups

Ito’s construction of Markovian solutions to stochastic equations driven by a Lévy noise is extended to nonlinear distribution dependent integrands aiming at the effective construction of linear and nonlinear Markov semigroups and the corresponding processes with a given pseudo-differential generator. It is shown that a conditionally positive integro-differential operator (of the Lévy–Khintchine type) with variable coefficients (diffusion, drift and Lévy measure) depending Lipschitz continuously on its parameters (position and/or its distribution) generates a linear or nonlinear Markov semigroup, where the measures are metricized by the Wasserstein–Kantorovich metrics. This is a non-trivial but natural extension to general Markov processes of a long known fact for ordinary diffusions.

Sequential Control Variates for Functionals of Markov Processes

Using a sequential control variates algorithm, we compute Monte Carlo approximations of solutions of linear partial differential equations connected to linear Markov processes by the Feynman--Kac formula. It includes diffusion processes with or without absorbing/reflecting boundary and jump processes. We prove that the bias and the variance decrease geometrically with the number of steps of our algorithm. Numerical examples show the efficiency of the method on elliptic and parabolic problems.