Continuous-time Process Research Articles

Optimality of a Network Monitoring Agent and Validation in a Real Probe

The evolution of commodity hardware makes it possible to use this type of equipment to implement traffic monitoring systems. A preliminary empirical evaluation of a network traffic probe based on Linux indicates that the system performance has significant losses as the network rate increases. To assess this issue, we consider a model with two tandem queues and a moving server. In this system, we formulate a three-dimensional Markov Decision Process in continuous time. The goal of the proposed model is to determine the position of the server in each time slot so as to optimize the system performance which is measured in terms of throughput. We first formulate an equivalent discrete-time Markov Decision Process and we propose a numerical method to characterize the solution of our problem in a general setting. The solution we obtain in this problem has been tested for a wide range of scenarios and, in all the instances, we observe that the optimality is close to a threshold type policy. We also consider a real probe and we validate the good performance of threshold policies in real applications.

Innovation Development of an Enterprise: Modeling Dynamics

At the present stage of economic development, the leading role in ensuring the competitiveness of both an singular enterprise and the country as a whole, as well as in creating conditions for the transition to sustainable development, is played by the successful implementation of the latest scientific developments in production processes, comprehensive support for the strategy of innovative development. The development and implementation of innovations is a complex dynamic process that requires the use of special research methods. Such a method is system dynamics, which makes it possible to take into account the nonlinearity of the impact of innovation on the state of the economy. The paper considers the methodology for building a model for managing innovation processes, taking into account the self-organization of the logistic type. The main danger that can accompany the evolution of innovation processes is the emergence of unacceptable dynamic modes, so one of the tasks of the study was to determine the conditions capable of ensuring the stability of equilibrium states of a complex dynamic system according to the proposed models. The object of research is a complex dynamic system, between the elements of which there is both positive and negative feedback. To build a model of the dynamics of innovation processes, the mathematical apparatus of the theory of differential equations was applied, which made it possible to consider the development of the innovation process in continuous time. With the help of the instrumentarium of nonlinear dynamics, a study of the stability of diffusion of innovations depending on the parameters of the control influence was carried out. The conditions for the transition of the system to a critical state, which may be accompanied by the occurrence of bifurcations and chaos, have been determined. Particular attention was paid to determining the structural stability of the regulated innovation process in the case when both equilibrium positions are close in terms of parameter values. It is expedient to apply the proposed model to solve the problem of innovation management both at the State level and at the level of an individual industry or an individual enterprise. The obtained theoretical conclusions were confirmed through the use of simulation modeling.

Exponential quasi-ergodicity for processes with discontinuous trajectories

This paper tackles the issue of establishing an upper-bound on the asymptotic ratio of survival probabilities between two different initial conditions, asymptotically in time for a given Markov process with extinction. Such a comparison is a crucial step in recent techniques for proving exponential convergence to a quasi-stationary distribution. We introduce a weak form of the Harnack’s inequality as the essential ingredient for such a comparison. This property is actually a consequence of the convergence property that we intend to prove. Its complexity appears as the price to pay for the level of flexibility required by our applications, notably for processes with jumps on a multidimensional state-space. We show in our illustrations how simply and efficiently it can be used nonetheless. As illustrations, we consider two continuous-time processes on ℝd that do not satisfy the classical Harnack’s inequality, even in a local version. The first one is a piecewise deterministic process while the second is a pure jump process with restrictions on the directions of its jumps.

Distances in a geographical attachment network model

Distances between nodes are one of the most essential subjects in the study of complex networks. In this paper, we investigate the asymptotic behaviors of two types of distances in a model of geographic attachment networks: the typical distance and the flooding time. By generating an auxiliary tree and using a continuous-time branching process, we demonstrate that in this model the typical distance is asymptotically normal, and the flooding time converges to a given constant in probability as well.

An exact and explicit formula for pricing lookback options with regime switching

<p style='text-indent:20px;'>This paper investigates the pricing of European-style lookback options when the price dynamics of the underlying risky asset are assumed to follow a Markov-modulated Geometric Brownian motion; that is, the appreciation rate and the volatility of the underlying risky asset depend on states of the economy described by a continuous-time Markov chain process. We derive an exact, explicit and closed-form solution for European-style lookback options in a two-state regime switching model.</p>

Drift in Ratios of Continuous-Time Stochastic Processes, Implications for Financial Analysis, a Note

Drift in Ratios of Continuous-Time Stochastic Processes, Implications for Financial Analysis, a Note

General criteria for the study of quasi-stationarity

For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in weighted total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of attraction by an integrability condition, prove the existence of a right eigenvector for the semigroup of the process and the existence and exponential ergodicity of the Q-process. These results are applied to one-dimensional and multi-dimensional diffusion processes, to pure jump continuous time processes, to reducible processes with several communication classes, to perturbed dynamical systems and discrete time processes evolving in discrete state spaces.

Approximation of the invariant distribution for a class of ergodic SDEs with one-sided Lipschitz continuous drift coefficient using an explicit tamed Euler scheme

We study the behavior in a large time regime of an explicit tamed Euler-Maruyama scheme applied to a class of ergodic Ito stochastic differential equations with one-sided Lipschitz continuous drift coefficient and bounded globally Lipschitz diffusion coefficient. Our first main contribution is to prove moments for the numerical scheme, which, on the one hand, are uniform with respect to the time-step size, and which, on the other hand, may not be uniform but have at most polynomial growth with respect to time. Our second main contribution is to apply this result to obtain weak error estimates to quantify the error to approximate averages with respect to the invariant distribution of the continuous-time process, as a function of the time-step size and of the time horizon. The explicit tamed Euler scheme is shown to be computationally effective for the approximation of the invariant distribution: even if the moment bounds and error estimates are not proved to be uniform with respect to time, the obtained polynomial growth results in a marginal increase in the upper bound of the computational cost. To the best of our knowledge, this is the first result in the literature concerning the approximation of the invariant distribution for stochastic differential equations with non-globally Lipschitz coefficients using an explicit tamed Euler-Maruyama scheme.

Monotonicity and phase transition for the VRJP and the ERRW

The vertex-reinforced jump process (VRJP), introduced by Davis and Volkov, is a continuous-time process that tends to come back to already visited vertices. It is closely linked to the edge-reinforced random walk (ERRW) introduced by Coppersmith and Diaconis in 1986 which is more likely to traverse edges it has already traversed. On \mathbb{Z}^d for d\geq 3 , both models were shown to be recurrent for small enough initial weights and transient for large enough initial weights. We show through a coupling of VRJPs for different weights that the VRJP (and the ERRW) exhibits some monotonicity. In particular, we show that increasing the initial weights of the VRJP and the ERRW makes them more transient, which means that the recurrence/transience phase transition is necessarily unique. Furthermore, by making the weights go to infinity, we show that the recurrence of the ERRW and the VRJP is implied by the recurrence of a random walk in a deterministic electrical network.

Mixtures of Gaussian Processes for Robot Motion Planning Using Stochastic Trajectory Optimization

Robot motion planning methods based on trajectory optimization can efficiently generate feasible and optimal trajectories by minimizing a suitable cost function, even in high-dimensional spaces. However, the main drawback of these methods lies in their proneness to infeasible local minima, especially in complex environments. To mitigate this issue, we propose a novel motion planning method that represents trajectories as samples from a mixture of continuous-time Gaussian processes (MGP) and employs stochastic optimization in order to update the MGP parameters in a cost-minimizing manner. The contributions of the proposed trajectory optimization method arise from the introduced mixture representation and stochastic gradient estimation, dominantly enabling better exploration of the trajectory space and including nondifferentiable optimizing costs. We evaluated the proposed method in multiple simulation benchmarks featuring 7 degree-of-freedom (DOF) robot arms and a 10DOF mobile manipulator. We also conducted a real-world experiment with a 14DOF dual-arm robot. The experimental results demonstrated that the proposed method achieves higher success rate than several state-of-the-art methods, while the advantages stemming from MGPs and stochastic optimization, like trajectory smoothness, support of nondifferentiable cost functions, multiple trajectory solutions, and the ability to tackle high-dimensional planning problems, are inherently kept.

Squirrels can remember little: A random walk with jump reversals induced by a discrete-time renewal process

We consider a class of discrete-time random walks with directed unit steps on the integer line. The direction of the steps is reversed at the time instants of events in a discrete-time renewal process and is maintained at uneventful time instants. This model represents a discrete-time semi-Markovian generalization of the telegraph process. We derive exact formulae for the propagator using generating functions. We prove that for geometrically distributed waiting times in the diffusive limit, this walk converges to the classical telegraph process. We consider the large-time asymptotics of the expected position: For waiting time densities with finite mean the walker remains in the average localized close to the departure site whereas escapes for fat-tailed waiting-time densities (i.e. densities with infinite mean) by a sublinear power-law. We explore anomalous diffusion features by accounting for the ‘aging effect’ as a hallmark of non-Markovianity where the discrete-time version of the ‘aging renewal process’ comes into play. By deriving pertinent distributions of this process we obtain explicit formulae for the variance when the waiting-times are Sibuya-distributed. In this case and generally for fat-tailed waiting time PDFs a t2-ballistic superdiffusive scaling emerges in the large time limit. In contrast if the waiting time PDF between the step reversals is light-tailed (‘narrow’ with finite mean and variance) the walk exhibits normal diffusion and for ‘broad’ waiting time PDFs (with finite mean and infinite variance) superdiffusive large time scaling. We also consider time-changed versions where the walk is subordinated to a continuous-time point process such as the time-fractional Poisson process. This defines a new class of biased continuous-time random walks exhibiting several regimes of anomalous diffusion.

Asymptotic F test in regressions with observations collected at high frequency over long span

This paper proposes tests of linear hypotheses when the variables may be continuous-time processes with observations collected at a high sampling frequency over a long span. Utilizing series long run variance (LRV) estimation in place of the traditional kernel LRV estimation, we develop easy-to-implement and more accurate F tests in both stationary and nonstationary environments. The nonstationary environment accommodates exogenous regressors that are general semimartingales. Endogenous regressors are allowed in a nonstationary environment similar to cointegration models in the usual discrete-time setting. The F tests can be implemented in exactly the same way as in the discrete-time setting. The F tests are, therefore, robust to the continuous-time or discrete-time nature of the data. Simulations demonstrate the improved size accuracy and competitive power of the F tests relative to existing continuous-time testing procedures and their improved versions. The F tests are of practical interest as recent work by Chang et al. (2021) demonstrates that traditional inference methods can become invalid and produce spurious results when continuous-time processes are observed on finer grids over a long span.

Railway Bridge Management System Based on Visual Inspections with Semi-Markov Continuous Time Process

This article describes a model for predicting the degradation of in-service railway bridges based on a semi-Markov continuous time process. This model relies on the history of inspections of 588 bridges located on a heavy-haul railway line in Brazil, between 2016 and 2020. A dedicated computational tool developed in Matlab allows the automated data processing. A parametric study is performed to understand which factors derived from the bridge structural characteristics, as well as operational and environmental factors, most influence the deterioration model. The type of material proves to be a decisive factor and therefore two specific prediction models are stablished, one for concrete bridges and other to steel bridges. The prediction models have an efficiency equal to 93.7%, for concrete bridges, and 95.1% for steel bridges. Additionally, the analysis of several preventive and corrective maintenance scenarios, specifically for concrete bridges, allows to optimize the condition ratings during the life cycle.

Automatic Zig-Zag sampling in practice

Novel Monte Carlo methods to generate samples from a target distribution, such as a posterior from a Bayesian analysis, have rapidly expanded in the past decade. Algorithms based on Piecewise Deterministic Markov Processes (PDMPs), non-reversible continuous-time processes, are developing into their own research branch, thanks their important properties (e.g., super-efficiency). Nevertheless, practice has not caught up with the theory in this field, and the use of PDMPs to solve applied problems is not widespread. This might be due, firstly, to several implementational challenges that PDMP-based samplers present with and, secondly, to the lack of papers that showcase the methods and implementations in applied settings. Here, we address both these issues using one of the most promising PDMPs, the Zig-Zag sampler, as an archetypal example. After an explanation of the key elements of the Zig-Zag sampler, its implementation challenges are exposed and addressed. Specifically, the formulation of an algorithm that draws samples from a target distribution of interest is provided. Notably, the only requirement of the algorithm is a closed-form differentiable function to evaluate the log-target density of interest, and, unlike previous implementations, no further information on the target is needed. The performance of the algorithm is evaluated against canonical Hamiltonian Monte Carlo, and it is proven to be competitive, in simulation and real-data settings. Lastly, we demonstrate that the super-efficiency property, i.e. the ability to draw one independent sample at a lesser cost than evaluating the likelihood of all the data, can be obtained in practice.

On decay–surge population models

AbstractWe consider continuous space–time decay–surge population models, which are semi-stochastic processes for which deterministically declining populations, bound to fade away, are reinvigorated at random times by bursts or surges of random sizes. In a particular separable framework (in a sense made precise below) we provide explicit formulae for the scale (or harmonic) function and the speed measure of the process. The behavior of the scale function at infinity allows us to formulate conditions under which such processes either explode or are transient at infinity, or Harris recurrent. A description of the structures of both the discrete-time embedded chain and extreme record chain of such continuous-time processes is supplied.

Empirical spectral processes for stationary state space models

In this paper, we consider function-indexed normalized weighted integrated periodograms for equidistantly sampled multivariate continuous-time state space models which are multivariate continuous-time ARMA processes. Thereby, the sampling distance is fixed and the driving Lévy process has at least a finite fourth moment. Under different assumptions on the function space and the moments of the driving Lévy process we derive a central limit theorem for the function-indexed normalized weighted integrated periodogram. Either the assumption on the function space or the assumption on the existence of moments of the Lévy process is weaker. Furthermore, we show the weak convergence in both the space of continuous functions and in the dual space to a Gaussian process and give an explicit representation of the covariance function. The results can be used to derive the asymptotic behavior of the Whittle estimator and to construct goodness-of-fit test statistics as the Grenander–Rosenblatt statistic and the Cramér–von Mises statistic. We present the exact limit distributions of both statistics and show their performance through a simulation study.

Functional Limit Theorem for the Sums of PSI-Processes with Random Intensities

We consider a sequence of i.i.d. random variables, (ξ)=(ξi)i=0,1,2,⋯, Eξ0=0, Eξ02=1, and subordinate it by a doubly stochastic Poisson process Π(λt), where λ≥0 is a random variable and Π is a standard Poisson process. The subordinated continuous time process ψ(t)=ξΠ(λt) is known as the PSI-process. Elements of the triplet (Π,λ,(ξ)) are supposed to be independent. For sums of n, independent copies of such processes, normalized by n, we establish a functional limit theorem in the Skorokhod space D[0,T], for any T>0, under the assumption E|ξ0|2h<∞ for some h>1/γ2. Here, γ∈(0,1] reflects the tail behavior of the distribution of λ, in particular, γ≡1 when Eλ<∞. The limit process is a stationary Gaussian process with the covariance function Ee−λu, u≥0. As a sample application, we construct a martingale from the PSI-process and establish a convergence of normalized cumulative sums of such i.i.d. martingales.

Importance measures for critical components in complex system based on Copula Hierarchical Bayesian Network

In order to identify the vulnerable components and ensure the required reliability of mechatronics systems, importance measures of critical components are crucially used in the early design of systems. However, complex mechatronics systems have the properties of hierarchy, nonlinearity, dependency, uncertainty, and randomness, which make it difficult to analyze the coupling failure mechanisms, model the system, estimate its reliability, and complete importance measures of its components. This paper proposes importance measures for components with continuous time degradation. The Wiener process model is used to describe the continuous-time degradation process, and the Copula Hierarchical Bayesian Network (CHBN) is developed for system reliability estimation. Six importance measures are proposed for continuous-time degrading components. These importance measures provide a time-dependent analysis of the criticality of components, thus adding insights on the contributions of the components on the system reliability or performance over time. A case study on the harmonic gear drive is then conducted to demonstrate the use of the proposed importance measures. The results of the study show that the CHBN-based importance measures can be a valuable decision-support tool for designers in the early design of systems.

Continuous-time targeted minimum loss-based estimation of intervention-specific mean outcomes

This paper generalizes the targeted minimum loss-based estimation (TMLE) framework to allow for estimating the effects of time-varying interventions in settings where both interventions, covariates, and outcome can happen at subject-specific time-points on an arbitrarily fine time-scale. TMLE is a general template for constructing asymptotically linear substitution estimators for smooth low-dimensional parameters in infinite-dimensional models. Existing longitudinal TMLE methods are developed for data where observations are made on a discrete time-grid. We consider a continuous-time counting process model where intensity measures track the monitoring of subjects, and focus on a low-dimensional target parameter defined as the intervention-specific mean outcome at the end of follow-up. To construct our TMLE algorithm for the given statistical estimation problem, we derive an expression for the efficient influence curve and represent the target parameter as a functional of intensities and conditional expectations. The high-dimensional nuisance parameters of our model are estimated and updated in an iterative manner according to separate targeting steps for the involved intensities and conditional expectations. The resulting estimator solves the efficient influence curve equation. We state a general efficiency theorem and describe a highly adaptive lasso estimator for nuisance parameters that allows us to establish asymptotic linearity and efficiency of our estimator under minimal conditions on the underlying statistical model.

A closed-form expansion for the conditional expectations of the extended CIR process

This paper derives a closed-form expansion for the conditional expectation of a continuous-time stochastic process, given by Vt,T:=e−∫tTg(vs)dsf(vT) for 0≤t≤T, where vt evolves according to the extended Cox-Ingersoll-Ross process, for any C∞ functions f and g. We apply the Feynman-Kac theorem to state a Cauchy problem associated with Vt,T and solve the problem by using the reduction method. Furthermore, we extend our method to any piecewise C∞ function f; demonstrating our method can be applied to price options in financial derivative markets. In numerical study, we employ Monte Carlo simulations to demonstrate the performance of the current method.