Continuous-time Process Research Articles

Random sparse generators of Markovian evolution and their spectral properties.

The evolution of a complex multistate system is often interpreted as a continuous-time Markovian process. To model the relaxation dynamics of such systems, we introduce an ensemble of random sparse matrices which can be used as generators of Markovian evolution. The sparsity is controlled by a parameter φ, which is the number of nonzero elements per row and column in the generator matrix. Thus, a member of the ensemble is characterized by the Laplacian of a directed regular graph with D vertices (number of system states) and 2φD edges with randomly distributed weights. We study the effects of sparsity on the spectrum of the generator. Sparsity is shown to close the large spectral gap that is characteristic of nonsparse random generators. We show that the first moment of the eigenvalue distribution scales as ∼φ, while its variance is ∼sqrt[φ]. By using extreme value theory, we demonstrate how the shape of the spectral edges is determined by the tails of the corresponding weight distributions and clarify the behavior of the spectral gap as a function of D. Finally, we analyze complex spacing ratio statistics of ultrasparse generators, φ=const, and find that starting already at φ⩾2, spectra of the generators exhibit universal properties typical of Ginibre's orthogonal ensemble.

Effect of the U.S.–China Trade War on Stock Markets: A Financial Contagion Perspective

In this article, to model risk contagion between the U.S. and China stock markets based on high-frequency financial data, we develop a novel continuous-time jump-diffusion process. For example, we consider three channels for volatility contagion—such as integrated volatility, positive jump variation, and negative jump variation—and each stock market is able to affect the other stock market as an overnight risk factor. We develop a quasi-maximum likelihood estimator for model parameters and establish its asymptotic properties. Furthermore, to identify contagion channels and test the existence of a structural break with a known structural break date, we propose hypothesis test procedures. Using the proposed diffusion model with high-frequency financial data, we investigate the effect of the U.S.–China trade war on stock markets from a financial contagion perspective. From the empirical study, we find evidence of financial contagion from the United States to China and evidence that the risk contagion channel has changed from integrated volatility to negative jump variation.

Estimation of separable direct and indirect effects in a continuous-time illness-death model.

In this article we study the effect of a baseline exposure on a terminal time-to-event outcome either directly or mediated by the illness state of a continuous-time illness-death process with baseline covariates. We propose a definition of the corresponding direct and indirect effects using the concept of separable (interventionist) effects (Robins and Richardson in Causality and psychopathology: finding the determinants of disorders and their cures, Oxford University Press, 2011; Robins et al. in arXiv:2008.06019 , 2021; Stensrud et al. in J Am Stat Assoc 117:175-183, 2022). Our proposal generalizes Martinussen and Stensrud (Biometrics 79:127-139, 2023) who consider similar causal estimands for disentangling the causal treatment effects on the event of interest and competing events in the standard continuous-time competing risk model. Unlike natural direct and indirect effects (Robins and Greenland in Epidemiology 3:143-155, 1992; Pearl in Proceedings of the seventeenth conference on uncertainty in artificial intelligence, Morgan Kaufmann, 2001) which are usually defined through manipulations of the mediator independently of the exposure (so-called cross-world interventions), separable direct and indirect effects are defined through interventions on different components of the exposure that exert their effects through distinct causal mechanisms. This approach allows us to define meaningful mediation targets even though the mediating event is truncated by the terminal event. We present the conditions for identifiability, which include some arguably restrictive structural assumptions on the treatment mechanism, and discuss when such assumptions are valid. The identifying functionals are used to construct plug-in estimators for the separable direct and indirect effects. We also present multiply robust and asymptotically efficient estimators based on the efficient influence functions. We verify the theoretical properties of the estimators in a simulation study, and we demonstrate the use of the estimators using data from a Danish registry study.

Causal predictability between stochastic processes and filtrations

In this paper we further develop a notion of causal predictability defined in [A. Merkle, Predictability and Uniqueness of Weak Solutions of Stochastic Differential Equations, Analele Stiintifice ale Universitatii Ovidius Constanta, 2022] as a concept of dependence which is based on Granger's definition of causality. More precisely, in [A. Merkle, Predictability and Uniqueness of Weak Solutions of Stochastic Differential Equations, Analele Stiintifice ale Universitatii Ovidius Constanta, 2022] causal predictability is defined between filtrations, but now we introduce causal predictability between stochastic processes and filtrations. Also, we provide some properties of this new concept. Then we apply the given causality concept to the uniqueness of weak solutions of the stochastic differential equations and in financial mathematics. Granger [Investigating causal relations by econometric models and cross spectral methods, Econometrica. 37 (1969), pp. 424–438] has considered causality concept between time series. In this paper we consider continuous time processes, since continuous time models represent the first step in various applications, such as in finance, econometric practice, neuroscience, epidemiology, climatology, demographic, etc.

Kalikow decomposition for counting processes with stochastic intensity and application to simulation algorithms

AbstractWe propose a new Kalikow decomposition for continuous-time multivariate counting processes, on potentially infinite networks. We prove the existence of such a decomposition in various cases. This decomposition allows us to derive simulation algorithms that hold either for stationary processes with potentially infinite network but bounded intensities, or for processes with unbounded intensities in a finite network and with empty past before zero. The Kalikow decomposition is not unique, and we discuss the choice of the decomposition in terms of algorithmic efficiency in certain cases. We apply these methods to several examples: the linear Hawkes process, the age-dependent Hawkes process, the exponential Hawkes process, and the Galves–Löcherbach process.

Predicting the distribution of serotonergic axons: a supercomputing simulation of reflected fractional Brownian motion in a 3D-mouse brain model.

The self-organization of the brain matrix of serotonergic axons (fibers) remains an unsolved problem in neuroscience. The regional densities of this matrix have major implications for neuroplasticity, tissue regeneration, and the understanding of mental disorders, but the trajectories of its fibers are strongly stochastic and require novel conceptual and analytical approaches. In a major extension to our previous studies, we used a supercomputing simulation to model around one thousand serotonergic fibers as paths of superdiffusive fractional Brownian motion (FBM), a continuous-time stochastic process. The fibers produced long walks in a complex, three-dimensional shape based on the mouse brain and reflected at the outer (pial) and inner (ventricular) boundaries. The resultant regional densities were compared to the actual fiber densities in the corresponding neuroanatomically-defined regions. The relative densities showed strong qualitative similarities in the forebrain and midbrain, demonstrating the predictive potential of stochastic modeling in this system. The current simulation does not respect tissue heterogeneities but can be further improved with novel models of multifractional FBM. The study demonstrates that serotonergic fiber densities can be strongly influenced by the geometry of the brain, with implications for brain development, plasticity, and evolution.

Multivariate continuous-time autoregressive moving-average processes on cones

In this article we study multivariate continuous-time autoregressive moving-average (MCARMA) processes with values in convex cones. More specifically, we introduce matrix-valued MCARMA processes with Lévy noise and present necessary and sufficient conditions for processes from this class to be cone valued. We derive specific hands-on conditions in the following two cases: First, for classical MCARMA on Rd with values in the positive orthant Rd+. Second, for MCARMA processes on real square matrices assuming values in the cone of symmetric and positive semi-definite matrices. Both cases are relevant for applications and we give several examples of positivity ensuring parameter specifications. In addition to the above, we discuss the capability of positive semi-definite MCARMA processes to model the spot covariance process in multivariate stochastic volatility models. We justify the relevance of MCARMA based stochastic volatility models by an exemplary analysis of the second order moment structure of positive semi-definite well-balanced Ornstein–Uhlenbeck based models.

Loss and premium calculation of network nodes under the spread of SIS virus

In the big data and “Internet+” era, the research related cybersecurity risk has attracted much attention. However, Premium pricing for cybersecurity insurance remains in its early days. In this paper, we established a premium pricing method for cybersecurity risks. Firstly, the losses during the cyber infection is modeled by an interacting Markov SIS (Susceptible-Infected-Susceptible) epidemic model. we also proposed a premium simulation method called the Gillespie algorithm, which can be used for simulation of a continuous-time stochastic process. At last, as an example, we calculated the premiums by using premium principles and simulation in a simple network respectively. The numerical case studies demonstrate the premium pricing model performs well, and the premiums based on simulations are rather conservative, and recommended using in practice by comparing the results of premiums.

A direct discretization recurrent neurodynamics method for time-variant nonlinear optimization with redundant robot manipulators

Discrete time-variant nonlinear optimization (DTVNO) problems are commonly encountered in various scientific researches and engineering application fields. Nowadays, many discrete-time recurrent neurodynamics (DTRN) methods have been proposed for solving the DTVNO problems. However, these traditional DTRN methods currently employ an indirect technical route in which the discrete-time derivation process requires to interconvert with continuous-time derivation process. In order to break through this traditional research method, we develop a novel DTRN method based on the inspiring direct discrete technique for solving the DTVNO problem more concisely and efficiently. To be specific, firstly, considering that the DTVNO problem emerging in the discrete-time tracing control of robot manipulator, we further abstract and summarize the mathematical definition of DTVNO problem, and then we define the corresponding error function. Secondly, based on the second-order Taylor expansion, we can directly obtain the DTRN method for solving the DTVNO problem, which no longer requires the derivation process in the continuous-time environment. Whereafter, such a DTRN method is theoretically analyzed and its convergence is demonstrated. Furthermore, numerical experiments confirm the effectiveness and superiority of the DTRN method. In addition, the application experiments of the robot manipulators are presented to further demonstrate the superior performance of the DTRN method.

Risk of Stochastic Systems for Temporal Logic Specifications

The wide availability of data coupled with the computational advances in artificial intelligence and machine learning promise to enable many future technologies such as autonomous driving. While there has been a variety of successful demonstrations of these technologies, critical system failures have repeatedly been reported. Even if rare, such system failures pose a serious barrier to adoption without a rigorous risk assessment. This article presents a framework for the systematic and rigorous risk verification of systems. We consider a wide range of system specifications formulated in signal temporal logic (STL) and model the system as a stochastic process, permitting discrete-time and continuous-time stochastic processes. We then define the STL robustness risk as the risk of lacking robustness against failure . This definition is motivated as system failures are often caused by missing robustness to modeling errors, system disturbances, and distribution shifts in the underlying data generating process. Within the definition, we permit general classes of risk measures and focus on tail risk measures such as the value-at-risk and the conditional value-at-risk. While the STL robustness risk is in general hard to compute, we propose the approximate STL robustness risk as a more tractable notion that upper bounds the STL robustness risk. We show how the approximate STL robustness risk can accurately be estimated from system trajectory data. For discrete-time stochastic processes, we show under which conditions the approximate STL robustness risk can even be computed exactly. We illustrate our verification algorithm in the autonomous driving simulator CARLA and show how a least risky controller can be selected among four neural network lane-keeping controllers for five meaningful system specifications.

On the use of Kolmogorov equations for determining ion channels characteristics

The paper proposes a method for determination of the dynamic properties of conduction within voltage-gated ion channels with several sequentially located energy barriers named gate particles using a theory of Markov random processes in continuous time and with a discrete state space. The number of states was taken equal to the number of energy barriers in the channel plus one. If the hypothesis that random flows switching gate particles between an open and a closed are Poisson process, is true, then mathematical description of the system’s state of similar channels can be a system of Kolmogorov linear differential equations for state probabilities. Using this model and based on Volt-Clamp test results, published in open access journals, it is possible to find out how fixed values of membrane potential depend on the intensities of Poisson process for different types of potassium channels. Function parameters, that describe the intensity of ion transition across the channel, in the Kolmogorov equation can be restored using the generalized least squares method. This paper contains the examples of determining the intensities of transitions for two types of voltage-dependent potassium channels known as «delayed rectifiier potassium channels» (IKdr, two identical activation gate particles) and channels with fast activation and inactivation processes (IKa, three identical activation and one inactivation gate particles). It is shown that channel activation and deactivation is described by solving the general Kolmogorov equation.

Accelerating inference for stochastic kinetic models

Stochastic kinetic models (SKMs) are increasingly used to account for the inherent stochasticity exhibited by interacting populations of species in areas such as epidemiology, population ecology and systems biology. Species numbers are modelled using a continuous-time stochastic process, and, depending on the application area of interest, this will typically take the form of a Markov jump process or an Itô diffusion process. Widespread use of these models is typically precluded by their computational complexity. In particular, performing exact fully Bayesian inference in either modelling framework is challenging due to the intractability of the observed data likelihood, necessitating the use of computationally intensive techniques such as particle Markov chain Monte Carlo (particle MCMC). It is proposed to increase the computational and statistical efficiency of this approach by leveraging the tractability of an inexpensive surrogate derived directly from either the jump or diffusion process. The surrogate is used in three ways: in the design of a gradient-based parameter proposal, to construct an appropriate bridge and in the first stage of a delayed-acceptance step. The resulting approach, which exactly targets the posterior of interest, offers substantial gains in efficiency over a standard particle MCMC implementation.

Managing production-inventory-maintenance systems with condition monitoring

Providing spare parts and maintenance services to customers has become an important revenue stream for equipment manufacturers. This paper studies a Production-Inventory-Maintenance (PIM) system where a manufacturer serves an installed base of machines. Each machine contains the same critical component, whose degradation behaviour is described by a Markov process and captured via condition monitoring. When the state of a component reaches a prespecified replacement threshold, the machine is considered to have failed and incurs downtime penalties until being repaired. There are a few repair servers and queueing dynamics follows the Machine Interference Problem (MIP). Component deterioration and server congestion provide advance demand information, which helps plan the production of spare parts in a make-to-stock routine. The unsatisfied requests for spare parts are fulfilled by emergency replenishments with lost-sales penalties. We formulate the production-control problem as a continuous-time Markov decision process, characterize the structure of the optimal policy, and develop an effective heuristic policy. Through numerical experiments, we reveal a number of notable differences between managing PIM systems and operating MIP systems alone: (1) Although identifying more degradation states has no value to MIP systems, PIM systems benefit substantially from such a refined state identification. (2) While MIP systems prefer more repair servers and higher repair operation efficiency, greater repair capacity may worsen the performance of PIM systems. (3) When the product of repair service rate and server number is fixed, a single server is always preferred in MIP systems. However, PIM systems could perform better with multiple less-efficient servers.

Behavioural equivalences for continuous-time Markov processes

AbstractBisimulation is a concept that captures behavioural equivalence of states in a variety of types of transition systems. It has been widely studied in a discrete-time setting. The core of this work is to generalise the discrete-time picture to continuous time by providing a notion of behavioural equivalence for continuous-time Markov processes. In Chen et al. [(2019). Electronic Notes in Theoretical Computer Science347 45–63.], we proposed two equivalent definitions of bisimulation for continuous-time stochastic processes where the evolution is a flow through time: the first one as an equivalence relation and the second one as a cospan of morphisms. In Chen et al. [(2020). Electronic Notes in Theoretical Computer Science.], we developed the theory further: we introduced different concepts that correspond to different behavioural equivalences and compared them to bisimulation. In particular, we studied the relation between bisimulation and symmetry groups of the dynamics. We also provided a game interpretation for two of the behavioural equivalences. The present work unifies the cited conference presentations and gives detailed proofs.

Studies on Morphological Parameters, Radius of Gyration, Correlation Length and Invariant of HPMC Polymer Doped with Ferrous (II) Oxide Using SAXS Method

It shed light on morphological parameters which influence radius of gyration, correlation length and invariant of Hydroxy Propyl Methyl Cellulose (HPMC) polymer doped with ferrous (II) oxide (FeO) using small angle X-ray scattering (SAXS) method. SAXS patterns have been recorded for different composition and gives an information about the structure of the polymer on a scale of about 800Ǻ, in which the variation of scattered intensity with angles from 0o to 2.5o has been measured for different composition of FeO with host polymer of HPMC. Employing para-crystalline model and exponential distribution function of phase lengths (amorphous and crystalline), we find a good agreement between the measured and predicted scattering except for a very small range of angles on the low angle side of the peak intensity. The method enabled quantitative parameters describing morphology of the two phases, amorphous and crystalline, consistent with the SAXS intensity distribution. A stepping procedure has been employed to compute the final parameters which gave the best fit between the calculated and experimental data. This was done by minimizing the statistical goodness-of-fit parameter chi-square. It is observed that there are significant changes in distribution of phase lengths due to different percentage weight of FeO in HPMC contents. Additionally, SAXS data has been used to estimate (i) radius of gyration (Rg), ii) correlation length (Lcor) and (iii) invariant (Q) parameters. A functional representation of the long period and also radius of gyration at different dopant concentrations is adopted by viewing them as realizations of a continuous-time stochastic process observed with measurement error. This allows one to estimate the mean functional relationship between long period and the dopant concentration.

Gradient-Based Simulation Optimization Algorithms via Multi-Resolution System Approximations

We propose gradient-based simulation-optimization algorithms to optimize systems that have complicated stochastic structure. The presence of complicated stochastic structure, such as the involvement of infinite-dimensional continuous-time stochastic processes, may cause the exact simulation of the system to be costly or even impossible. On the other hand, for a complicated system, one can sometimes construct a sequence of approximations at different resolutions, where the sequence has finer and finer approximation resolution but higher and higher cost to simulate. With the goal of optimizing the complicated system, we propose algorithms that strategically use the approximations with increasing resolution and higher simulation cost to construct stochastic gradients and perform gradient search in the decision space. To accommodate scenarios where approximations cause discontinuities and lead path-wise gradient estimators to have an uncontrollable bias, stochastic gradients for the proposed algorithms are constructed through finite difference. As a theory support, we prove algorithm convergence, convergence rate, and optimality of algorithm design under the assumption that the objective function for the complicated system is strongly convex, whereas no such assumptions are imposed on the approximations of the complicated system. We then present a multilevel version of the proposed algorithms to further improve convergence rates, when in addition the sequence of approximations can be naturally coupled. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1279 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0289 ) at ( http://dx.doi.org/10.5281/zenodo.7485443 ).

Solving the joint military medical evacuation problem via a random forest approximate dynamic programming approach

Members of the armed forces rely on having an effective and efficient medical evacuation (MEDEVAC) process for evacuating casualties from the battlefield to medical treatment facilities during combat operations. Determining which MEDEVAC units to task to respond to incoming requests for service, known as the MEDEVAC dispatching problem, is a natural sequential decision making problem and is the focus of this paper. Unlike previous research in this area, this paper incorporates triage classification errors and the possibility of having blood transfusion kits on board select MEDEVAC units. A discounted, infinite-horizon continuous-time Markov decision process model is formulated to examine the MEDEVAC dispatching problem and to compare generated dispatching policies to the currently practiced United States military policy. We utilize an approximate dynamic programming (ADP) technique that leverages a random forest value function approximation within an approximate policy iteration algorithmic framework to develop high-quality policies for both small-scale and large-scale problem instances. A representative planning scenario involving joint combat operations in South Korea is developed and utilized to investigate the differences between the various policies. Results from the analysis indicate that applying ADP techniques can improve current practices by as much as 29% with regard to a life-saving performance metric. This research is of particular interest to the military medical community and can inform the procedures of future military MEDEVAC operations.

A note on the existence of optimal stationary policies for average Markov decision processes with countable states

In many practical stochastic dynamic optimization problems with countable states, the optimal policy possesses certain structural properties. For example, the (s,S) policy in inventory control, the well-known cμ-rule and the recently discovered c/μ-rule (Xia et al. (2022)) in scheduling of queues. A presumption of such results is that an optimal stationary policy exists. There are many research works regarding to the existence of optimal stationary policies of Markov decision processes with countable state spaces (see, e.g., Bertsekas (2012); Hernández-Lerma and Lasserre (1996); Puterman (1994); Sennott (1999)). However, these conditions are usually not easy to verify in such optimization problems. In this paper, we study the optimization of long-run average of continuous-time Markov decision processes with countable state spaces. We provide an intuitive approach to prove the existence of an optimal stationary policy. The approach is simply based on compactness of the policy space, with a special designed metric, and the continuity of the long-run average in the space. Our method is capable to handle cost functions unbounded from both above and below, which makes a complementary contribution to the literature work where the cost function is unbounded from only one side. Examples are provided to illustrate the application of our main results.

Formation mechanism and quantitative analysis of pores in Al2O3–ZrO2 ceramic different structures by laser additive manufacturing

Pores is one of the most defects during laser additive manufacturing of ceramic material. In this paper, different structures of Al2O3–ZrO2 ceramics are deposited by using powder-feeding laser additive manufacturing. The ceramic structures can be simplified into three typical structures: linear-type, face-type and body-type structure. The formation mechanism and quantitative analysis of pores in different ceramic structures are studied. The pores of ceramic structure is mainly divided into four types: intercrystalline pore, interlayer pore, intralayer pore and shrinkage pore. In the linear-type structure, the number of pores with the interval 0–1000 μm2 accounts for 98.8%, while the area of pores with the interval above 104 μm2 accounts for 58.33%. The remelting rate of deposition layer has great influence on the pores. The Marangoni convection changed from inflow mode to outflow mode, which leads to that bubbles have easily escape from molten pool. In the face-type structure, the number and area of pores at the interval 0–1000 μm2 account for 98.96% and 51.62%, respectively. Overlap ratio is the key factor affecting pore. Under the effects of the Marangoni convection and the overlap of deposition layer, the irregular pores are easily formed in the bottom of overlap region of deposition layer. The pores characteristic in the body-type structure is similar to that in linear-type structure. Due to the laser remelting and overlap phenomenon, the thermal accumulation effect is more obvious in the short time interval and continuous deposition process of body-type structure. Thus, the gas of molten pool can be easily escaped. This study can provide the theoretical basis for the suppression of pores in ceramic laser additive manufacturing.

Optimality of a Network Monitoring Agent and Validation in a Real Probe

The evolution of commodity hardware makes it possible to use this type of equipment to implement traffic monitoring systems. A preliminary empirical evaluation of a network traffic probe based on Linux indicates that the system performance has significant losses as the network rate increases. To assess this issue, we consider a model with two tandem queues and a moving server. In this system, we formulate a three-dimensional Markov Decision Process in continuous time. The goal of the proposed model is to determine the position of the server in each time slot so as to optimize the system performance which is measured in terms of throughput. We first formulate an equivalent discrete-time Markov Decision Process and we propose a numerical method to characterize the solution of our problem in a general setting. The solution we obtain in this problem has been tested for a wide range of scenarios and, in all the instances, we observe that the optimality is close to a threshold type policy. We also consider a real probe and we validate the good performance of threshold policies in real applications.