Continuous-time Process Research Articles (Page 1)

Abstract This paper develops a hybrid deep reinforcement learning approach to manage an insurance portfolio for diffusion models. To address the model uncertainty, we adopt the recently developed modelling of exploration and exploitation strategies in a continuous-time decision-making process with reinforcement learning. We consider an insurance portfolio management problem in which an entropy-regularized reward function and corresponding relaxed stochastic controls are formulated. To obtain the optimal relaxed stochastic controls, we develop a Markov chain approximation and stochastic approximation-based iterative deep reinforcement learning algorithm where the probability distribution of the optimal stochastic controls is approximated by neural networks. In our hybrid algorithm, both Markov chain approximation and stochastic approximation are adopted in the learning processes. The idea of using the Markov chain approximation method to find initial guesses is proposed. A stochastic approximation is adopted to estimate the parameters of neural networks. Convergence analysis of the algorithm is presented. Numerical examples are provided to illustrate the performance of the algorithm.

The role of selective extinction in promoting cooperation in a many demes model.

For highly reliable products that degrade over time, it is not uncommon to observe an initial period during which no failure occurs. In this paper, a degradation model that gives rise to a first passage time distribution with failure-free life is proposed as a more plausible model compared to many common models. We call it Beta process as its increments are approximated by three-parameter Beta distributions. We start with a discrete time process as a degradation process is commonly measured at regular time intervals. We derive a closed-form first passage time distribution under a constant failure threshold. Closed-form maximum likelihood (ML)-type estimators are developed for the parameters of the Beta process. Then, the discrete time process is extended to the case of a continuous time process. Comprehensive simulations and case studies show that the Beta processes outperform many common degradation processes in reliability estimation.

Accurate path travel time prediction is often hindered by sparse and heterogeneous traffic data. This paper proposes FusionODE-TT, a novel model designed to address these challenges by modeling traffic as a continuous-time process. The model features a Recurrent Neural Network encoder that processes multi-source time-series data to initialize a latent state vector, which then evolves over the prediction horizon using a Neural Ordinary Differential Equation (NODE). The core innovation is a guided fusion mechanism that leverages sparse but high-fidelity Automatic Vehicle Identification (AVI) data to apply strong, event-based corrections to the model’s continuous latent state, mitigating error accumulation in the prediction process. Experiments were conducted on a real-world dataset comprising AVI, GPS, and point sensor data from a major urban expressway. The experimental results demonstrate that the proposed model achieves superior accuracy, outperforming a suite of baseline models in terms of prediction accuracy and robustness. Furthermore, a comprehensive ablation study was performed to validate the efficacy of our design. The study quantitatively confirms that both the continuous-time dynamics modeled by the NODE and the guided fusion mechanism are essential components, each providing a significant and independent contribution to the model’s overall performance.

Abstract We consider a continuous time process that is self‐exciting and ergodic, called the threshold Chan–Karolyi–Longstaff–Sanders (CKLS) process. This process is a generalization of various models in econometrics, such as the Vasicek model, the Cox–Ingersoll–Ross model, and the Black–Scholes model, allowing for the presence of several thresholds which determine changes in the dynamics. We study the asymptotic behavior of maximum‐likelihood and quasi‐maximum‐likelihood estimators of the drift parameters in the case of continuous time and discrete time observations. We show that for high frequency observations and infinite horizon the estimators satisfy the same asymptotic normality property as in the case of continuous time observations. We also discuss diffusion coefficient estimation. Finally, we apply our estimators to simulated and real data to motivate the consideration of (multiple) thresholds.

Serotonergic axons (fibers) are a universal feature of all vertebrate brains. They form meshworks, typically quantified with regional density measurements, and appear to support neuroplasticity. The self-organization of this system remains poorly understood, partly because of the strong stochasticity of individual fiber trajectories. In an extension to our previous analyses of the mouse brain, serotonergic fibers were investigated in the brain of the Pacific angelshark (Squatina californica), a representative of a unique (ray-like) lineage of the squalomorph sharks. First, the fundamental cytoarchitecture of the angelshark brain was examined, including the expression of ionized calcium-binding adapter molecule 1 (Iba1, AIF-1) and the mesencephalic trigeminal nucleus. Second, serotonergic fibers were visualized with immunohistochemistry, which showed that fibers in the forebrain have the tendency to move toward the dorsal pallium and also accumulate at higher densities at pial borders. Third, a population of serotonergic fibers was modeled inside a digital model of the angelshark brain by using a supercomputing simulation. The simulated fibers were defined as sample paths of reflected fractional Brownian motion (FBM), a continuous-time stochastic process. The regional densities generated by these simulated fibers reproduced key features of the biological serotonergic fiber densities in the telencephalon, a brain division with a considerable physical uniformity and no major “obstacles” (dense axon tracts). These results demonstrate that the paths of serotonergic fibers may be inherently stochastic, and that a large population of such paths can give rise to a consistent, non-uniform, and biologically-realistic fiber density distribution. Local densities may be induced by the constraints of the three-dimensional geometry of the brain, with no axon guidance cues. However, they can be further refined by anisotropies that constrain fiber movement (e.g., major axon tracts, active self-avoidance, chemical gradients). In the angelshark forebrain, such constraints may be reduced to an attractive effect of the dorsal pallium, suggesting that anatomically complex distributions of fiber densities can emerge from the interplay of a small set of stochastic and deterministic processes.

The local long-time behaviour for continuous-time branching processes

Neural variability poses a major challenge to understanding the information content of neural codes. Recent work by Goris et al. [1] has provided new insights into variability in the visual pathway by partitioning it into two components: a stimulus component reflecting sensory tuning, and a modulatory component arising from stimulus-independent fluctuations in excitability. However, a key limitation of this framework is that it lacks a continuous-time interpretation; it applies only to spike counts measured in time bins of a given size, and does not generalize to different bin sizes. Here we overcome this limitation using a new model for continuous-time partitioning of neural variability, the “continuous modulated Poisson” (CMP). This model extends the partitioning model of Goris et al. [1] by modeling a neuron’s instantaneous firing rate as the product of a time-varying stimulus drive and a continuous-time stochastic gain process. We apply this model to spike responses from four different visual areas (LGN, V1, V2, and MT) and show that it accurately captures spike train variability across timescales and throughout the visual hierarchy. Moreover, we demonstrate that the modulatory gain process decays according to an exponentiated power law, with higher variance and slower decay at later stages of the visual hierarchy. This model provides new insights into the organization of both stimulus-driven and stimulus-independent modulatory processes and provides a powerful framework for characterizing neural variability across brain regions.

Predicting time-series data is inherently complex, spurring the development of advanced neural network approaches. Monitoring and predicting PM2.5 levels is especially challenging due to the interplay of diverse natural and anthropogenic factors influencing its dispersion, making accurate predictions both costly and intricate. A key challenge in predicting PM2.5 concentrations lies in its variability, as the data distribution fluctuates significantly over time. Meanwhile, neural networks provide a cost-effective and highly accurate solution in managing such complexities. Deep learning models like Long Short-Term Memory (LSTM) and Bidirectional LSTM (BiLSTM) have been widely applied to PM2.5 prediction tasks. However, prediction errors increase as the forecasting window expands from 1 to 72 hours, underscoring the rising uncertainty in longer-term predictions. Recurrent Neural Networks (RNNs) with continuous-time hidden states are well-suited for modeling irregularly sampled time series but struggle with long-term dependencies due to gradient vanishing or exploding, as revealed by the ordinary differential equation (ODE) based hidden state dynamics–regardless of the ODE solver used. Continuous-time neural processes, defined by differential equations, are limited by numerical solvers, restricting scalability and hindering the modeling of complex phenomena like neural dynamics–ideally addressed via closed-form solutions. In contrast to ODE-based continuous models, closed-form networks demonstrate superior scalability over traditional deep-learning approaches. As continuous-time neural networks, Neural ODEs excel in modeling the intricate dynamics of time-series data, presenting a robust alternative to traditional LSTM models. We propose two ODE-based models: a transformer-based ODE model and a closed-form ODE model. Empirical evaluations show these models significantly enhance prediction accuracy, with improvements ranging from 2.91 to 14.15% for 1-hour to 8-hour predictions when compared to LSTM-based models. Moreover, after conducting the paired t-test, the RMSE values of the proposed model (CCCFC) were found to be significantly different from those of BILSTM, LSTM, GRU, ODE-LSTM, and PCNN,CNN-LSSTM. This implies that CCCFC demonstrates a distinct performance advantage, reinforcing its effectiveness in hourly PM2.5 forecasting.

Building on the work of Almasan etal. [IEEE Trans. Netw. Sci. Eng. 12, 1649 (2025)10.1109/TNSE.2025.3537162], we propose a continuous-time Markov model for human contact dynamics denoted the continuous random walkers induced temporal graph model (CRWIG). In CRWIG, M walkers move randomly and independently of each other on a Markov graph with N nodes in continuous time. If walkers are in the same state (node of the Markov graph) at time t, a link is created between them in their temporal contact graph G(t), where each walker corresponds to one of the M nodes. We define the exact Markov governing equationthat describes the movement of the ensemble of M walkers. We investigate the consequences of the time discretization of CRWIG. We prove that CRWIG is characterized by exponential decay of the initial condition and exponentially tailed intermeeting times of the walkers. We investigate two special cases of CRWIG and derive analytical results supported by simulations. We extend the model to allow for nonexponential sojourn times for the single walkers. The non-Markovian model extension of CRWIG is able to reproduce empirical properties of human mobility observed on data: arbitrary flight length distribution, arbitrary pause-time distribution, and intermeeting time distributions that are power-law with an exponential tail.

Critical infrastructure, such as telecom base stations, comprises spatially distributed facilities that alternate between operational and down states. When the facilities are down, service engineers need to travel to the site to repair and restore them to operation. In the event of a disruption, multiple facilities might be down. However, repair priorities are often assigned on an ad hoc basis, leading to suboptimal decisions. This paper studies the optimal repair priority policy for a service engineer managing facilities within their service region. We formulate a continuous-time Markov decision process for optimal decision-making. We show that the famous c μ -rule in the job scheduling literature is not optimal for assigning repair priorities in our setting. For the general case, we identify certain facilities that should always be prioritized over others when they are down and determine the optimal repair priority at some system states. Nevertheless, a full characterization of the optimal policy structure of the general case is intractable due to its complexity. We identify three special cases where the optimal policy structure exhibits an index structure, referred to as the c μ / λ -rule. By combining these structural properties with the famous c μ -rule, we develop an index-based heuristic. Numerical experiments, based on a telecom base station failure dataset from Qiqihar, China, demonstrate the effectiveness of the heuristic in a variety of settings, including non-Markovian system dynamics.

Feeling and expressing love in daily life are interconnected and perhaps mutually influential experiences. In this study we examined the reciprocal dynamics of feeling and expressing love and its relation to well-being using an ecological momentary assessment design. Over a four-week period, we asked participants (N = 52; 67% Female; 80% White) to report their levels of feeling loved and expressing love six times a day. Using a continuous-time process model, we explored individual differences in inertia (i.e., persistence of a process over time) and cross-influences of felt and expressed love over time. We found that increases in expressing love led to increased feelings of being loved over time; however, increases in felt love did not lead to increases in expressing love. Notably, participants who experienced more persistent feelings of love (that is, greater inertia over time) indicated higher levels of flourishing. These results suggest new avenues for psychological well-being interventions which target increasing loving feelings through encouraging more expressions of love.

Feeling and expressing love in daily life are interconnected and perhaps mutually influential experiences. In this study we examined the reciprocal dynamics of feeling and expressing love and its relation to well-being using an ecological momentary assessment design. Over a four-week period, we asked participants (N = 52; 67% Female; 80% White) to report their levels of feeling loved and expressing love six times a day. Using a continuous-time process model, we explored individual differences in inertia (i.e., persistence of a process over time) and cross-influences of felt and expressed love over time. We found that increases in expressing love led to increased feelings of being loved over time; however, increases in felt love did not lead to increases in expressing love. Notably, participants who experienced more persistent feelings of love (that is, greater inertia over time) indicated higher levels of flourishing. These results suggest new avenues for psychological well-being interventions which target increasing loving feelings through encouraging more expressions of love.

Intensive longitudinal data (ILD) collected in mobile health (mHealth) studies contain rich information on the dynamics of multiple outcomes measured frequently over time. Motivated by an mHealth study in which participants self-report the intensity of many emotions multiple times per day, we describe a dynamic factor model that summarizes ILD as a low-dimensional, interpretable latent process. This model consists of (i) a measurement submodel-a factor model-that summarizes the multivariate longitudinal outcome as lower-dimensional latent variables and (ii) a structural submodel-an Ornstein-Uhlenbeck (OU) stochastic process-that captures the dynamics of the multivariate latent process in continuous time. We derive a closed-form likelihood for the marginal distribution of the outcome and the computationally-simpler sparse precision matrix for the OU process. We propose a block coordinate descent algorithm for estimation and use simulation studies to show that it has good statistical properties with ILD. Then, we use our method to analyze data from the mHealth study. We summarize the dynamics of 18 emotions using models with one, two, and three time-varying latent factors, which correspond to different behavioral science theories of emotions. We demonstrate how results can be interpreted to help improve behavioral science theories of momentary emotions, latent psychological states, and their dynamics.

Sometimes, limits can be singular, implying that they take on different values depending on the order of arithmetic operations. In other words, the limit map lacks commutativity. While all such limits are mathematically valid, only one can be the physical limit. The change of measure for Brownian processes illustrates this phenomenon. A substantial body of elegant mathematics centered around continuous-time Brownian processes has been embraced by the physics community to investigate the nonequilibrium and equilibrium thermodynamics of systems composed of atoms and molecules. In this paper, we derive the continuous-time limit of discrete-time Brownian dynamics, specifically focusing on the change of measure. We demonstrate that this result yields the physical limit that differs from the commonly used expression. Consequently, the concepts of “the most probable path”, “minimum thermodynamic action”, and “the small-noise limit” are unphysical mathematical artifacts.

We examine continuous-time stochastic processes with a general compact state space, which is organized by a fundamental graph defining a neighborhood structure of states. These neighborhoods establish local interactions among the coordinates of the spatial process. At any given moment, the random state of the system, as described by the stochastic process, forms a random field concerning the neighborhood graph.The process is assumed to have a semi-Markov temporal property, and its transition kernels exhibit a spatial Markov property relative to the basic graph. Additionally, a local control structure is introduced to optimize the evolution of the system over time. Here, optimality is defined in terms of the criterion of asymptotic average reward over time. Only discrete stepwise control is considered, meaning decisions are made exclusively at process jump moments. As is customary, random policies are represented by a conditionally independent structure. It is also assumed that this structure exists within the transition kernels of jump-like chains (synchronized kernels). The controlled random fields described, featuring local and synchronous component interactions, are applied to queueing systems—specifically, to the extended and generalized closed Gordon-Newell network. The modification primarily involves synchronizing customer service times at nodes. Based on the queue length at the node and its vicinity, a decision is made regarding serving the customer. If service is provided, a decision is also made regarding the customer’s next direction. Consequently, the enhanced Gordon-Newell network satisfies the conditions for synchronous and local system node interactions. A procedure is outlined for determining optimal non-randomized control strategies in the enhanced Gordon-Newell network.

Treatment-confounder feedback is present in time-to-recurrent or longitudinal event analysis when time-dependent confounders are themselves influenced by previous treatments. Conventional models produce misleading statistical inference of causal effects due to conditioning on these factors on the causal pathway. Marginal structural models are often applied to quantify the causal treatment effect, estimated using longitudinal weights that mimic the randomization procedure by balancing the covariate distributions across the treatment groups. The weights are usually constructed in discrete time intervals, which is appropriate if the follow-up visits are scheduled and regular. However, in primary care, visit times can be irregular and informative, and the treatment history consists of duration and doses. This can be modeled through a continuous-time marked point process. We constructed a continuous-time marginal structural model to estimate the effect of cumulative exposure to Sodium-Glucose co-Transporters 2 Inhibitor (SGLT-2i) medications on time-to-recurrent urinary tract infection (UTI). We used a cohort of type II diabetes patients with chronic kidney disease and constructed a marked point process that characterized the recurrent flare episodes of primary care visits (i.e., point process) with marks for the multinominal dose (none, low, high) of SGLT-2i medications and recurrent episodes of UTI. We applied the stabilized and optimal treatment weights to estimate the hypothesized causal effect. Our results are concordant with earlier findings in which the recurrent episodes of UTI did not increase when patients were prescribed low dose or high dose of SGLT-2i medications.

Abstract This paper investigates a well-known downside protection strategy called the constant proportion portfolio insurance (CPPI) in defined contribution (DC) pension fund modeling. Under discrete time trading CPPI, an investor faces the risk of portfolio value hitting the floor which denotes the process of guaranteed portfolio values. In this paper, we question how to deal with so-called ‘gap risk’ which may appear due to uncontrollable events resulting in a sudden drop in the market. In the market model considered, the risky asset price and the labor income are assumed to be continuous-time stochastic processes, whereas trading is restricted to discrete-time. In this setting, an exotic option (namely, the ‘cushion option’) is proposed with the aim of reducing the risk that the portfolio value falls below the defined floor. We analyze the effectiveness of the proposed exotic option for a DC plan CPPI strategy through Monte Carlo simulations and sensitivity analyses with respect to the parameters reflecting different setups.

In this work, we propose a wavelet-based framework for estimating the derivatives of a density function in the setting of continuous, stationary, and ergodic processes. Our primary focus is the derivation of the integrated mean square error (IMSE) over compact subsets of Rd, which provides a quantitative measure of the estimation accuracy. In addition, a uniform convergence rate and normality are established. To establish the asymptotic behavior of the proposed estimators, we adopt a martingale approach that accommodates the ergodic nature of the underlying processes. Importantly, beyond ergodicity, our analysis does not require additional assumptions regarding the data. By demonstrating that the wavelet methodology remains valid under these weaker dependence conditions, we extend earlier results originally developed in the context of independent observations.

Two dynamical indicators, the local dimension and the extremal index, used to quantify persistence in phase space have been developed and applied to different data across various disciplines. These are computed using the asymptotic limit of exceedances over a threshold, which turns to be a generalized Pareto distribution in many cases. However, the derivation of the asymptotic distribution requires mathematical properties, which are not present even in highly idealized dynamical systems and unlikely to be present in the real data. Here, we examine in detail the issues that arise when estimating these quantities for some known dynamical systems. We focus on how the geometry of an invariant set can affect the regularly varying properties of the invariant measure. We demonstrate that singular measures supported on sets of the non-integer dimension are typically not regularly varying and that the absence of regular variation makes the estimates resolution dependent. We show as well that the most common extremal index estimation method is not well defined for continuous time processes sampled at fixed time steps, which is an underlying assumption in its application to data.