Continuous Time Research Articles

Zero-sum stochastic games in continuous-time with risk-sensitive average cost criterion on a countable state space

We consider zero-sum stochastic games in continuous time with controlled Markov chains and with risk-sensitive average cost criterion. Here the transition and the cost rates may be unbounded. We prove the existence of the value of the game and a saddle-point equilibrium in the class of all stationary strategies under a Lyapunov stability condition. This is accomplished by establishing the existence of a principal eigenpair for the corresponding Hamilton-Jacobi-Isaacs (HJI) equation. This, in turn, is established by using a nonlinear version of Krein-Rutman theorem. We then obtain a characterization of the saddle-point equilibrium in terms of the corresponding HJI equation. Finally, we use a controlled population system to illustrate our results.

Self-Supervised Temporal Graph Learning With Temporal and Structural Intensity Alignment.

Temporal graph learning aims to generate high-quality representations for graph-based tasks with dynamic information, which has recently garnered increasing attention. In contrast to static graphs, temporal graphs are typically organized as node interaction sequences over continuous time rather than an adjacency matrix. Most temporal graph learning methods model current interactions by incorporating historical neighborhood. However, such methods only consider first-order temporal information while disregarding crucial high-order structural information, resulting in suboptimal performance. To address this issue, we propose a self-supervised method called S2T for temporal graph learning, which extracts both temporal and structural information to learn more informative node representations. Notably, the initial node representations combine first-order temporal and high-order structural information differently to calculate two conditional intensities. An alignment loss is then introduced to optimize the node representations, narrowing the gap between the two intensities and making them more informative. Concretely, in addition to modeling temporal information using historical neighbor sequences, we further consider structural knowledge at both local and global levels. At the local level, we generate structural intensity by aggregating features from high-order neighbor sequences. At the global level, a global representation is generated based on all nodes to adjust the structural intensity according to the active statuses on different nodes. Extensive experiments demonstrate that the proposed model S2T achieves at most 10.13% performance improvement compared with the state-of-the-art competitors on several datasets.

An SIS epidemic model with individual variation

<abstract><p>We study an extension of the stochastic SIS (Susceptible-Infectious-Susceptible) model in continuous time that accounts for variation amongst individuals. By examining its limiting behaviour as the population size grows we are able to exhibit conditions for the infection to become endemic.</p></abstract>

Note on control for hybrid stochastic systems by intermittent feedback rooted in discrete observations of state and mode with delays

<abstract> <p>For a hybrid stochastic system, most existing feedback controllers need to observe modes at continuous times, which is feasible when the system's mode is observable and does not incur any cost. However, in most cases, the mode is not readily apparent, and identifying it always incurs a certain expense. Therefore, in order to reduce control costs, when designing a feedback controller, both the state and the mode should be observed at discrete moments. This paper introduces an intermittent feedback controller for stabilizing an unstable hybrid stochastic system through discrete delayed observations of state and mode. By utilizing M-matrix theory, intermittent control approach, and the comparison principle, we propose sufficient conditions for the stabilization theory of hybrid stochastic systems. An illustrative example is taken to validate the proposed theory.</p> </abstract>

Research on Simulation Approximate Solution Strategy for Complex Kinematic Models

Abstract In order to meet the needs of military, road construction, multimedia industry and other aspects, UAVs are gradually given more functions. As the basic function of UAV applications, the fixed-point delivery problem model has higher and higher accuracy requirements. However, in the actual scene, the UAV delivery problem is affected by the interaction of various factors such as flight height, air resistance, and dive angle, which makes it difficult to achieve high stability and high hit accuracy. This paper will analyze the complex motion model based on the fixed-point delivery of explosives by UAV, study the relationship between the stability of UAV delivery and the hit accuracy, and analyze the influence of relevant parameters on the problem by using modeling. In this paper, a multivariate nonlinear continuous time change model is proposed, and a continuous time slice discretization idea operation model is introduced to approximate the time slice splitting inside the UAV launch motion. Secondly, the design quantified evaluation index reaction the initial velocity of the explosive, the launch Angle, the height off the ground and other parameters to analyze the model. Finally, the best scheduling strategy is obtained and verified by using the idea of variable traversal and trial- and-error simulation. The experimental results show that the variation of UAV flying height, speed, depression and other interference factors is consistent with the prediction of score and hit accuracy, according to the environment setting of this model, when the UAV is 300 meters above the ground and 290 meters away from the target horizontal position, the delivery speed is 250m/s, and the pitch angle is about 27°, the fixed-point delivery of explosives is the best strategy.

Multi-View Time-Series Hypergraph Neural Network for Action Recognition.

Recently, action recognition has attracted considerable attention in the field of computer vision. In dynamic circumstances and complicated backgrounds, there are some problems, such as object occlusion, insufficient light, and weak correlation of human body joints, resulting in skeleton-based human action recognition accuracy being very low. To address this issue, we propose a Multi-View Time-Series Hypergraph Neural Network (MV-TSHGNN) method. The framework is composed of two main parts: the construction of a multi-view time-series hypergraph structure and the learning process of multi-view time-series hypergraph convolutions. Specifically, given the multi-view video sequence frames, we first extract the joint features of actions from different views. Then, limb components and adjacent joints spatial hypergraphs based on the joints of different views at the same time are constructed respectively, temporal hypergraphs are constructed joints of the same view at continuous times, which are established high-order semantic relationships and cooperatively generate complementary action features. After that, we design a multi-view time-series hypergraph neural network to efficiently learn the features of spatial and temporal hypergraphs, and effectively improve the accuracy of skeleton-based action recognition. To evaluate the effectiveness and efficiency of MV-TSHGNN, we conduct experiments on NTU RGB+D, NTU RGB+D 120 and imitating traffic police gestures datasets. The experimental results indicate that our proposed method model achieves the new state-of-the-art performance.

Cooperative Event-Triggered Fuzzy-Neural Multivariable Control With Multitask Learning for Municipal Solid Waste Incineration Process

Municipal solid waste incineration (MSWI) is an industrial process with multiple mechanism reactions, which has strong coupling and time-varying dynamics. It is extremely difficult to design a feasible multivariable controller for MSWI process due to the complex composition of municipal solid waste and fluctuation of calorific value. To solve these problems, a cooperative event-triggered fuzzy-neural multivariable controller with multi-task learning (CETFNMC-MTL) is proposed to realize the adaptive multivariable control of MSWI process. First, a fuzzy-neural multivariable controller is established to control furnace temperature and oxygen content synchronously. Second, a dynamic self-organizing mechanism based on multi-task learning is designed, which splits and merges neurons by calculating the dynamic time warping distance and cumulative contribution of neurons in the continuous time. Third, a cooperative event-triggered mechanism is introduced to improve controller update efficiency while reducing mechanical wear and computational burden. Then, the stability of parameters learning and structure self-organizing process is analyzed to guarantee the successful application of CETFNMC-MTL. Finally, the effectiveness of the controller is tested with process data from an MSWI plant in Beijing, China. The results show that the proposed CETFNMC-MTL has adaptive learning ability, while reducing energy consumption and improving control accuracy.

Robust Optimal Parallel Tracking Control Based on Adaptive Dynamic Programming.

This article focuses on a novel robust optimal parallel tracking control method for continuous-time (CT) nonlinear systems subject to uncertainties. First, the designed virtual controller facilitates the transformation of the original nonlinear system into an affine system with an augmented state vector, which promotes the introduction of the optimal parallel tracking control problem. Then, this article generates fresh insight into counteracting the effects of uncertainty by developing a novel parallel control system that invokes the formulated virtual control law and an auxiliary variable obtained from the relationship between the solutions of the optimal control problems for the uncertain system and the nominal one. Next, critic neural networks (NNs) approximate the Hamilton-Jacobi-Bellman (HJB) equations' solution to implement the proposed robust optimal control method via adaptive dynamic programming (ADP). Finally, simulation experiments demonstrate the proposed method's remarkable effectiveness.

Drift Estimation of the Threshold Ornstein-Uhlenbeck Process From Continuous and Discrete Observations

We refer by threshold Ornstein-Uhlenbeck to a continuous-time threshold autoregressive process. It follows the Ornstein-Uhlenbeck dynamics when above or below a fixed level, yet at this level (threshold) its coefficients can be discontinuous. We discuss (quasi)-maximum likelihood estimation of the drift parameters, both assuming continuous and discrete time observations. In the ergodic case, we derive consistency and speed of convergence of these estimators in long time and high frequency. Based on these results, we develop a test for the presence of a threshold in the dynamics. Finally, we apply these statistical tools to short-term US interest rates modeling.

Associations of Near Work, Time Outdoors, and Sleep Duration With Myopic Regression 5 Years After SMILE and FS-LASIK: A Cross-sectional Study.

To investigate the association of near work, time outdoors, and sleep duration with myopic regression 5 years after small incision lenticule extraction (SMILE) and femtosecond laser-assisted in situ keratomileusis (FS-LASIK) . This cross-sectional study included patients who received SMILE or FS-LASIK at Beijing Tongren Hospital 5 years ago. The patients underwent ophthalmic examinations including visual acuity, intraocular pressure, subjective refraction, slit-lamp examination, keratometry, corneal topography, optical coherence tomography, and fundus examination from January 2020 to March 2023. Fluorescein break-up time was measured and the Ocular Surface Disease Index questionnaire was completed to exclude dry eye. A self-administered questionnaire was used to collect data on near work exposure, physical activities, and sleep duration. A total of 323 eyes were included in the analysis, with a 5-year incidence rate of myopic regression after SMILE or FSLASIK of 16.1%. After adjusted for all confounders, total near work more than 8 hours/day revealed a significant association with myopic regression (odds ratio: 2.461; 95% CI: 1.143 to 5.298, P = .021), particularly in younger adults, women, and patients with high myopia and FS-LASIK treatment. The significant association between sleep duration 8 hours/day or more and myopic regression was restricted to women and patients with FS-LASIK (near significant). However, no significant associations were observed between continuous near work or time outdoors and myopic regression. Excessive near work exposure was associated with greater myopic regression 5 years after SMILE or FS-LASIK. It is crucial to maintain good visual behavior and care for preventing regression after SMILE or FS-LASIK, especially for younger patients and eyes with high myopia. [J Refract Surg. 2024;40(1):e10-e19.].

Effect of Virtual Versus In Person Interpreting on Diabetes Outcomes in Non-English Language Preference Patients: A Pilot Study.

The objective of this pilot study was to explore the impact of interpreter format (virtual vs in person) on clinical outcomes in patients with non-English language preference (NELP) and type 2 diabetes mellitus (T2DM) in a primary care setting. We hypothesized that NELP patients utilizing in person interpreters would have improved HbA1c values, better follow-up rate, and more complex care plans compared to patients utilizing virtual interpreters. We completed a retrospective chart review of 137 NELP patients with T2DM who required a medical interpreter (February to June 2021). We calculated univariate and bivariate statistics to characterize the sample and assess the extent to which measures of continuity (follow-up visit rate and time to follow-up visit), quality (change in HbA1c), and complexity (medication intervention complexity) were associated with interpreter type. There was no statistically significant difference in follow-up rate or average days to follow-up visit for NELP patients with in person as opposed to virtual interpreters. Patients with virtual interpreters demonstrated a non-statistically significant decrease in HbA1c compared to those with in person interpreters. Finally, there was no statistically significant association between interpreter format and intervention complexity. Quality medical interpretation contributes to optimal health outcomes in NELP patients with diabetes. Our study suggests that both in person and virtual interpreters can be effective in providing care for NELP patients, especially for chronic disease management in the context of a primary care relationship. It also highlights the importance of pursuing additional qualitative and mixed method studies to better understand the benefits of various interpreter formats across different visit types.

Layerwise Quantum Deep Reinforcement Learning for Joint Optimization of UAV Trajectory and Resource Allocation

This study proposes a layerwise quantum-based deep reinforcement learning (LQ-DRL) method for optimizing continuous large space and time series problems using deep layer training. The actions in LQ-DRL are optimized using a layerwise quantum embedding that leverages the advantages of quantum computing to maximize reward and reduce training loss. Moreover, this study employs a local loss to minimize the occurrence of barren plateaus phenomena and further enhance performance. As a particular case, the proposed scheme is employed to jointly optimize: (1) UAV trajectory planning, (2) user grouping, and (3) power allocation for higher energy efficiency of a UAV as the reward. The combination of these optimized factors is referred to as action space in the presented LQ-DRL. The LQ-DRL is employed to solve the optimization problem due to its non-convexity, continuous and large action space, and time-series domain. In a practical view, LQ-DRL aims to solve the issue of energy consumption related to limited-battery energy of a UAV base station while maintaining quality-of-service (QoS) for users, by gaining maximum energy efficiency as the reward. One of real applications, as an example, LQ-DRL can be employed to maximize the energy efficiency of a UAV base station in UAV empowered disaster recovery networks scenario. The quantum circuits of layerwise quantum embedding are presented to show the practical implementation in noisy intermediate-scale quantum computers. Based on the results, LQ-DRL outperformed the classical DRL by achieving higher effective dimension, rewards, and lower learning losses. In addition, better performances were achieved using more layers.

Analysis of the vanishing discount limit for optimal control problems in continuous and discrete time

Analysis of the vanishing discount limit for optimal control problems in continuous and discrete time

Extreme measures in continuous time Conic Finance

Extreme measures in continuous time Conic Finance

On Model of Recurrent Neural Network on a Time Scale: Exponential Convergence and Stability Research.

The majority of the results on modeling recurrent neural networks (RNNs) are obtained using delayed differential equations, which imply continuous time representation. On the other hand, these models must be discrete in time, given their practical implementation in computer systems, requiring their versatile utilization across arbitrary time scales. Hence, the goal of this research is to model and investigate the architecture design of a delayed RNN using delayed differential equations on a time scale. Internal memory can be utilized to describe the calculation of the future states using discrete and distributed delays, which is a representation of the deep learning architecture for artificial RNNs. We focus on qualitative behavior and stability study of the system. Special attention is paid to taking into account the effect of the time-scale parameters on neural network dynamics. Here, we delve into the exploration of exponential stability in RNN models on a time scale that incorporates multiple discrete and distributed delays. Two approaches for constructing exponential estimates, including the Hilger and the usual exponential functions, are considered and compared. The Lyapunov-Krasovskii (L-K) functional method is employed to study stability on a time scale in both cases. The established stability criteria, resulting in an exponential-like estimate, utilizes a tuple of positive definite matrices, decay rate, and graininess of the time scale. The models of RNNs for the two-neuron network with four discrete and distributed delays, as well as the ring lattice delayed network of seven identical neurons, are numerically investigated. The results indicate how the time scale (graininess) and model characteristics (weights) influence the qualitative behavior, leading to a transition from stable focus to quasiperiodic limit cycles.

Applicability of Utilizing Remote Sensing Rainfall Products Data in Arid and Semi-Arid Poorly Gauged Catchments: Study of Wadi Ghoweiba Watershed, Egypt

AbstractDue to the scarcity of established rain gauge stations, obtaining continuous time series daily rainfall observation data is a major challenge. The availability of these data is very essential to conduct hydrological studies and predicting flood events. Nowadays, several satellites could be used to provide such data. This study aims to investigate the appropriate remote sensing (RS) rainfall product and use its data to develop frequency analysis and intensity–duration–frequency curve (IDF curve) at 5-, 10-, 25-, 50-, 100-, and 200-year return periods in the study area. The evaluation of the PERSIANN family products (PERSIANN, PERSIANN-CCS, PERSIANN-CDR, and PDIR-Now) with NASA-POWER datasets compared to the available annual maximum of daily observation rainfall in Suez Rain Gauge (SRG) station as a reference was conducted. The results show that the PERSIANN-CDR product is the appropriate satellite-based rainfall dataset product based on the outcomes of the comparison criteria (RMSE, Bias, CC, and R2 error measures) of rainfall characteristics analysis and Chi-squared test of distributions. The frequency analysis results confirmed that the values of about 30.3–38.6 and 40–53.3 mm/day corresponding to the biggest flood reported in 1965 and 2020 in SRG station and PERSIANN-CDR product have a return period of 50–100 years, classifying it as an extreme event in Wadi Ghoweiba. These findings would considerably benefit decision-makers in estimating flood risks and planning the appropriate structure protections in the Wadi Ghoweiba Watershed.

Reliability analysis of three state linear consecutive 2-out-of-4:F system by continuous time Markov model

Reliability analysis of three state linear consecutive 2-out-of-4:F system by continuous time Markov model

Signal Propagation: The Framework for Learning and Inference in a Forward Pass.

We propose a new learning framework, signal propagation (sigprop), for propagating a learning signal and updating neural network parameters via a forward pass, as an alternative to backpropagation (BP). In sigprop, there is only the forward path for inference and learning. So, there are no structural or computational constraints necessary for learning to take place, beyond the inference model itself, such as feedback connectivity, weight transport, or a backward pass, which exist under BP-based approaches. That is, sigprop enables global supervised learning with only a forward path. This is ideal for parallel training of layers or modules. In biology, this explains how neurons without feedback connections can still receive a global learning signal. In hardware, this provides an approach for global supervised learning without backward connectivity. Sigprop by construction has compatibility with models of learning in the brain and in hardware than BP, including alternative approaches relaxing learning constraints. We also demonstrate that sigprop is more efficient in time and memory than they are. To further explain the behavior of sigprop, we provide evidence that sigprop provides useful learning signals in context to BP. To further support relevance to biological and hardware learning, we use sigprop to train continuous time neural networks with the Hebbian updates and train spiking neural networks (SNNs) with only the voltage or with biologically and hardware-compatible surrogate functions.

A continuous time projection algorithm for solving elastic-net regularization and application to signal recovery

A continuous time projection algorithm for solving elastic-net regularization and application to signal recovery

Persistence in a dynamic moral hazard game

This paper explores how the persistence of past choices creates incentives in a continuous time stochastic game involving a large player (e.g., a firm) and a sequence of small players (e.g., customers). The large player faces moral hazard and her actions are distorted by a Brownian motion. Persistence refers to how actions impact a payoff‐relevant state variable (e.g., product quality depends on past investment). I characterize actions and payoffs in Markov perfect equilibria (MPE) for a fixed discount rate, show that the perfect public equilibrium (PPE) payoff set is the convex hull of the MPE payoff set, and derive sufficient conditions for a MPE to be the unique PPE. Persistence can serve as an effective channel for intertemporal incentives in a setting where traditional channels fail. Applications to persistent product quality and policy targeting demonstrate the impact of persistence on equilibrium behavior.