Continuous-time Model Research Articles

Layer Compression of Deep Networks with Straight Flows

Very deep neural networks lead to significantly better performance on various real tasks. However, it usually causes slow inference and is hard to be deployed on real-world devices. How to reduce the number of layers to save memory and to accelerate the inference is an eye-catching topic. In this work, we introduce an intermediate objective, a continuous-time network, before distilling deep networks into shallow networks. First, we distill a given deep network into a continuous-time neural flow model, which can be discretized with an ODE solver and the inference requires passing through the network multiple times. By forcing the flow transport trajectory to be straight lines, we find that it is easier to compress the infinite step model into a one-step neural flow model, which only requires passing through the flow model once. Secondly, we refine the one-step flow model together with the final head layer with knowledge distillation and finally, we can replace the given deep network with this one-step flow network. Empirically, we demonstrate that our method outperforms direct distillation and other baselines on different model architectures (e.g. ResNet, ViT) on image classification and semantic segmentation tasks. We also manifest that our distilled model naturally serves as an early-exit dynamic inference model.

When Prospect Theory Meets Mean-Reverting Asset Returns: A Behavioral Dynamic Trading Model

We develop a continuous-time asset allocation model to investigate the effects of mean-reverting stock returns on investors with Prospect Theory (PT) preferences. Our semi-analytical solution facilitates a comprehensive exploration of how the stock investment of PT investors may differ when accounting for mean reversion. We find that incorporating mean reversion attenuates the distinct V-shaped demand pattern in relation to contemporaneous prices, which is more pronounced when mean reversion is absent, by significantly reducing PT investors’ stock demand following price increases. This shift leads to a stock demand profile that demonstrates an inverse relationship with stock prices. In line with this change, we also show that combining PT utility with mean reversion predicts short-term contrarian behavior and the disposition effect more reliably than benchmark models that incorporate either PT utility or mean-reverting returns alone.

Abstract 1251: Tobacco smoke alters the adaptive landscape of lung adenocarcinoma and influences the strength of epistatic interactions

Abstract Tobacco smoke produces both mutagenic and physiological effects, altering the mutation rate of cells and the selection acting on them in the damaged lung environment, and facilitating its role in driving cancer progression. In addition to these gene-by-environment interactions, gene-by-gene epistatic interactions also shape the adaptive landscape, influencing the evolutionary trajectories of cancer. Understanding the influence of tobacco smoke on the epistatic trajectories of lung cancer progression would help guide the development of targeted therapies that are more effective for smoker or never-smoker populations. To address this need, we constructed a continuous-time Markov chain model for the evolutionary trajectories of lung adenocarcinoma (LUAD), the most common subtype of lung cancer and the most frequent subtype among never-smokers. Using thousands of tumor sample sequences aggregated across studies, we profiled the trajectories of LUAD in smokers and never-smokers and estimated the rates at which mutations are gained from each possible set of pre-existing mutations, or genetic states. We then accounted for differences in baseline mutation rates to quantify the selective benefit of mutations. We find that epistasis is prevalent in LUAD, with several strong synergistic interactions—such as between RB1 and EGFR mutations- and antagonistic interactions-such as between EGFR and KEAP1 or KRAS mutations. Additionally, we identify non-additive epistatic effects: STK11 mutations are synergistic with KEAP1 and KRAS mutations but do not experience additional selection when both are mutated. In contrast, GNAS mutations are not synergistic with KEAP1 or RBM10 mutations alone but are synergistic with their co-mutation. Smoking has a large influence on the adaptive landscape, with several common mutations being preferentially selected in never-smoker cancers, including EGFR, SMAD4, GNAS, and PIK3CA mutations. KEAP1, STK11, and KRAS mutations are preferentially selected in smoker cancers. Smoking’s physiological influence also affects the strength of certain epistatic interactions, such as between STK11 and KEAP1 mutations. Overall, we find that smoking not only increases the mutation rate of lung cells but also substantially alters the adaptive landscape of lung adenocarcinoma, leading to clinically relevant differences in selection on mutations between smoker and never-smoker cancers. We additionally detect frequent pairwise and higher-order epistatic effects in LUAD that may inform the personalized application of targeted therapies. Citation Format: Krishna Dasari, Jorge Alfaro-Murillo, Jeffrey Townsend. Tobacco smoke alters the adaptive landscape of lung adenocarcinoma and influences the strength of epistatic interactions [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 1251.

Estimating fish production in wetlands

Fish production integrates changes in biomass from growth, reproduction, and mortality, and is a useful indicator for fisheries management. However, calculation of fish production has been limited by the intensive requirements for data on abundance, biomass, age structure, and vital rates, and so it is uncommon to find estimates of fish production for wetlands. We developed an approach to directly estimate production that is suitable for data-limited systems: a continuous time model of production describing individual growth over short time intervals with continuous time models of abundance and biomass over longer timescales. We applied this model for 18 Great Lakes coastal wetlands (GLCWs) on Lake Ontario, including Big Island Wetland (BIW). In BIW, most species were dominated in abundance and biomass by younger cohorts and, as a result, these young, fast-growing individuals contributed disproportionately to fish production. In total, BIW produced 336.1 kg·ha·year of fish and the other 17 neighbouring ranged between 447.7 and 1119.9 kg·ha·year. These are some of the first estimates of fish production for GLCWs, highlighting their value for managing Great Lakes fisheries.

Resilience evaluation of train control on-board system considering common cause failure: Based on a beta-factor and continuous-time bayesian network model

Common cause failure is an inevitable factor in studying the reliability or resilience of a system with redundant structures. Train Control On Board System (TCOBS) is a typical redundant structural system. However, there are few studies on the resilience of TCOBS that consider the issue of common cause failure, which means the resilience assessment results might be imprecise. At the same time, previous TCOBS resilience evaluation works also have problems such as unreasonable resilience evaluation functions and low accuracy of resilience evaluation models. To address the above issues, in this paper, we firstly construct the Bayesian network structure of TCOBS and present the resilience function of TCOBS components. Secondly, a Beta-factor model is used to evaluate the common cause failure in TCOBS, and a continuous-time Bayesian network model is used to infer TCOBS resilience and the importance of each component. Finally, CTCS-3OBS is used as an example for case simulation and model comparison analysis. The results show that if the impact of common cause failure on TCOBS resilience is not considered, it will lead to a higher result of resilience assessment and significant differences in component importance assessment, both of which will lead to inaccurate formulation of TCOBS resilience improvement measures.

NIRK-based mixed-type accurate continuous–discrete Gaussian filters with deterministically sampled expectation and covariance for state estimation in continuous-time stochastic process models with discrete measurements

This paper advances further the idea of overall continuous–discrete Gaussian filtering with deterministically sampled expectation and covariance towards efficient mixed-type accurate Kalman-like schemes based on self-regulated Ordinary Differential Equation (ODE) solvers with automatic local and global error controls intended for treating continuous-time nonlinear stochastic process models with discrete measurements. The universal state estimation algorithms of such sort designed recently by Kulikov and Kulikova (2022) possess three potential limitations, which might affect their successful applications in practice. First, these deal with complicated Moment Differential Equations (MDEs) and, hence, can be rather time-consuming in estimating stochastic process models of large size, that compromises their efficiency for online computations. Second, the aforementioned filters do not employ any kind of global error regulation and, hence, can be insufficiently accurate in solving difficult MDEs arisen. Third, these are hardly effective in treating stiff continuous–discrete stochastic systems. Below, all the listed issues are addressed and solved via utilization of the much simpler but more robust MDEs arisen in the usual continuous–discrete Extended Kalman Filter (EKF) within the mixed-type accurate filtering approach initiated by Kulikova and Kulikov (2015), which is rooted in the variable-stepsize Nested Implicit Runge–Kutta (NIRK) methods. For the sake of stability of our novel methods to the numerical integration and round-off errors, we devise square-root versions of the mixed-type Gaussian filters, which are based either on hyperbolic QR factorizations or on hyperbolic Singular Value Decompositions (SVDs). Performances of these new mixed-type square-root Gaussian filters are assessed and compared to their original (non-mixed-type) counterparts as well as to their non-square-root versions in a simulated non-stiff ill-conditioned radar tracking scenario and on a stiff stochastic Oregonator reaction.

Discrete-time zeroing neural network with quintic error mode for time-dependent nonlinear equation and its application to robot arms

Time-dependent nonlinear equation (TDNE) arises in numerous engineering applications. Recently, zeroing neural network (ZNN) has been proven to be an effective alternative for solving the TDNE. In this paper, we present a new solution to the TDNE by using the discrete-time ZNN (DTZNN). Specifically, a special difference formula is first constructed via Taylor series expansion. Then, by utilizing such a formula to discretize the existing continuous-time ZNN model, the new DTZNN model is proposed to determine the TDNE solution. Theoretical analysis and numerical results further indicate the validity and superiority of the proposed DTZNN model in comparison with the previous models. Finally, the DTZNN practicality is presented by the simulation and experiment on the DOBOT arm using the proposed model.

Household Decision-Making Choices: Investment in Children's Education or Self-Consumption.

Analyzing the sustainable decision-making mechanism between household consumption and education investment can theoretically develop education. This study uses the continuous-time utility model to demonstrate the independent characteristics of consumption and education investment, as well as the principle of decision incompatibility in the decision-making process of the utility maximization problem. Then, we establish a three-phase logarithmic utility model to obtain the intertemporal decision-making path of a family. The analysis shows that the investment allocation ratio between the two phases depends on the expected and discounted level of the offsprings' abilities, while the total investment level is related to parental altruism. When parents, with foresight, factor in prospective transfer payments from progeny, the optimal decision is to maximize their children's ultimate human capital within a given total investment. Education investment not only squeezes out consumption but also promotes consumption in various periods due to future transfer payments. The decision-making process of three typical growth stages indicates that as offspring mature and their human capital increases, parents' willingness to invest in education decreases while self-consumption escalates. This study provides a new perspective and theoretical basis for studying household education expenditure, motivation, and related policy formulation.

Optimal Consumption and Investment with Independent Stochastic Labor Income

We develop a new dynamic continuous-time model of optimal consumption and investment to include independent stochastic labor income. We reduce the problem of solving the Bellman equation to a problem of solving an integral equation. We then explicitly characterize the optimal consumption and investment strategy as a function of income-to-wealth ratio. We provide some analytical comparative statics associated with the value function and optimal strategies. We also develop a quite general numerical algorithm for control iteration and solve the Bellman equation as a sequence of solutions to ordinary differential equations. This numerical algorithm can be readily applied to many other optimal consumption and investment problems especially with extra nondiversifiable Brownian risks, resulting in nonlinear Bellman equations. Finally, our numerical analysis illustrates how the presence of stochastic labor income affects the optimal consumption and investment strategy. Funding: A. Bensoussan was supported by the National Science Foundation under grant [DMS-2204795]. S. Park was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea, South Korea [NRF-2022S1A3A2A02089950].

Supercritical and subcritical aeroelastic behaviors of a three-dimensional wing coupled with a nonlinear energy sink

This paper studies the flutter control of a three-dimensional wing using a nonlinear energy sink (NES), which has a purely cubic stiffness and a linear damper. The structural model is derived by Lagrange's equations and the aerodynamics is modeled using the auto regressive with exogenous input method. Both models are solved in a tightly coupling manner to obtain time domain results. Then, the structural describing function and the aerodynamic continuous-time state space model are used to derive the equivalent linearized aeroelastic system, so that eigenvalue analyses can be used to obtain stable and unstable limit cycle oscillations (LCOs). The LCOs obtained by time domain simulations agree well with the equivalent linearization method, while differences are observed for aperiodic motions which are only predicted in time domain for small damping coefficients. Results show the square of aeroelastic amplitude is inversely proportional to the NES stiffness coefficient for both periodic and aperiodic motions. The NES can induce LCOs below flutter speed for small damping coefficients, indicating a subcritical bifurcation which can change to a supercritical one by increasing the damping coefficient. It is discovered that the upper bound of partial suppression region is quantified by a turning point speed. The regions of complete suppression, partial suppression and no suppression are clarified.

Optimal Inference for Spot Regressions

Betas from return regressions are commonly used to measure systematic financial market risks. “Good” beta measurements are essential for a range of empirical inquiries in finance and macroeconomics. We introduce a novel econometric framework for the nonparametric estimation of time-varying betas with high-frequency data. The “local Gaussian” property of the generic continuous-time benchmark model enables optimal “finite-sample” inference in a well-defined sense. It also affords more reliable inference in empirically realistic settings compared to conventional large-sample approaches. Two applications pertaining to the tracking performance of leveraged ETFs and an intraday event study illustrate the practical usefulness of the new procedures. (JEL C22, C58, G12, G23)

A Bayesian approach to identifying the role of hospital structure and staff interactions in nosocomial transmission of SARS-CoV-2.

Nosocomial infections threaten patient safety, and were widely reported during the COVID-19 pandemic. Effective hospital infection control requires a detailed understanding of the role of different transmission pathways, yet these are poorly quantified. Using patient and staff data from a large UK hospital, we demonstrate a method to infer unobserved epidemiological event times efficiently and disentangle the infectious pressure dynamics by ward. A stochastic individual-level, continuous-time state-transition model was constructed to model transmission of SARS-CoV-2, incorporating a dynamic staff-patient contact network as time-varying parameters. A Metropolis-Hastings Markov chain Monte Carlo (MCMC) algorithm was used to estimate transmission rate parameters associated with each possible source of infection, and the unobserved infection and recovery times. We found that the total infectious pressure exerted on an individual in a ward varied over time, as did the primary source of transmission. There was marked heterogeneity between wards; each ward experienced unique infectious pressure over time. Hospital infection control should consider the role of between-ward movement of staff as a key infectious source of nosocomial infection for SARS-CoV-2. With further development, this method could be implemented routinely for real-time monitoring of nosocomial transmission and to evaluate interventions.

Bid-ask spread dynamics: large upward jump with geometric catastrophes

We propose a simple continuous-time stochastic model for capturing the dynamics of a limit order book in the presence of liquidity fluctuations, manifested by gaps in filled price levels within the OB. Inspired by [D. Farmer, L. Gillemot, F. Lillo, S. Mike and A. Sen, Quant. Finance 4 (2004) 383–397.], we define a model for the dynamics of spread that incorporates liquidity fluctuations and undertake a comprehensive theoretical study of the model’s properties, providing rigorous proofs of several key asymptotic theorems. Furthermore, we show how large deviations manifest in the spread under this regime.

Development and Implementation of Physics-Informed Neural ODE for Dynamics Modeling of a Fixed-Wing Aircraft Under Icing/Fault

Accurate dynamics modeling is crucial for the safety and control of fixed-wing aircraft under perturbation (e.g. icing/fault). In this work, we propose a physics-informed Neural Ordinary Differential Equation (PI-NODE)-based scheme for aircraft dynamics modeling under icing/fault. First, icing accumulation and control surface faults are considered and injected into the nominal (clean) aircraft dynamics model. Second, the physics knowledge of aircraft dynamics modeling is divided into kinematics and kinetics. The former is universally applicable and borrows directly from the nominal aircraft. The latter kinetics knowledge, which hinges on external forces and moments, is inaccurate and challenging under icing/fault. To address this issue, we employ Neural ODE to compensate for the residual between the aircraft dynamics under icing/fault and the nominal (clean) condition, resulting in a naturally continuous-time modeling approach. In experiments, we benchmark the proposed PI-NODE against three baseline methods in a dedicated flight scenario. Comparative studies validate the higher accuracy and improve the generalization ability of the proposed PI-NODE for aircraft dynamics modeling under icing/fault.

Quantum transport on multilayer generalized scale-free networks

Abstract We study single-particle quantum transport on multilayer generalized scale-free networks using the continuous-time quantum walk model. Our focus is directed at the average return probability and its long-time average value as measures for the transport efficiency. In the continuous-time model these quantities are completely determined by all the eigenvalues and eigenvectors of the connectivity matrix. For all multilayer networks a nontrivial interplay between good spreading and localization effects is observed. The spreading is enhanced by increasing the number of layers L or the power-law exponent γ of the degree distribution. For our choice of the parameters, namely L (1 ≤ L ≤ 50) or γ (1 ≤ γ ≤ 4), the quantum efficiency is increased by at least one order of magnitude. The topological transition between networks without loops, which corresponds to a single scale-free network layer (L = 1), and networks with loops (L = 2) is the most impactful. Another important change occurs when L gets higher than the average diameter d of the layers, namely a new scaling behavior for random walks and lower fluctuations around the long-time average value for quantum walks. The quantum transport is more sensitive to changes of the minimum allowed degree, K min, than to the maximum allowed degree, K max. The same quantum efficiency is found by varying at least one of the parameters: L, γ, K min, or K max, although the network’s topology is different. The quantum efficiency of all multilayer scale-free networks shows a universal behavior for any size of the layers, more precise, is inversely proportional to the number of layers.

Explosion and non-explosion for the continuous-time frog model

We consider the continuous-time frog model on Z. At time t=0, there are η(x) particles at x∈Z, each of which is represented by a random variable. In particular, (η(x))x∈Z is a collection of independent random variables with a common distribution μ, μ(Z+)=1, Z+≔N∪{0}, N={1,2,3,…}. The particles at the origin are active, all other ones being assumed as dormant, or sleeping, hence not active. Active particles perform a simple symmetric continuous-time random walk in Z (that is, a random walk with exp(1)-distributed jump times and jumps −1 and 1, each with probability 1/2), independently of all other particles. Sleeping particles stay still until the first arrival of an active particle to their location; upon arrival they become active and start their own simple random walks. Different sets of conditions are given ensuring explosion, respectively non-explosion, of the continuous-time frog model. Our results show in particular that if μ is the distribution of eYlnY with a non-negative random variable Y satisfying EY<∞, then a.s. no explosion occurs. On the other hand, if a∈(0,1) and μ is the distribution of eX, where P{X≥t}=t−a, t≥1, then explosion occurs a.s. The proof relies on a certain type of comparison to a percolation model which we call totally asymmetric discrete inhomogeneous Boolean percolation.

Utilizing a multi-stage transition model for analysing child stunting in two urban slum settlements of Nairobi: A longitudinal analysis, 2011-2014.

Stunting is common among children in many low and middle income countries, particularly in rural and urban slum settings. Few studies have described child stunting transitions and the associated factors in urban slum settlements. We describe transitions between stunting states and associated factors among children living in Nairobi slum settlements. This study used data collected between 2010 and 2014 from the Nairobi Urban and Demographic Surveillance System (NUHDSS) and a vaccination study nested within the surveillance system. A subset of 692 children aged 0 to 3 years, with complete anthropometric data, and household socio-demographic data was used for the analysis. Height-for-age Z-scores (HAZ) was used to define stunting: normal (HAZ ≥ 1), marginally stunted (-2 ≤ HAZ < -1), moderately stunted (-3 ≤ HAZ < -2), and severely stunted (HAZ < -3). Transitions from one stunting level to another and in the reverse direction were computed. The associations between explanatory factors and the transitions between four child stunting states were modeled using a continuous-time multi-state model. We observed that 48%, 39%, 41%, and 52% of children remained in the normal, marginally stunted, moderately stunted, and severely stunted states, respectively. About 29% transitioned from normal to marginally stunted state, 15% to the moderately stunted state, and 8% to the severely stunted state. Also, 8%, 12%, and 29% back transitioned from severely stunted, moderately stunted, and marginally stunted states, to the normal state, respectively. The shared common factors associated with all transitions to a more severe state include: male gender, ethnicity (only for mild and severe transition states), child's age, and household food insecurity. In Korogocho, children whose parents were married and those whose mothers had attained primary or post-primary education were associated with a transition from a mild state into a moderately stunted state. Children who were breastfed exclusively were less likely to transition from moderate to severe stunting state. These findings reveal a high burden of stunting and transitions in urban slums. Context-specific interventions targeting the groups of children identified by the socio-demographic factors are needed. Improving food security and exclusive breastfeeding could potentially reduce stunting in the slums.

High-dimensional stochastic control models for newsvendor problems and deep learning resolution

AbstractThis paper studies continuous-time models for newsvendor problems with dynamic replenishment, financial hedging and Stackelberg competition. These factors are considered simultaneously and the high-dimensional stochastic control models are established. High-dimensional Hamilton-Jacobi-Bellman (HJB) equations are derived for the value functions. To circumvent the curse of dimensionality, a deep learning algorithm is proposed to solve the HJB equations. A projection is introduced in the algorithm to avoid the gradient explosion during the training phase. The deep learning algorithm is implemented for HJB equations derived from the newsvendor models with dimensions up to six. Numerical outcomes validate the algorithm’s accuracy and demonstrate that the high-dimensional stochastic control models can successfully mitigate the risk.

Rayleigh step-selection functions and connections to continuous-time mechanistic movement models

BackgroundThe process known as ecological diffusion emerges from a first principles view of animal movement, but ecological diffusion and other partial differential equation models can be difficult to fit to data. Step-selection functions (SSFs), on the other hand, have emerged as powerful practical tools for ecologists studying the movement and habitat selection of animals.MethodsSSFs typically involve comparing resources between a set of used and available points at each step in a sequence of observed positions. We use change of variables to show that ecological diffusion implies certain distributions for available steps that are more flexible than others commonly used. We then demonstrate advantages of these distributions with SSF models fit to data collected for a mountain lion in Colorado, USA.ResultsWe show that connections between ecological diffusion and SSFs imply a Rayleigh step-length distribution and uniform turning angle distribution, which can accommodate data collected at irregular time intervals. The results of fitting an SSF model with these distributions compared to a set of commonly used distributions revealed how precision and inference can vary between the two approaches.ConclusionsOur new continuous-time step-length distribution can be integrated into various forms of SSFs, making them applicable to data sets with irregular time intervals between successive animal locations.

Abstract IA014: Detecting epistatic effects among somatic cancer mutations across oncogenesis

Abstract The somatic evolution of cancer depends on the rate at which new mutations are acquired, as well as the selective pressure acting on them. Further, mutual exclusivity or co-occurrence of somatic variants suggests that current somatic variants can exert antagonistic or synergistic epistatic effects on the selection of new mutations, contributing to a complex landscape of evolutionary trajectories. However, quantifying these epistatic effects—which are essential to estimation of the trajectory of likely cancer genotoypes—has been a challenge, especially for higher-order epistatic effects. To estimate these epistatic effects, we have developed a continuous-time Markov chain model that enables the estimation of mutation origination and fixation, dependent on somatic cancer genotype. Coupling the continuous-time Markov chain model with a mutation-rate deconvolution approach, we have estimated the underlying mutation rates and selection intensities across trajectories of oncogenesis. Specifically, we report the mutation rates, selection intensities, and consequent variant fluxes for lung adenocarcinoma, informed by a compilation of samples from multiple data sets including The Cancer Genome Atlas and the AACR Project GENIE. Furthermore, we compare smoker and nonsmoker tumor cohorts describing the distinct evolutionary trajectories jointly, influenced by differential mutation and also by the differential physiological effects that smoking can have on the selective regime of lung tissue. Our approach enables inference of the most likely routes of site-specific variant evolution, as well as estimation of the selection strength operating on each step along the route—a key component of what we need to know to prioritize, develop, and implement personalized cancer therapies. Citation Format: Jorge A. Alfaro-Murillo, Krishna Dasari, Jeffrey P. Townsend. Detecting epistatic effects among somatic cancer mutations across oncogenesis [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Translating Cancer Evolution and Data Science: The Next Frontier; 2023 Dec 3-6; Boston, Massachusetts. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_2):Abstract nr IA014.