Parametric Ordinary Differential Equations Research Articles

Comprehending symmetry in epidemiology: A review of analytical methods and insights from models of COVID-19, Ebola, Dengue, and Monkeypox.

The challenge of developing comprehensive mathematical models for guiding public health initiatives in disease control is varied. Creating complex models is essential to understanding the mechanics of the spread of infectious diseases. We reviewed papers that synthesized various mathematical models and analytical methods applied in epidemiological studies with a focus on infectious diseases such as Severe Acute Respiratory Syndrome Coronavirus-2, Ebola, Dengue, and Monkeypox. We address past shortcomings, including difficulties in simulating population growth, treatment efficacy and data collection dependability. We recently came up with highly specific and cost-effective diagnostic techniques for early virus detection. This research includes stability analysis, geographical modeling, fractional calculus, new techniques, and validated solvers such as validating solver for parametric ordinary differential equation. The study examines the consequences of different models, equilibrium points, and stability through a thorough qualitative analysis, highlighting the reliability of fractional order derivatives in representing the dynamics of infectious diseases. Unlike standard integer-order approaches, fractional calculus captures the memory and hereditary aspects of disease processes, resulting in a more complex and realistic representation of disease dynamics. This study underlines the impact of public health measures and the critical importance of spatial modeling in detecting transmission zones and informing targeted interventions. The results highlight the need for ongoing financing for research, especially beyond the coronavirus, and address the difficulties in converting analytically complicated findings into practical public health recommendations. Overall, this review emphasizes that further research and innovation in these areas are crucial for addressing ongoing and future public health challenges.

A Hybrid Sobolev Gradient Method for Learning NODEs

The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in ordinary differential equations is considered, with the typical application of finding weights of a neural ordinary differential equation (NODE) for a residual network with time continuous layers. The differential equation is treated as an abstract and isolated entity, termed a standalone NODE (sNODE), to facilitate for a wide range of applications. The proposed parameter reconstruction is performed by minimizing a cost functional covering a variety of loss functions and penalty terms. Regularization via penalty terms is incorporated to enhance ethical and trustworthy AI formulations. A nonlinear conjugate gradient mini-batch optimization scheme (NCG) is derived for the training having the benefit of including a sensitivity problem. The model (differential equation)-based approach is thus combined with a data-driven learning procedure. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with the sensitivity problem. The sensitivity problem itself can estimate changes in the output under perturbation of the trained parameters. To preserve smoothness during the iterations, the Sobolev gradient is calculated and incorporated. Numerical results are included to validate the procedure for a NODE and synthetic datasets and compared with standard gradient approaches. For stability, using the sensitivity problem, a strategy for adversarial attacks is constructed, and it is shown that the given method with Sobolev gradients is more robust than standard approaches for parameter identification.

End-to-End Statistical Model Checking for Parameterization and Stability Analysis of ODE Models

We propose a simulation-based technique for the parameterization and the stability analysis of parametric Ordinary Differential Equations. This technique is an adaptation of Statistical Model Checking, often used to verify the validity of biological models, to the setting of Ordinary Differential Equations systems. The aim of our technique is to estimate the probability of satisfying a given property under the variability of the parameter or initial condition of the ODE, with any metrics of choice. To do so, we discretize the values space and use statistical model checking to evaluate each individual value w.r.t. provided data. Contrary to other existing methods, we provide statistical guarantees regarding our results that take into account the unavoidable approximation errors introduced through the numerical integration of the ODE system performed while simulating. In order to show the potential of our technique, we present its application to two case studies taken from the literature, one relative to the growth of a jellyfish population, and the other concerning a well-known oscillator model.

Computing subgradients of convex relaxations for solutions of parametric ordinary differential equations

A novel subgradient evaluation method is proposed for nonsmooth convex relaxations of parametric solutions of ordinary differential equations (ODEs) arising in global dynamic optimization, assuming that the relaxations always lie strictly within interval bounds during integration. We argue that this assumption is reasonable in practice. These subgradients are computed as the unique solution of an auxiliary parametric affine ODE, analogous to classical forward/tangent sensitivity evaluation methods for smooth dynamic systems. Unlike established subgradient evaluation approaches for nonsmooth dynamic systems, this new method does not require smoothness or transversality assumptions, and is compatible with existing subgradient evaluation methods for closed-form convex functions, as implemented in subgradient evaluation software such as EAGO.jl and MC ++. Moreover, we show that a subgradient for a lower-bounding problem in global dynamic optimization can be directly evaluated using reverse/adjoint sensitivity analysis, which may reduce the overall computational effort for an overarching global optimization method. Numerical examples are presented, based on a proof-of-concept implementation in Julia.

Characterization of partial wetting by CMAS droplets using multiphase many-body dissipative particle dynamics and data-driven discovery based on PINNs

The molten sand that is a mixture of calcia, magnesia, alumina and silicate, known as CMAS, is characterized by its high viscosity, density and surface tension. The unique properties of CMAS make it a challenging material to deal with in high-temperature applications, requiring innovative solutions and materials to prevent its buildup and damage to critical equipment. Here, we use multiphase many-body dissipative particle dynamics simulations to study the wetting dynamics of highly viscous molten CMAS droplets. The simulations are performed in three dimensions, with varying initial droplet sizes and equilibrium contact angles. We propose a parametric ordinary differential equation (ODE) that captures the spreading radius behaviour of the CMAS droplets. The ODE parameters are then identified based on the physics-informed neural network (PINN) framework. Subsequently, the closed-form dependency of parameter values found by the PINN on the initial radii and contact angles are given using symbolic regression. Finally, we employ Bayesian PINNs (B-PINNs) to assess and quantify the uncertainty associated with the discovered parameters. In brief, this study provides insight into spreading dynamics of CMAS droplets by fusing simple parametric ODE modelling and state-of-the-art machine-learning techniques.

A Bayesian Collocation Integral Method for Parameter Estimation in Ordinary Differential Equations

Inferring the parameters of ordinary differential equations (ODEs) from noisy observations is an important problem in many scientific fields. Currently, most parameter estimation methods that bypass numerical integration tend to rely on basis functions or Gaussian processes to approximate the ODE solution and its derivatives. Due to the sensitivity of the ODE solution to its derivatives, these methods can be hindered by estimation error, especially when only sparse time-course observations are available. We present a Bayesian collocation framework that operates on the integrated form of the ODEs and also avoids the expensive use of numerical solvers. Our methodology has the capability to handle general nonlinear ODE systems. We demonstrate the accuracy of the proposed method through simulation studies, where the estimated parameters and recovered system trajectories are compared with other recent methods. A real data example is also provided. Supplementary materials for this article are available online.

Fourier series-based approximation of time-varying parameters in ordinary differential equations

Many real-world systems modeled using differential equations involve unknown or uncertain parameters. Standard approaches to address parameter estimation inverse problems in this setting typically focus on estimating constants; yet some unobservable system parameters may vary with time without known evolution models. In this work, we propose a novel approximation method inspired by the Fourier series to estimate time-varying parameters (TVPs) in deterministic dynamical systems modeled with ordinary differential equations. Using ensemble Kalman filtering in conjunction with Fourier series-based approximation models, we detail two possible implementation schemes for sequentially updating the time-varying parameter estimates given noisy observations of the system states. We demonstrate the capabilities of the proposed approach in estimating periodic parameters, both when the period is known and unknown, as well as non-periodic TVPs of different forms with several computed examples using a forced harmonic oscillator. Results emphasize the importance of the frequencies and number of approximation model terms on the time-varying parameter estimates and corresponding dynamical system predictions.

ИНТЕГРАЦИЯ МЕТОДА ФУНКЦИЙ ЛЯПУНОВА ДЛЯ АНАЛИЗА УСТОЙЧИВОСТИ ЭЛЕКТРООБОРУДОВАНИЯ АПК

The functioning of agro-industrial enterprises is considered from electrical equipment operation, quality and stability of automated controlled systems. Energy consumption management is an important task of agro-industrial enterprises, directly related to the stability of electric energy conversion devices. The stability analysis model implemented by the authors makes it possible to improve the quality of control of electrical equipment. (Research purpose) The research purpose is developing a method for ensuring the stability of electrical equipment of agricultural enterprises using Lyapunov functions. (Materials and methods) Proposed the integration of the Lyapunov function method into the agro-industrial complex management systems for stability analysis and forecasting of the state of electrical equipment. The stator currents of the drive motor and the speed characteristics of the rotor rotation were analyzed. We studied the electric motor as a second-order link, evaluated the equilibrium position of the system under consideration, and discussed the stability range of the system described by a homogeneous differential equation. The method of Lyapunov functions was used to assess the stability of dynamic systems. This allows stability analysis and determining critical points of failure of electrical equipment. (Results and discussion) It was determined that the stability conditions found out using Lyapunov functions depend both on the selected function V(x) and on the equations of the system in state variables. Practical examples of the integration of the Lyapunov function method into automated agricultural production systems, which demonstrate an increase in the stability of electrical equipment, were given. (Conclusions) The possibility and method of using Lyapunov functions for estimating the parameters of an electrical control system have been shown, which makes it possible to successfully solve the problem under consideration, and to propose this approach for solving similar applied problems related to estimating the parameters of ordinary differential equations. It was revealed that the proposed method has shown its efficiency and can be used to automate control facilities in agricultural production.

Modeling personalized heart rate response to exercise and environmental factors with wearables data

Heart rate (HR) response to workout intensity reflects fitness and cardiorespiratory health. Physiological models have been developed to describe such heart rate dynamics and characterize cardiorespiratory fitness. However, these models have been limited to small studies in controlled lab environments and are challenging to apply to noisy—but ubiquitous—data from wearables. We propose a hybrid approach that combines a physiological model with flexible neural network components to learn a personalized, multidimensional representation of fitness. The physiological model describes the evolution of heart rate during exercise using ordinary differential equations (ODEs). ODE parameters are dynamically derived via a neural network connecting personalized representations to external environmental factors, from area topography to weather and instantaneous workout intensity. Our approach efficiently fits the hybrid model to a large set of 270,707 workouts collected from wearables of 7465 users from the Apple Heart and Movement Study. The resulting model produces fitness representations that accurately predict full HR response to exercise intensity in future workouts, with a per-workout median error of 6.1 BPM [4.4–8.8 IQR]. We further demonstrate that the learned representations correlate with traditional metrics of cardiorespiratory fitness, such as VO2 max (explained variance 0.81 ± 0.003). Lastly, we illustrate how our model is naturally interpretable and explicitly describes the effects of environmental factors such as temperature and humidity on heart rate, e.g., high temperatures can increase heart rate by 10%. Combining physiological ODEs with flexible neural networks can yield interpretable, robust, and expressive models for health applications.

Evaluating subgradients for convex relaxations of dynamic process models by adapting current tools

Global dynamic optimization problems are often represented as nonlinear optimization problems with embedded parametric ordinary differential equations. Deterministic methods for global optimization typically employ subgradients of convex relaxations to construct crucial lower bounds. Our recent work shows that subgradients in dynamic optimization problems may be obtained by adapting standard forward or adjoint sensitivity approaches; the adjoint approach ought to be computationally favorable except for small problems. However, established adjoint implementations are incompatible with established libraries for subgradient evaluation. At FOCAPO/CPC 2023, we outlined an automated proof-of-concept implementation of adjoint subgradient evaluation in C++, adapting the convexification package MC++, the ODE solver CVODES, and our own differentiation and code generation tools. This article details how these tools were nontrivially adapted, to ultimately implement adjoint ODE sensitivities embedded with either the forward or reverse modes of subgradient automatic differentiation (AD). Numerical examples are presented for illustration.

PCODE: Estimating Parameters of ODE Models

The ordinary differential equation (ODE) models are prominent to characterize the mechanism of dynamical systems with various applications in biology, engineering, and many other areas. While the form of ODE models is often proposed based on the understanding or assumption of the dynamical systems, the values of ODE model parameters are often unknown. Hence, it is of great interest to estimate the ODE parameters once the observations of dynamic systems become available. The parameter cascade method initially proposed by [@parcascade] is shown to provide an accurate estimation of ODE parameters from the noisy observations at a low computational cost. This method is further promoted with the implementation in the R package CollocInfer by [@CollocInfer]. However, one bottleneck in using CollocInfer to implement the parameter cascade method is the tedious derivations and coding of the Jacobian and Hessian matrices required by the objective functions for doing estimation. We develop an R package pCODE to implement the parameter cascade method, which has the advantage that the users are not required to provide any Jacobian or Hessian matrices. Functions in the pCODE package accommodate users for estimating ODE parameters along with their variances and tuning the smoothing parameters. The package is demonstrated and assessed with four simulation examples with various settings. We show that pCODE offers a derivative-free procedure to estimate any ODE models where its functions are easy to understand and apply. Furthermore, the package has an online Shiny app at <https://pcode.shinyapps.io/pcode/>.

Bayesian Multi-level Mixed-effects Model for Influenza Dynamics

Abstract Influenza A viruses (IAV) are the only influenza viruses known to cause flu pandemics. Understanding the evolution of different sub-types of IAV on their natural hosts is important for preventing and controlling the virus. We propose a mechanism-based Bayesian multi-level mixed-effects model for characterising influenza viral dynamics, described by a set of ordinary differential equations (ODE). Both strain-specific and subject-specific random effects are included for the ODE parameters. Our models can characterise the common features in the population while taking into account the variations among individuals. The random effects selection is conducted at strain level through re-parameterising the covariance parameters of the corresponding random effect distribution. Our method does not need to solve ODE directly. We demonstrate that the posterior computation can proceed via a simple and efficient Markov chain Monte Carlo algorithm. The methods are illustrated using simulated data and a real data from a study relating virus load estimates from influenza infections in ducks.

Identifiability analysis of linear ordinary differential equation systems with a single trajectory

Ordinary differential equations (ODEs) are widely used to model dynamical behavior of systems. It is important to perform identifiability analysis prior to estimating unknown parameters in ODEs (a.k.a. inverse problem), because if a system is unidentifiable, the estimation procedure may fail or produce erroneous and misleading results. Although several qualitative identifiability measures have been proposed, much less effort has been given to developing quantitative (continuous) scores that are robust to uncertainties in the data, especially for those cases in which the data are presented as a single trajectory beginning with one initial value. In this paper, we first derived a closed-form representation of linear ODE systems that are not identifiable based on a single trajectory. This representation helps researchers design practical systems and choose the right prior structural information in practice. Next, we proposed several quantitative scores for identifiability analysis in practice. In simulation studies, the proposed measures outperformed the main competing method significantly, especially when noise was presented in the data. We also discussed the asymptotic properties of practical identifiability for high-dimensional ODE systems and conclude that, without additional prior information, many random ODE systems are practically unidentifiable when the dimension approaches infinity.

Individualizing deep dynamic models for psychological resilience data

Deep learning approaches can uncover complex patterns in data. In particular, variational autoencoders achieve this by a non-linear mapping of data into a low-dimensional latent space. Motivated by an application to psychological resilience in the Mainz Resilience Project, which features intermittent longitudinal measurements of stressors and mental health, we propose an approach for individualized, dynamic modeling in this latent space. Specifically, we utilize ordinary differential equations (ODEs) and develop a novel technique for obtaining person-specific ODE parameters even in settings with a rather small number of individuals and observations, incomplete data, and a differing number of observations per individual. This technique allows us to subsequently investigate individual reactions to stimuli, such as the mental health impact of stressors. A potentially large number of baseline characteristics can then be linked to this individual response by regularized regression, e.g., for identifying resilience factors. Thus, our new method provides a way of connecting different kinds of complex longitudinal and baseline measures via individualized, dynamic models. The promising results obtained in the exemplary resilience application indicate that our proposal for dynamic deep learning might also be more generally useful for other application domains.

Dynamical modeling for non-Gaussian data with high-dimensional sparse ordinary differential equations

Ordinary differential equations (ODE) have been widely used for modeling dynamical complex systems. For high-dimensional ODE models where the number of differential equations is large, it remains challenging to estimate the ODE parameters and to identify the sparse structure of the ODE models. Most existing methods exploit the least-square based approach and are only applicable to Gaussian observations. However, as discrete data are ubiquitous in applications, it is of practical importance to develop dynamic modeling for non-Gaussian observations. New methods and algorithms are developed for both parameter estimation and sparse structure identification in high-dimensional linear ODE systems. First, the high-dimensional generalized profiling method is proposed as a likelihood-based approach with ODE fidelity and sparsity-inducing regularization, along with efficient computation based on parameter cascading. Second, two versions of the two-step collocation methods are extended to the non-Gaussian set-up by incorporating the iteratively reweighted least squares technique. Simulations show that the profiling procedure has excellent performance in latent process and derivative fitting and ODE parameter estimation, while the two-step collocation approach excels in identifying the sparse structure of the ODE system. The usefulness of the proposed methods is also demonstrated by analyzing three real datasets from Google trends, stock market sectors, and yeast cell cycle studies.

Dynamic calibration of differential equations using machine learning, with application to turbulence models

We present a methodology for calibration of parametric ordinary and partial differential equation models, using off-the-shelf software for back-propagation in Neural Networks (NN). As a prototypical example, we consider calibration of a Reynolds-averaged Navier-Stokes (RANS) turbulence closure model, against ground truth data from direct numerical simulations (DNS) of two different turbulent flows. Numerical time integration is represented as a custom NN, where only the RANS model parameters are trainable. A loss function is defined to quantify the mismatch between the NN prediction and the ground truth over a predefined, finite time integration window. This loss function is then minimized using a gradient descent method utilizing the back-propagation algorithm. This dynamic approach to training is to be contrasted with a static approach, wherein a least square regression estimate for parameters is obtained in the limit of an infinitesimal time integration window. In a first test of static and dynamic approaches against ground truth data generated by the model, the former proves to be significantly faster and more accurate than the latter at recovering the parameters. When both calibration approaches are tested against DNS data, for which it is known that the model cannot achieve a perfect fit, the static approach yields a good prediction only for short times, while the dynamic approach results in physical and stable predictions over the entire integration window. After optimization of the dynamic approach for time step, spatial resolution, stability, and physics-based constraints, we obtain a 50% improvement of outcomes over those obtained from the existing, manually calibrated set of parameters, demonstrating the merits of this systematic and automated procedure.

Chaotic Behaviors and Coexisting Attractors in a New Nonlinear Dissipative Parametric Chemical Oscillator

In this study, complex dynamics of Briggs–Rauscher reaction system is investigated analytically and numerically. First, the Briggs–Rauscher reaction system is reduced into a new nonlinear parametric oscillator. The Melnikov method is used to derive the condition of the appearance of horseshoe chaos in the cases ω = Ω and ω ≠ Ω. The performed numerical simulations confirm the obtained analytical predictions. Second, the prediction of coexisting attractors is investigated by solving numerically the new nonlinear parametric ordinary differential equation via the fourth‐order Runge–Kutta algorithm. As results, it is found that the new nonlinear chemical system displays various coexisting behaviors of symmetric and asymmetric attractors. In addition, the system presents a rich variety of bifurcations phenomena such as symmetry breaking, symmetry restoring, period doubling, reverse period doubling, period‐m bubbles, reverse period‐m bubbles, intermittency, and antimonotonicity. On the contrary, emerging chaotic band attractors and period‐1, period‐3, period‐9, and period‐m bubbles routes to chaos occur in this system.

Generation of multicellular spatiotemporal models of population dynamics from ordinary differential equations, with applications in viral infection

BackgroundThe biophysics of an organism span multiple scales from subcellular to organismal and include processes characterized by spatial properties, such as the diffusion of molecules, cell migration, and flow of intravenous fluids. Mathematical biology seeks to explain biophysical processes in mathematical terms at, and across, all relevant spatial and temporal scales, through the generation of representative models. While non-spatial, ordinary differential equation (ODE) models are often used and readily calibrated to experimental data, they do not explicitly represent the spatial and stochastic features of a biological system, limiting their insights and applications. However, spatial models describing biological systems with spatial information are mathematically complex and computationally expensive, which limits the ability to calibrate and deploy them and highlights the need for simpler methods able to model the spatial features of biological systems.ResultsIn this work, we develop a formal method for deriving cell-based, spatial, multicellular models from ODE models of population dynamics in biological systems, and vice versa. We provide examples of generating spatiotemporal, multicellular models from ODE models of viral infection and immune response. In these models, the determinants of agreement of spatial and non-spatial models are the degree of spatial heterogeneity in viral production and rates of extracellular viral diffusion and decay. We show how ODE model parameters can implicitly represent spatial parameters, and cell-based spatial models can generate uncertain predictions through sensitivity to stochastic cellular events, which is not a feature of ODE models. Using our method, we can test ODE models in a multicellular, spatial context and translate information to and from non-spatial and spatial models, which help to employ spatiotemporal multicellular models using calibrated ODE model parameters. We additionally investigate objects and processes implicitly represented by ODE model terms and parameters and improve the reproducibility of spatial, stochastic models.ConclusionWe developed and demonstrate a method for generating spatiotemporal, multicellular models from non-spatial population dynamics models of multicellular systems. We envision employing our method to generate new ODE model terms from spatiotemporal and multicellular models, recast popular ODE models on a cellular basis, and generate better models for critical applications where spatial and stochastic features affect outcomes.

Learning Dynamic Systems Using Gaussian Process Regression with Analytic Ordinary Differential Equations as Prior Information

Recently the data-driven learning of dynamic systems has become a promising approach because no physical knowledge is needed. Pure machine learning approaches such as Gaussian process regression (GPR) learns a dynamic model from data, with all physical knowledge about the system discarded. This goes from one extreme, namely methods based on optimizing parametric physical models derived from physical laws, to the other. GPR has high flexibility and is able to model any dynamics as long as they are locally smooth, but can not generalize well to unexplored areas with little or no training data. The analytic physical model derived under assumptions is an abstract approximation of the true system, but has global generalization ability. Hence the optimal learning strategy is to combine GPR with the analytic physical model. This paper proposes a method to learn dynamic systems using GPR with analytic ordinary differential equations (ODEs) as prior information. The one-time-step integration of analytic ODEs is used as the mean function of the Gaussian process prior. The total parameters to be trained include physical parameters of analytic ODEs and parameters of GPR. A novel method is proposed to simultaneously learn all parameters, which is realized by the fully Bayesian GPR and more promising to learn an optimal model. The standard Gaussian process regression, the ODE method and the existing method in the literature are chosen as baselines to verify the benefit of the proposed method. The predictive performance is evaluated by both one-time-step prediction and long-term prediction. By simulation of the cart-pole system, it is demonstrated that the proposed method has better predictive performances.

Physics-Informed Neural Networks and Functional Interpolation for Data-Driven Parameters Discovery of Epidemiological Compartmental Models

In this work, we apply a novel and accurate Physics-Informed Neural Network Theory of Functional Connections (PINN-TFC) based framework, called Extreme Theory of Functional Connections (X-TFC), for data-physics-driven parameters’ discovery of problems modeled via Ordinary Differential Equations (ODEs). The proposed method merges the standard PINNs with a functional interpolation technique named Theory of Functional Connections (TFC). In particular, this work focuses on the capability of X-TFC in solving inverse problems to estimate the parameters governing the epidemiological compartmental models via a deterministic approach. The epidemiological compartmental models treated in this work are Susceptible-Infectious-Recovered (SIR), Susceptible-Exposed-Infectious-Recovered (SEIR), and Susceptible-Exposed-Infectious-Recovered-Susceptible (SEIRS). The results show the low computational times, the high accuracy, and effectiveness of the X-TFC method in performing data-driven parameters’ discovery systems modeled via parametric ODEs using unperturbed and perturbed data.