Integration Of Ordinary Differential Equations Research Articles

A hybrid CPU‐GPU paradigm to accelerate reactive CFD simulations

AbstractThe solution of reactive computational fluid dynamics (CFD) simulations is accelerated by the implementation of a hybrid central processing unit/graphics processing units (CPU/GPU) Finite Volume solver based on the operator‐splitting strategy, where the chemistry integration is treated independently of the flow solution. The integration of ordinary differential equations (ODEs) describing the finite‐rate chemical kinetics is solved by an adaptive multi‐block explicit solver on GPUs, while the load of the fluid solution is distributed on a multicore CPU algorithm. The resulting speed‐up for reactive CFD simulations is up to 10; the performance gain increases with the size of the mechanism. The proposed implementation is general and can be applied to any CFD problem where the governing equations for the fluid transport are coupled with an ODE system. Code validation is performed against reference solutions on a selection of test cases involving reacting flows.

Designing stable neural networks using convex analysis and ODEs

Motivated by classical work on the numerical integration of ordinary differential equations we present a ResNet-styled neural network architecture that encodes non-expansive (1-Lipschitz) operators, as long as the spectral norms of the weights are appropriately constrained. This is to be contrasted with the ordinary ResNet architecture which, even if the spectral norms of the weights are constrained, has a Lipschitz constant that, in the worst case, grows exponentially with the depth of the network. Further analysis of the proposed architecture shows that the spectral norms of the weights can be further constrained to ensure that the network is an averaged operator, making it a natural candidate for a learned denoiser in Plug-and-Play algorithms. Using a novel adaptive way of enforcing the spectral norm constraints, we show that, even with these constraints, it is possible to train performant networks. The proposed architecture is applied to the problem of adversarially robust image classification, to image denoising, and finally to the inverse problem of deblurring.

Integrable geodesic flows and metrisable second-order ordinary differential equations

It is well known that the system of ordinary differential equations (ODEs) describing geodesic flows of some Riemannian metrics on 2-surfaces admits a projection on a special class of second-order ODEs. In this paper we study in detail this special class of ODEs. We classify all such autonomous ODEs possessing autonomous first integrals that are fractional-quadratic in the first derivative. We construct all the families of integrable geodesic flows related to the ODEs under study. These families are parameterized by two arbitrary functions and contain metrics with superintegrable geodesic flows. We also study the inverse problem and find novel families of second-order ODEs that are related to integrable geodesic flows. We explicitly present first integrals of such ODEs. We find the general structure of metrisable second-order ODEs related to conformal metrics.

Modelling disease mitigation at mass gatherings: A case study of COVID-19 at the 2022 FIFA World Cup.

The 2022 FIFA World Cup was the first major multi-continental sporting Mass Gathering Event (MGE) of the post COVID-19 era to allow foreign spectators. Such large-scale MGEs can potentially lead to outbreaks of infectious disease and contribute to the global dissemination of such pathogens. Here we adapt previous work and create a generalisable model framework for assessing the use of disease control strategies at such events, in terms of reducing infections and hospitalisations. This framework utilises a combination of meta-populations based on clusters of people and their vaccination status, Ordinary Differential Equation integration between fixed time events, and Latin Hypercube sampling. We use the FIFA 2022 World Cup as a case study for this framework (modelling each match as independent 7 day MGEs). Pre-travel screenings of visitors were found to have little effect in reducing COVID-19 infections and hospitalisations. With pre-match screenings of spectators and match staff being more effective. Rapid Antigen (RA) screenings 0.5 days before match day performed similarly to RT-PCR screenings 1.5 days before match day. Combinations of pre-travel and pre-match testing led to improvements. However, a policy of ensuring that all visitors had a COVID-19 vaccination (second or booster dose) within a few months before departure proved to be much more efficacious. The State of Qatar abandoned all COVID-19 related travel testing and vaccination requirements over the period of the World Cup. Our work suggests that the State of Qatar may have been correct in abandoning the pre-travel testing of visitors. However, there was a spike in COVID-19 cases and hospitalisations within Qatar over the World Cup. Given our findings and the spike in cases, we suggest a policy requiring visitors to have had a recent COVID-19 vaccination should have been in place to reduce cases and hospitalisations.

An optimized deep learning approach for blood-brain barrier permeability prediction with ODE integration

Blood-brain barrier (BBB) permeability prediction plays a pivotal role in drug discovery for neurological disorders which is essential for the development of central nervous system (CNS) drugs. Compounds having high permeability must be found to synthesize brain drugs for the treatment of different brain disorders such as Alzheimer's, Parkinson's, and brain tumors. Developing an accurate mathematical computational model to determine the exact brain permeability value for a possible drug is essential for advancing and improving the success rate of the development of drugs for neurological treatment. We developed a combined method capable of forecasting the logBB value of the compound in question for lightBBB and B3DB datasets by using the Convolutional neural network (CNNs), Recurrent Neural Networks (RNNs) and Ordinary Differential Equations (ODEs). The results demonstrate the overall assessment of the prediction ability of BBB permeability. CNN-LSTM model tests the performance for the prediction of logBB values in two different datasets LightBBB and B3DB. In both datasets, CNN-LSTM achieves lower RMSE values as compared to other models, showing that it has better predictive performance. Specifically, in the LightBBB dataset, the CNN-LSTM model achieves an RMSE of 0.59, while in the B3DB datasets, it achieves an even lower RMSE of 0.85. CNN-LSTM model shows highly effective in accurately predicting logBB values in both datasets.

ORRN: An ODE-Based Recursive Registration Network for Deformable Respiratory Motion Estimation With Lung 4DCT Images.

Deformable Image Registration (DIR) plays a significant role in quantifying deformation in medical data. Recent Deep Learning methods have shown promising accuracy and speedup for registering a pair of medical images. However, in 4D (3D + time) medical data, organ motion, such as respiratory motion and heart beating, can not be effectively modeled by pair-wise methods as they were optimized for image pairs but did not consider the organ motion patterns necessary when considering 4D data. This article presents ORRN, an Ordinary Differential Equations (ODE)-based recursive image registration network. Our network learns to estimate time-varying voxel velocities for an ODE that models deformation in 4D image data. It adopts a recursive registration strategy to progressively estimate a deformation field through ODE integration of voxel velocities. We evaluate the proposed method on two publicly available lung 4DCT datasets, DIRLab and CREATIS, for two tasks: 1) registering all images to the extreme inhale image for 3D+t deformation tracking and 2) registering extreme exhale to inhale phase images. Our method outperforms other learning-based methods in both tasks, producing the smallest Target Registration Error of 1.24 mm and 1.26 mm, respectively. Additionally, it produces less than 0.001% unrealistic image folding, and the computation speed is less than 1 s for each CT volume. ORRN demonstrates promising registration accuracy, deformation plausibility, and computation efficiency on group-wise and pair-wise registration tasks. It has significant implications in enabling fast and accurate respiratory motion estimation for treatment planning in radiation therapy or robot motion planning in thoracic needle insertion.

PC-Reg: A pyramidal prediction–correction approach for large deformation image registration

Deformable image registration plays an important role in medical image analysis. Deep neural networks such as VoxelMorph and TransMorph are fast, but limited to small deformations and face challenges in the presence of large deformations. To tackle large deformations in medical image registration, we propose PC-Reg, a pyramidal Prediction and Correction method for deformable registration, which treats multi-scale registration akin to solving an ordinary differential equation (ODE) across scales. Starting with a zero-initialized deformation at the coarse level, PC-Reg follows the predictor–corrector regime and progressively predicts a residual flow and a correction flow to update the deformation vector field through different scales. The prediction in each scale can be regarded as a single step of ODE integration. PC-Reg can be easily extended to diffeomorphic registration and is able to alleviate the multiscale accumulated upsampling and diffeomorphic integration error. Further, to transfer details from full resolution to low scale, we introduce a distillation loss, where the output is used as the target label for intermediate outputs. Experiments on inter-patient deformable registration show that the proposed method significantly improves registration not only for large but also for small deformations.

Exploiting the Abstract Calculus Pattern for the Integration of Ordinary Differential Equations for Dynamics Systems: An Object-Oriented Programming Approach in Modern Fortran

This manuscript relates to the exploiting of the abstract calculus pattern (ACP) for the (numerical) solution of ordinary differential equation (ODEs) systems, which are ubiquitous mathematical formulations of many physical (dynamical) phenomena. We present FOODIE, a software suite aimed to numerically solve ODE problems by means of a clear, concise, and efficient abstract interface. The results presented prove manifold findings, in particular that our ACP approach enables ease of code development, clearness and robustness, maximization of code re-usability, and conciseness comparable with computer algebra system (CAS) programming (interpreted) but with the computational performance of compiled programming. The proposed programming model is also proven to be agnostic with respect to the parallel paradigm of the computational architecture: the results show that FOODIE applications have good speedup with both shared (OpenMP) and distributed (MPI, CAF) memory architectures. The present paper is the first announcement of the FOODIE project: the current implementation is extensively discussed, and its capabilities are proved by means of tests and examples.

Modelling the discretization error of initial value problems using the Wishart distribution

This paper presents a new discretization error quantification method for the numerical integration of ordinary differential equations. The error is modelled by using the Wishart distribution, which enables us to capture the correlation between variables. Error quantification is achieved by solving an optimization problem under the order constraints for the covariance matrices. An algorithm for the optimization problem is also established in a slightly broader context.

Implementation of the Adams method for solving ordinary differential equations in the Sage computer algebra system

This work is devoted to the implementation and testing of the Adams method for solving ordinary differential equations in the Sage computer algebra system. The Sage computer algebra system has, to some extent, trivial means for numerical integration of ordinary differential equations, but at the same time, it is worth noting that this environment is convenient and practical for conducting computer experiments related to symbolic numerical calculations in it. The article presents the FDM package developed on the basis of the RUDN, which contains the developments of recent years, performed by M. D. Malykh and his students, for numerical integration of differential equations. In this package, attention is paid to the visualization of the calculation results, including the construction of various kinds of auxiliary diagrams, such as Richardson diagrams, as well as graphs of dependence, for example, the value of a function or step from a moment in time. The implementation of the Adams method will be considered from this package. In this article, this implementation of the Adams method will be tested on various examples of input data, and the method will also be compared with the Jacobi system. Exact and approximate values will be found and compared, and an estimate for the error will be obtained.

Discretization of inherent ODEs and the geometric integration of DAEs with symmetries

Discretization methods for differential-algebraic equations (DAEs) are considered that are based on the integration of an associated inherent ordinary differential equation (ODE). This allows to make use of any discretization scheme suitable for the numerical integration of ODEs. For DAEs with symmetries it is shown that the inherent ODE can be constructed in such a way that it inherits the symmetry properties of the given DAE and geometric properties of its flow. This in particular allows the use of geometric integration schemes with a numerical flow that has analogous geometric properties.

Approximate Integration of Ordinary Differential Equations Using the Chebyshev Series with Precision Control

Approximate Integration of Ordinary Differential Equations Using the Chebyshev Series with Precision Control

Wave propagation and soliton solutions of the Allen–Cahn model

The Allen–Cahn equation (ACE), which has applications in solid-state physics, imaging, plasma physics, material science and other fields, is one of the most important models of the modern era for describing the dynamics of oil pollution, reaction-diffusion mechanisms, and the mechanics of crystalline solids. By using the [Formula: see text]-expansion method (GEM) and the Bernoulli sub-ODE schemes, some new traveling wave solutions for the governing model are created in this study (BSODE). The reduced integrable ordinary differential equation is produced using the traveling wave hypothesis. To better understand their behavior, the 3D, contour, and 2D graphs are displayed for a number of fascinating exact solutions. Additionally, we use numerical simulation to confirm the stability of the derived analytical solutions. It results the propagation of temporal soliton for long time of simulation. These results will be used to explain physical phenomenon in crystalline solids and others fields.

Modeling of bioprocesses via MINLP-based symbolic regression of S-system formalisms

Mathematical modeling helps guide experiments more effectively, support process monitoring and control tasks, stabilize product quality, increase consumer safety, or ease specific decision-making tasks for subject matter experts. However, constructing accurate process models can be challenging, especially with bioprocesses, due to complex metabolic mechanisms and data scarcity. This work proposes a method for building models combining a mass balance backbone with a canonical kinetic representation, i.e., the S-system formalism. The model structure and parameters that best describe the studied system are automatically identified by solving a mixed-integer nonlinear programming (MINLP) problem. Following an incremental approach, the integration of ordinary differential equations is avoided. Numerical examples show that our method performs similarly to models based on artificial neural networks, outperforming them in some cases while providing an analytical, closed-form model. Such expressions can be more easily interpreted and optimized in existing algebraic modeling systems.

On the Method of Differential Invariants for Solving Higher Order Ordinary Differential Equations

There are many routines developed for solving ordinary differential Equations (ODEs) of different types. In the case of an nth-order ODE that admits an r-parameter Lie group (3≤r≤n), there is a powerful method of Lie symmetry analysis by which the ODE is reduced to an (n−r)th-order ODE plus r quadratures provided that the Lie algebra formed by the infinitesimal generators of the group is solvable. It would seem this method is not widely appreciated and/or used as it is not mentioned in many related articles centred around integration of higher order ODEs. In the interest of mainstreaming the method, we describe the method in detail and provide four illustrative examples. We use the case of a third-order ODE that admits a three-dimensional solvable Lie algebra to present the gist of the integration algorithm.

Personalized Algorithm Generation: A Case Study in Learning ODE Integrators

We study the learning of numerical algorithms for scientific computing, which combines mathematically driven, handcrafted design of general algorithm structure with a data-driven adaptation to specific classes of tasks. This represents a departure from the classical approaches in numerical analysis, which typically do not feature such learning-based adaptations. As a case study, we develop a machine learning approach that automatically learns effective solvers for initial value problems in the form of ordinary differential equations (ODEs), based on the Runge--Kutta (RK) integrator architecture. We show that we can learn high-order integrators for targeted families of differential equations without the need for computing integrator coefficients by hand. Moreover, we demonstrate that in certain cases we can obtain superior performance to classical RK methods. This can be attributed to certain properties of the ODE families being identified and exploited by the approach. Overall, this work demonstrates an effective learning-based approach to the design of algorithms for the numerical solution of differential equations. This can be readily extended to other numerical tasks.

Exploring Inductive Linearization for simulation and estimation with an application to the Michaelis–Menten model

Nonlinear ordinary differential equations (ODEs) are common in pharmacokinetic–pharmacodynamic systems. Although their exact solutions cannot generally be determined via algebraic methods, their rapid and accurate solutions are desirable. Thus, numerical methods have a critical role. Inductive Linearization was proposed as a method to solve systems of nonlinear ODEs. It is an iterative approach that converts a nonlinear ODE into a linear time-varying (LTV) ODE, for which a range of standard integration techniques can then be used to solve (e.g., eigenvalue decomposition [EVD]). This study explores the properties of Inductive Linearization when coupled with EVD for integration of the LTV ODE and illustrates how the efficiency of the method can be improved. Improvements were based on three approaches, (1) incorporation of a convergence criterion for the iterative linearization process (for simulation and estimation), (2) creating more efficient step sizes for EVD (for simulation and estimation), and (3) updating the initial conditions of the Inductive Linearization (for estimation). The performance of these improvements were evaluated using single subject stochastic simulation-estimation with an application to a simple pharmacokinetic model with Michaelis–Menten elimination. The reference comparison was a standard non-stiff Runge–Kutta method with variable step size (ode45, MATLAB). Each of the approaches improved the speed of the Inductive Linearization technique without diminishing accuracy which, in this simple case, was faster than ode45 with comparable accuracy in the parameter estimates. The methods described here can easily be implemented in standard software programme such as R or MATLAB. Further work is needed to explore this technique for estimation in a population approach setting.

An Algorithm for Numerical Integration of ODE with Sampled Unknown Functional Factors

The problem of having ordinary differential equations (ODE) whose coefficients are unknown functions is frequent in several fields. Sometimes, it is possible to obtain samples of the values of these functions in different instants or spatial points. The present paper presents a methodology for the numeric solving of these ODE. There are approximations to the problem for specific cases of equations, especially in the case where the parameters correspond to constants. Other studies focus on the case in which the functions under consideration are linear or meet a certain condition. There are two main advantages of the proposed algorithm. First, it does not impose any condition over the data or the subsequent function from where these sample data are derived. Additionally, the methodology used in the functions modeling can control the possibility of overfitting in the function modeling. This is a crucial point in order to limit the influence of model biases in the numerical solution of the ordinary differential equation under study.

A numerical verification method to specify homoclinic orbits as application of local Lyapunov functions

AbstractWe propose a verification method for specification of homoclinic orbits as application of our previous work for constructing local Lyapunov functions by verified numerics. Our goal is to specify parameters appeared in the given systems of ordinary differential equations (ODEs) which admit homoclinic orbits to equilibria. Here we restrict ourselves to cases that each equilibrium is independent of parameters. The feature of our methods consists of Lyapunov functions, integration of ODEs by verified numerics, and Brouwer’s coincidence theorem on continuous mappings. Several techniques for constructing continuous mappings from a domain of parameter vectors to a region of the phase space are shown. We present numerical examples for problems in 3 and 4-dimensional cases.

Implementation of a practical Markov chain Monte Carlo sampling algorithm in PyBioNetFit.

Bayesian inference in biological modeling commonly relies on Markov chain Monte Carlo (MCMC) sampling of a multidimensional and non-Gaussian posterior distribution that is not analytically tractable. Here, we present the implementation of a practical MCMC method in the open-source software package PyBioNetFit (PyBNF), which is designed to support parameterization of mathematical models for biological systems. The new MCMC method, am, incorporates an adaptive move proposal distribution. For warm starts, sampling can be initiated at a specified location in parameter space and with a multivariate Gaussian proposal distribution defined initially by a specified covariance matrix. Multiple chains can be generated in parallel using a computer cluster. We demonstrate that am can be used to successfully solve real-world Bayesian inference problems, including forecasting of new Coronavirus Disease 2019 case detection with Bayesian quantification of forecast uncertainty. PyBNF version 1.1.9, the first stable release with am, is available at PyPI and can be installed using the pip package-management system on platforms that have a working installation of Python 3. PyBNF relies on libRoadRunner and BioNetGen for simulations (e.g. numerical integration of ordinary differential equations defined in SBML or BNGL files) and Dask.Distributed for task scheduling on Linux computer clusters. The Python source code can be freely downloaded/cloned from GitHub and used and modified under terms of the BSD-3 license (https://github.com/lanl/pybnf). Online documentation covering installation/usage is available (https://pybnf.readthedocs.io/en/latest/). A tutorial video is available on YouTube (https://www.youtube.com/watch?v=2aRqpqFOiS4&t=63s). Supplementary data are available at Bioinformatics online.