Sparse Identification Of Nonlinear Dynamics Research Articles

SBMLToolkit.jl: a Julia package for importing SBML into the SciML ecosystem.

Julia is a general purpose programming language that was designed for simplifying and accelerating numerical analysis and computational science. In particular the Scientific Machine Learning (SciML) ecosystem of Julia packages includes frameworks for high-performance symbolic-numeric computations. It allows users to automatically enhance high-level descriptions of their models with symbolic preprocessing and automatic sparsification and parallelization of computations. This enables performant solution of differential equations, efficient parameter estimation and methodologies for automated model discovery with neural differential equations and sparse identification of nonlinear dynamics. To give the systems biology community easy access to SciML, we developed SBMLToolkit.jl. SBMLToolkit.jl imports dynamic SBML models into the SciML ecosystem to accelerate model simulation and fitting of kinetic parameters. By providing computational systems biologists with easy access to the open-source Julia ecosystevnm, we hope to catalyze the development of further Julia tools in this domain and the growth of the Julia bioscience community. SBMLToolkit.jl is freely available under the MIT license. The source code is available at https://github.com/SciML/SBMLToolkit.jl.

TC-SINDy: Improving physics-based deterministic tropical cyclone track and intensity model via data-driven sparse identification of Nonlinear Dynamics

Physics-based deterministic tropical cyclone (TC) simulation models play an important role in TC risk analysis, for their unique capability to represent physical principles and incorporate climate change impacts. However, establishing effective deterministic models can be challenging. In this paper, we present a simple yet powerful framework that distills physics-based TC models from historical TC data. By treating TC evolutions as dynamical systems, we adaptively discover the TC track and intensity governing equations from historical TC and environmental data via the Sparse Identification of Nonlinear Dynamics (SINDy) approach. Our proposed TC-SINDy approach is able to identify parsimonious, interpretable yet effective physics-based TC models. Compared to alternative physics-based deterministic models, our models match historical records much better in terms of the intensity evolution, prevailing tracks, and key parameter statistics across the coastlines of the Western North Pacific (WNP) basin and the North Atlantic (NA) basin. Our proposed models were finally coupled with a widely used parametric wind field, and the obtained probabilistic distribution of the surface wind also matched the historical histograms. Overall, the proposed method presents a general data-driven framework for discovering physics-based TC models, enabling reliable effective TC simulations while reserving the capacity to accommodate the changing climate.

Atmospheric Chemistry Surrogate Modeling With Sparse Identification of Nonlinear Dynamics

AbstractModeling atmospheric chemistry is computationally expensive and limits the widespread use of chemical transport models. This computational cost arises from solving high‐dimensional systems of stiff differential equations. Previous work has demonstrated the promise of machine learning (ML) to accelerate air quality model simulations but has suffered from numerical instability during long‐term simulations. This may be because previous ML‐based efforts have relied on explicit Euler time integration—which is known to be unstable for stiff systems—and have used neural networks which are prone to overfitting. We hypothesize that parsimonious models combined with modern numerical integration techniques can overcome this limitation. Using a small‐scale mechanism to explore the potential of these methods, we have created a machine‐learned surrogate by (a) reducing dimensionality using singular value decomposition to create an interpretably‐compressed low‐dimensional latent space and (b) using Sparse Identification of Nonlinear Dynamics (SINDy) to create a differential‐equation‐based representation of the underlying dynamics in the compressed latent space with reduced stiffness. The root mean square error of ML model prediction for ozone concentration over 9 days is 37.8% of the root mean square concentration across all simulations in our testing data set. The surrogate model is 10× faster with 12× fewer integration timesteps compared to reference model and is numerically stable in all tested simulations. Overall, we find that SINDy can be used to create fast, stable, and accurate surrogates of a simple photochemical mechanism. In future work, we will explore the application of this method to more detailed mechanisms and their use in large‐scale simulations.

Learning Difference Equations With Structured Grammatical Evolution for Postprandial Glycaemia Prediction.

People with diabetes must carefully monitor their blood glucose levels, especially after eating. Blood glucose management requires a proper combination of food intake and insulin boluses. Glucose prediction is vital to avoid dangerous post-meal complications in treating individuals with diabetes. Although traditional methods, and also artificial neural networks, have shown high accuracy rates, sometimes they are not suitable for developing personalised treatments by physicians due to their lack of interpretability. This study proposes a novel glucose prediction method emphasising interpretability: Interpretable Sparse Identification by Grammatical Evolution. Combined with a previous clustering stage, our approach provides finite difference equations to predict postprandial glucose levels up to two hours after meals. We divide the dataset into four-hour segments and perform clustering based on blood glucose values for the two-hour window before the meal. Prediction models are trained for each cluster for the two-hour windows after meals, allowing predictions in 15-minute steps, yielding up to eight predictions at different time horizons. Prediction safety was evaluated based on Parkes Error Grid regions. Our technique produces safe predictions through explainable expressions, avoiding zones D (0.2% average) and E (0%) and reducing predictions on zone C (6.2%). In addition, our proposal has slightly better accuracy than other techniques, including sparse identification of non-linear dynamics and artificial neural networks. The results demonstrate that our proposal provides interpretable solutions without sacrificing prediction accuracy, offering a promising approach to glucose prediction in diabetes management that balances accuracy, interpretability, and computational efficiency.

Identification of High-Order Nonlinear Coupled Systems Using a Data-Driven Approach

Most works related to the identification of mathematical nonlinear systems suggest that such approaches can always be directly applied to any nonlinear system. This misconception is greatly discouraging when the obtained results are not expected. Thus, the current work hypothesizes that the more information one has about the mathematical structure of the model, the most precise the identification result. Therefore, a variant of the Sparse Identification of Nonlinear Dynamics (SINDY) approach is presented to obtain the full mathematical nonlinear model of a high-order system with coupled dynamics, namely, a commercial quadcopter. Furthermore, due to its high sensitivity to inputs, a control system is devised using the identified model to stabilize the quadcopter. This illustrates the effectiveness of the proposed identification method.

Weak-form latent space dynamics identification

Recent work in data-driven modeling has demonstrated that a weak formulation of model equations enhances the noise robustness of a wide range of computational methods. In this paper, we demonstrate the power of the weak form to enhance the LaSDI (Latent Space Dynamics Identification) algorithm, a recently developed data-driven reduced order modeling technique.We introduce a weak form-based version WLaSDI (Weak-form Latent Space Dynamics Identification). WLaSDI first compresses data, then projects onto the test functions and learns the local latent space models. Notably, WLaSDI demonstrates significantly enhanced robustness to noise. With WLaSDI, the local latent space is obtained using weak-form equation learning techniques. Compared to the standard sparse identification of nonlinear dynamics (SINDy) used in LaSDI, the variance reduction of the weak form guarantees a robust and precise latent space recovery, hence allowing for a fast, robust, and accurate simulation. We demonstrate the efficacy of WLaSDI vs. LaSDI on several common benchmark examples including viscid and inviscid Burgers’, radial advection, and heat conduction. For instance, in the case of 1D inviscid Burgers’ simulations with the addition of up to 100% Gaussian white noise, the relative error remains consistently below 6% for WLaSDI, while it can exceed 10,000% for LaSDI. Similarly, for radial advection simulations, the relative errors stay below 16% for WLaSDI, in stark contrast to the potential errors of up to 10,000% with LaSDI. Moreover, speedups of several orders of magnitude can be obtained with WLaSDI. For example applying WLaSDI to 1D Burgers’ yields a 140X speedup compared to the corresponding full order model.Python code to reproduce the results in this work is available at (https://github.com/MathBioCU/PyWSINDy_ODE) and (https://github.com/MathBioCU/PyWLaSDI).

Physics-informed and data-driven discovery of governing equations for complex phenomena in heterogeneous media.

Rapid evolution of sensor technology, advances in instrumentation, and progress in devising data-acquisition software and hardware are providing vast amounts of data for various complex phenomena that occur in heterogeneous media, ranging from those in atmospheric environment, to large-scale porous formations, and biological systems. The tremendous increase in the speed of scientific computing has also made it possible to emulate diverse multiscale and multiphysics phenomena that contain elements of stochasticity or heterogeneity, and to generate large volumes of numerical data for them. Thus, given a heterogeneous system with annealed or quenched disorder in which a complex phenomenon occurs, how should one analyze and model the system and phenomenon, explain the data, and make predictions for length and time scales much larger than those over which the data were collected? We divide such systems into three distinct classes. (i) Those for which the governing equations for the physical phenomena of interest, as well as data, are known, but solving the equations over large length scales and long times is very difficult. (ii) Those for which data are available, but the governing equations are only partially known, in the sense that they either contain various coefficients that must be evaluated based on the data, or that the number of degrees of freedom of the system is so large that deriving the complete equations is very difficult, if not impossible, as a result of which one must develop the governing equations with reduced dimensionality. (iii) In the third class are systems for which large amounts of data are available, but the governing equations for the phenomena of interest are not known. Several classes of physics-informed and data-driven approaches for analyzing and modeling of the three classes of systems have been emerging, which are based on machine learning, symbolic regression, the Koopman operator, the Mori-Zwanzig projection operator formulation, sparse identification of nonlinear dynamics, data assimilation combined with a neural network, and stochastic optimization and analysis. This perspective describes such methods and the latest developments in this highly important and rapidly expanding area and discusses possible future directions.

Identification of control equations using low-dimensional flow representations of pitching airfoil

This study investigates the application of data-driven modeling techniques for understanding the complex dynamics of pitching airfoils at low Reynolds numbers and high angles of attack. Linear and nonlinear dimensionality reduction methods, namely principal component analysis (PCA) and isometric mapping (ISOMAP), are employed to obtain low-dimensional representations of the flow field. Subsequently, sparse identification of nonlinear dynamics (SINDy) is utilized to model the governing equations. The key findings are as follows: PCA primarily captures linear information, with the first two to three dimensions maintaining relatively low reconstruction errors. In contrast, ISOMAP excels in capturing nonlinear features, exhibiting noticeably smaller reconstruction errors. The main information is concentrated in the two-dimensional plane constructed by PCA1 and PCA2 (or ISOMAP1 and ISOMAP2). Differences in trajectory planes formed by combinations of other axes reflect flow field disparities. ISOMAP provides a nonlinear low-dimensional representation, advantageous for capturing nonlinear relationships between flow field characteristics and governing equations. The combination of ISOMAP and SINDy yields virtually no errors in identifying governing equations. Conversely, PCA and SINDy result in significantly different linear trajectories, leading to higher reconstruction errors. The identified governing equations using ISOMAP and SINDy remain consistent across different datasets, demonstrating the method's stability and robustness in accurately characterizing flow field properties under similar conditions.

Analyzing Russia–Ukraine War Patterns Based on Lanchester Model Using SINDy Algorithm

In this paper, we present an effective method for analyzing patterns in the Russia–Ukraine war based on the Lanchester model. Due to the limited availability of information on combat powers of engaging forces, we utilize the loss of armored equipment as the primary data source. To capture the intricate dynamics of modern warfare, we partition the combat loss data into disjoint subsets by examining their geometric properties. Separate systems of ordinary differential equations for these subsets are then identified using the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm under a generalized formulation of the historical Lanchester model. We provide simulations of our method to demonstrate its effectiveness and performance in analyzing contemporary warfare dynamics.

Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants

Recent progress in autoencoder-based sparse identification of nonlinear dynamics (SINDy) under ℓ 1 constraints allows joint discoveries of governing equations and latent coordinate systems from spatio-temporal data, including simulated video frames. However, it is challenging for ℓ 1 -based sparse inference to perform correct identification for real data due to the noisy measurements and often limited sample sizes. To address the data-driven discovery of physics in the low-data and high-noise regimes, we propose Bayesian SINDy autoencoders, which incorporate a hierarchical Bayesian Spike-and-slab Gaussian Lasso prior. Bayesian SINDy autoencoder enables the joint discovery of governing equations and coordinate systems with uncertainty estimate. To resolve the challenging computational tractability of the Bayesian hierarchical setting, we adapt an adaptive empirical Bayesian method with Stochastic Gradient Langevin Dynamics (SGLD) which gives a computationally tractable way of Bayesian posterior sampling within our framework. Bayesian SINDy autoencoder achieves better physics discovery with lower data and fewer training epochs, along with valid uncertainty quantification suggested by the experimental studies. The Bayesian SINDy autoencoder can be applied to real video data, withaccurate physics discovery which correctly identifies the governing equation and provides a close estimate for standard physics constants like gravity g , for example, in videos of a pendulum.

Data‐driven identification of crystallization kinetics

AbstractA novel data‐driven methodology is presented for developing mathematical models for crystallization processes. The data‐driven approach is based on the sparse identification of nonlinear dynamics (SINDy) method, which iterates between a partial least‐squares fit and a sparsity‐promoting step leading to the discovery of sparse interpretable models. The performance of the SINDy methodology is characterized for the identification of crystallization kinetics in a mixed tank operated in a continuous mode, the isothermal crystallization of lysozyme in a batch stirred tank and cooling crystallization of paracetamol. The SINDy method is robust against noise. Good agreement is obtained between the data‐driven model and the data obtained from crystallization experiments. The presented data‐driven approach can be attractive for modeling industrial crystallization processes where process analytical technology tools are available for the measurement of process variables but functional forms of kinetic expressions are unknown.

Enhancing sparse regression modeling of hysteresis with optimized PIO algorithm in piezo actuator

Abstract The extensive application of piezo actuators is attributed to their high responsiveness and ability to achieve nanoscale steps. However, the accuracy and stability of motion are seriously affected by hysteresis caused by nonlinear characteristics. In this paper, a pigeon-inspired optimization (PIO) algorithm based on dynamic opposite learning (DOL) is proposed to address the issue of nonlinear modeling accuracy in piezo actuators by integrating the sparse identification of nonlinear dynamics (SINDy) method. Firstly, the DOL strategy is employed to introduce reverse pigeon flock into the PIO algorithm, thereby enhancing population diversity and optimization performance. Secondly, through combining the DOLPIO algorithm with the SINDy algorithm, sparse optimization for the penalty process in SINDy algorithm is conducted and the sparse coefficient is optimized based on modeling accuracy. Thirdly, the DOLPIO algorithm is utilized again to optimize the framework of optimized sparse penalty model in order to improve overall modeling accuracy. Finally, experiments are conducted on an established platform to validate the effectiveness of this algorithm.

Kernel functions embed into the autoencoder to identify the sparse models of nonlinear dynamics

Numerous researches have shown that there are three main challenges in data-driven model identification methods: high-dimensional measurements, system complexity and unknown underlying dynamical properties. For most nonlinear dynamics, the feature space defined by the coefficients of their control equations is sparse. Therefore, sparse regression methods are used to learn the sparse coefficients of the control equations of nonlinear dynamics. However, this method strongly depends on the appropriate selection of the sparse basis vectors. In this essay, the autoencoder is combined with the sparse regression method to simultaneously identify the sparse coordinate and a parsimonious, interpretable and generalizable model of the specified system. It also integrates kernel functions to map the intractable measurements in the hidden space of the autoencoder into a linearly distinguishable kernel space, which kernelizes the candidate function library of the sparse identification of nonlinear dynamics (SINDy) model as the sparse dictionaries. Therefore, the flexible representation of neural networks, the simplicity of sparse regression methods and the implicit non-linear representation of kernel functions are consolidated in this article. To inspect the reliability of the proposed model in this paper, a set of nonlinear dynamics formulated by ordinary differential equations (ODEs), second-order trigonometric functions and partial differential equations (PDEs) are utilized as test cases. And the comparisons between the proposed model and other model identification methods illustrate that the performance of the former is the best.

Data-Driven Modeling of the Nonlinear Dynamics of Passive Lower-Limb Prosthetic Systems

Abstract Modeling the nonlinear dynamics of prosthetic feet is an important tool for linking prosthesis mechanical properties to end-user outcomes. There has been a renewed interest in data-driven modeling of dynamical systems, with the development of the Extended Dynamic Mode Decomposition with control (eDMDc) and the Sparse Identification of Nonlinear Dynamics with Control (SINDYc). These algorithms do not require prior information about the system, including mechanical configuration, and are data-driven. The aim of this study was to assess the feasibility and accuracy of applying these data-driven algorithms to model prosthesis nonlinear load response dynamics. Different combinations of a dynamic response foot, a hydraulic ankle unit, and three shock-absorbing pylons of varying resistance were tested loaded and unloaded at three orientations reflecting critical positions during the stance phase of walking. We tested two different data-driven algorithms, the eDMDc, with two different kernels, and the SINDYc, which regresses the coefficients for a nonlinear ordinary differential equation. Each algorithm was able to model the nonlinear prosthesis dynamics, but the SINDYc outperformed the eDMDc methods with a root mean square error across orientations < 1.50 mm and a maximum error in peak displacement of 1.28 mm or 4% relative error. From the estimated SINDYc governing equation of the system dynamics, we were able to simulate different mechanical behavior by systematically varying parameter values, which offers a novel foundation for designing, controlling, and classifying prosthetic systems ultimately aimed at improving prosthesis user outcomes.

Quantifying changes in individual-specific template-based representations of center-of-mass dynamics during walking with ankle exoskeletons using Hybrid-SINDy

Ankle exoskeletons alter whole-body walking mechanics, energetics, and stability by altering center-of-mass (CoM) motion. Controlling the dynamics governing CoM motion is, therefore, critical for maintaining efficient and stable gait. However, how CoM dynamics change with ankle exoskeletons is unknown, and how to optimally model individual-specific CoM dynamics, especially in individuals with neurological injuries, remains a challenge. Here, we evaluated individual-specific changes in CoM dynamics in unimpaired adults and one individual with post-stroke hemiparesis while walking in shoes-only and with zero-stiffness and high-stiffness passive ankle exoskeletons. To identify optimal sets of physically interpretable mechanisms describing CoM dynamics, termed template signatures, we leveraged hybrid sparse identification of nonlinear dynamics (Hybrid-SINDy), an equation-free data-driven method for inferring sparse hybrid dynamics from a library of candidate functional forms. In unimpaired adults, Hybrid-SINDy automatically identified spring-loaded inverted pendulum-like template signatures, which did not change with exoskeletons (p > 0.16), except for small changes in leg resting length (p < 0.001). Conversely, post-stroke paretic-leg rotary stiffness mechanisms increased by 37–50% with zero-stiffness exoskeletons. While unimpaired CoM dynamics appear robust to passive ankle exoskeletons, how neurological injuries alter exoskeleton impacts on CoM dynamics merits further investigation. Our findings support Hybrid-SINDy’s potential to discover mechanisms describing individual-specific CoM dynamics with assistive devices.

Data driven models of the Madden-Julian Oscillation: understanding its evolution and ENSO modulation

Understanding the physical mechanisms of the Madden-Julian Oscillation (MJO) and its evolution is a major concern within the climate community. Its main importance relies on its ability to act as a source of predictability within the intra-seasonal time-scale in tropical and extratropical regions, therefore filling the gap between weather and climate forecasts. However, most atmospheric general circulation models fail to correctly represent MJO’s evolution, and their prediction skills are still far from MJO’s theoretical predictability. In this work we infer low dimensional models of the MJO from data by applying a recently developed machine learning technique, the Sparse Identification of Non-linear Dynamics (SINDy). We use the daily-mean outgoing longwave radiation MJO index (OMI) as input data to infer bi-dimensional climatological models of the MJO, and analyse the inferred models during El Niño and La Niña years. This approach allows us to diagnose the MJO’s behaviour in OMI’s phase space. Our results show that MJO can be most frequently represented by a harmonic oscillator, which represents the MJO’s eastward propagation and characteristic period. Upon this basic oscillatory behaviour, we find that small non-linear corrections play a fundamental role in representing MJO’s non-uniform speed of propagation, explaining its acceleration over the Pacific Ocean region. Particularly, we find that MJO’s evolution is most frequently non-linear [linear] during El Niño [La Niña] years. Overall, our work shows that SINDy can robustly model MJO’s evolution as a linear oscillator with small non-linear corrections, contributing to understand the MJO’s dynamics and dependency on El Niño-Southern Oscillation.

An explainable AI model for power plant NOx emission control

In recent years, developing Artificial Intelligence (AI) models for complex system has become a popular research area. There have been several successful AI models for predicting the Selective Non-Catalytic Reduction (SNCR) system in power plants and large boilers. However, all these models are in essence black box models and lack of explainability, which are not able to give new knowledge. In this study, a novel explainable AI (XAI) model that combines the polynomial kernel method with Sparse Identification of Nonlinear Dynamics (SINDy) model is proposed to find the governing equation of SNCR system based on 5-year operation data from a power plant. This proposed model identifies the system's governing equation in a simple polynomial format with polynomial order of 1 and only 1 independent variable among original 68 input variables. In addition, the explainable AI model achieves a considerable accuracy with less than 21 % deviation from base-line models of partial least squares model and artificial neural network model.

PSINDy: Probabilistic sparse identification of nonlinear dynamics for temporal process modeling and fault detection

BackgroundMultivariate statistical process monitoring (MSPM) methods have been widely studied and applied for dynamic process monitoring. However, limitations exist in the extant literature, as numerous extant MSPM approaches fail to concurrently account for the governing dynamics and inherent non-linearity of systems. MethodsTo handle the above-mentioned issue, a novel probabilistic dynamic process monitoring algorithm named probabilistic sparse identification of nonlinear dynamics (PSINDy) is proposed. In this algorithm, the Expectation–Maximization (EM) method is employed to estimate the parameters. In the E-step, the particle filtering technique is adopted to calculate corresponding latent expectations whose posterior distributions are not Gaussian. Furthermore, Hotelling's T2 and two novel monitoring statistics, termed Mahalanobis-based predictive error (MPE) and dynamic predictive error (DPE), are designed and utilized for fault detection. Significant FindingsThe effectiveness of the proposed method is validated through the Tennessee Eastman (TE) chemical process and the three-phase flow facility (TPFF).

Reduced-order model and attractor identification for large eddy simulation of squirrel cage fan

A large eddy simulation (LES) of a squirrel cage fan (SCF) provides a precise representation of turbulent flows with different degrees of complexity. This study comprehensively analyzes the coherent structures of turbulent flows in an SCF using an LES, proper orthogonal decomposition (POD), dynamic mode decomposition (DMD), and multi-resolution dynamic mode decomposition (mrDMD). An intelligent reduced-order model is established by integrating hierarchical deep learning and the sparse identification of nonlinear dynamics. The result shows that the evolution of the global DMD modes is attenuated due to the spatial distribution variations of localized high-frequency mrDMD modes, along with the fragmented and non-steady development of modal patterns. Unlike POD, DMD quantifies the quality of the impeller inlet environment and captures the antisymmetric low-dimensional flows associated with the shedding of rotating vortex structures. The interaction strength between stationary and dynamic rotating areas is accurately represented by attractors characterized by petal-like structures. The trajectory of the attractors faithfully maps the antisymmetric structural attributes, quasi-periodic behavior, and gradual attenuation characteristics exhibited by DMD modes. The number of petal-like systems and their temporal oscillations are in good agreement with the number of fan blades and their rotational cycles. This study provides new insight into fan engineering to advance flow control strategies and improve the understanding of the underlying flow mechanisms.

Sparse Identification of Chemical Processes: A Feasibility Study

In the ongoing search for efficient techniques to identify and validate dynamic models in various fields of science and engineering, the sparse identification of nonlinear dynamics (SINDy) algorithm has emerged as a promising tool to carry out this task. In this study, SINDy is applied to identify two chemical processes: pH neutralization and temperature control in a bioreactor. To accomplish this, firstly, the dynamic behavior of each system was modeled, and data were collected for both processes. Then, the studied algorithm was employed to identify the dynamic models of each process. Then, the identified models by SINDy were validated by comparing them with first-principles models. Several tests were conducted to assess the capability of SINDy-based models in predicting the dynamic behavior of the pH and temperature processes. The results revealed a satisfactory agreement between the models obtained by SINDy and the first-principles models. Hence, we show the SINDy algorithm could be reliable tool for identifying dynamic models for chemical processes.