Nonlinear Dynamic Identification Research Articles

Machine learning to identify environmental drivers of phytoplankton blooms in the Southern Baltic Sea

Phytoplankton blooms exhibit varying patterns in timing and number of peaks within ecosystems. These differences in blooming patterns are partly explained by phytoplankton:nutrient interactions and external factors such as temperature, salinity and light availability. Understanding these interactions and drivers is essential for effective bloom management and modelling as driving factors potentially differ or are shared across ecosystems on regional scales. Here, we used a 22-year data set (19 years training and 3 years validation data) containing chlorophyll, nutrients (dissolved and total), and external drivers (temperature, salinity, light) of the southern Baltic Sea coast, a European brackish shelf sea, which constituted six different phytoplankton blooming patterns. We employed generalized additive mixed models to characterize similar blooming patterns and trained an artificial neural network within the Universal Differential Equation framework to learn a differential equation representation of these pattern. Applying Sparse Identification of Nonlinear Dynamics uncovered algebraic relationships in phytoplankton:nutrient:external driver interactions. Nutrients availability was driving factor for blooms in enclosed coastal waters; nutrients and temperature in more open regions. We found evidence of hydrodynamical export of phytoplankton, natural mortality or external grazing not explicitly measured in the data. This data-driven workflow allows new insight into driver-differences in region specific blooming dynamics.

Uncovering Dynamical Equations of Stochastic Decision Models Using Data-Driven SINDy Algorithm.

Decision formation in perceptual decision making involves sensory evidence accumulation instantiated by the temporal integration of an internal decision variable toward some decision criterion or threshold, as described by sequential sampling theoretical models. The decision variable can be represented in the form of experimentally observable neural activities. Hence, elucidating the appropriate theoretical model becomes crucial to understanding the mechanisms underlying perceptual decision formation. Existing computational methods are limited to either fitting of choice behavioral data or linear model estimation from neural activity data. In this work, we made use of sparse identification of nonlinear dynamics (SINDy), a data-driven approach, to elucidate the deterministic linear and nonlinear components of often-used stochastic decision models within reaction time task paradigms. Based on the simulated decision variable activities of the models and assuming the noise coefficient term is known beforehand, SINDy, enhanced with approaches using multiple trials, could readily estimate the deterministic terms in the dynamical equations, choice accuracy, and decision time of the models across a range of signal-to-noise ratio values. In particular, SINDy performed the best using the more memory-intensive multitrial approach while trial averaging of parameters performed more moderately. The single-trial approach, although expectedly not performing as well, may be useful for real-time modeling. Taken together, our work offers alternative approaches for SINDy to uncover the dynamics in perceptual decision making and, more generally, for first-passage time problems.

Discovering the climate dependent disease transmission mechanism through learning-explaining framework.

There are evidence showing that meteorological factors, such as temperature and humidity, have critical effects on transmission of some infectious diseases, while quantifying the influence is challenging. In this study we develop a learning-explaining framework to discover the particular dependence of transmission mechanisms on meteorological factors based on multiple source data. The incidence rate based on the epidemic data and epidemic model is theoretically identified, and meanwhile the practical discovery of particular formula is feasible through deep neural networks (DNN), symbolic regression (SR) and sparse identification of nonlinear dynamics (SINDy). In particular, we initially learn the incidence rate in an SIRS model based on epidemic data, then use mechanism discovery methods to explore the possible explicit forms of the incidence rate, and consequently explore the possible relationship between transmission rate and meteorological factors. We finally use information criteria and a definition of evaluation score to make model selection, and hence suggest the optimal explicit formula. We illustrate the idea by derive the incidence rate and transmission rate of respiratory infectious diseases based on the case data on influenza-like illness (ILI) in Xi'an, Shaanxi Province of China and meteorological data from 1st January 2010 to 10th November 2016. The finding reveals that the influence of meteorological factors on transmission exhibits very strong nonlinearity, and modeling the effect should be of great care.

Data-driven model discovery and model selection for noisy biological systems.

Biological systems exhibit complex dynamics that differential equations can often adeptly represent. Ordinary differential equation models are widespread; until recently their construction has required extensive prior knowledge of the system. Machine learning methods offer alternative means of model construction: differential equation models can be learnt from data via model discovery using sparse identification of nonlinear dynamics (SINDy). However, SINDy struggles with realistic levels of biological noise and is limited in its ability to incorporate prior knowledge of the system. We propose a data-driven framework for model discovery and model selection using hybrid dynamical systems: partial models containing missing terms. Neural networks are used to approximate the unknown dynamics of a system, enabling the denoising of the data while simultaneously learning the latent dynamics. Simulations from the fitted neural network are then used to infer models using sparse regression. We show, via model selection, that model discovery using hybrid dynamical systems outperforms alternative approaches. We find it possible to infer models correctly up to high levels of biological noise of different types. We demonstrate the potential to learn models from sparse, noisy data in application to a canonical cell state transition using data derived from single-cell transcriptomics. Overall, this approach provides a practical framework for model discovery in biology in cases where data are noisy and sparse, of particular utility when the underlying biological mechanisms are partially but incompletely known.

Physics‐Informed Active Learning With Simultaneous Weak‐Form Latent Space Dynamics Identification

ABSTRACTThe parametric greedy latent space dynamics identification (gLaSDI) framework has demonstrated promising potential for accurate and efficient modeling of high‐dimensional nonlinear physical systems. However, it remains challenging to handle noisy data. To enhance robustness against noise, we incorporate the weak‐form estimation of nonlinear dynamics (WENDy) into gLaSDI. In the proposed weak‐form gLaSDI (WgLaSDI) framework, an autoencoder and WENDy are trained simultaneously to discover intrinsic nonlinear latent‐space dynamics of high‐dimensional data. Compared with the standard sparse identification of nonlinear dynamics (SINDy) employed in gLaSDI, WENDy enables variance reduction and robust latent space discovery, therefore leading to more accurate and efficient reduced‐order modeling. Furthermore, the greedy physics‐informed active learning in WgLaSDI enables adaptive sampling of optimal training data on the fly for enhanced modeling accuracy. The effectiveness of the proposed framework is demonstrated by modeling various nonlinear dynamical problems, including viscous and inviscid Burgers' equations, time‐dependent radial advection, and the Vlasov equation for plasma physics. With data that contains 5%–10 Gaussian white noise, WgLaSDI outperforms gLaSDI by orders of magnitude, achieving 1%–7 relative errors. Compared with the high‐fidelity models, WgLaSDI achieves 121 to 1779 speed‐up.

Rheo-SINDy: Finding a constitutive model from rheological data for complex fluids using sparse identification for nonlinear dynamics

Rheology plays a pivotal role in understanding the flow behavior of fluids by discovering governing equations that relate deformation and stress, known as constitutive equations. Despite the importance of these equations, current methods for deriving them lack a systematic methodology, often relying on sense of physics and incurring substantial costs. To overcome this problem, we propose a novel method named Rheo-SINDy, which employs the sparse identification of nonlinear dynamics (SINDy) algorithm for discovering constitutive models from rheological data. Rheo-SINDy was applied to five distinct scenarios, four with well-established constitutive equations, and one without predefined equations. Our results demonstrate that Rheo-SINDy successfully identified accurate models for the known constitutive equations and derived physically plausible approximate models for the scenario without established equations. Notably, the identified approximate models can accurately reproduce nonlinear shear rheological properties, especially at steady state, including shear thinning. These findings validate the availability of Rheo-SINDy in handling data complexities and underscore its potential for advancing the development of data-driven approaches in rheology. Nevertheless, further refinement of learning strategies is essential for enhancing robustness to fully account for the complexities of real-world rheological data.

Uncertainty Quantification in Reduced‐Order Gas‐Phase Atmospheric Chemistry Modeling Using Ensemble SINDy

AbstractUncertainty quantification during atmospheric chemistry modeling is computationally expensive as it typically requires a large number of simulations using complex models. As such, large‐scale modeling is typically performed with simplified chemical mechanisms for computational tractability. Here, we describe a probabilistic surrogate modeling method using principal components analysis (PCA) and Ensemble Sparse Identification of Nonlinear Dynamics (E‐SINDy) to both automatically simplify a gas‐phase chemistry mechanism and to quantify the uncertainty introduced when doing so. We demonstrate the application of this method on a small photochemical box model for ozone formation. With 100 ensemble members, the calibration ‐squared value is 0.86 among the three latent species on average and 0.98 for ozone, demonstrating that predicted model uncertainty aligns well with actual model error. In addition to uncertainty quantification, this probabilistic method also improves accuracy as compared to an equivalent deterministic version, by 64% for the ensemble prediction mean or 45% for deterministic prediction by the best‐performing single ensemble member. Overall, the ozone testing root mean square error (RMSE) is 20% of its root mean square (RMS) concentration. Although our probabilistic ensemble simulation ends up being slower than the reference model it emulates, we hypothesize that use of a more complex reference model in future work will result in additional opportunities for acceleration. Versions of this approach applied to full‐scale chemical mechanisms may result in improved uncertainty quantification in models of atmospheric composition, leading to enhanced atmospheric understanding and improved support for air quality control and regulation.

Dynamical systems-inspired machine learning methods for drought prediction

Drought is a naturally occurring phenomenon that affects millions of people and results in billions of dollars in damages each year, with impacts expected to worsen due to climate change. At the same time, definitions of drought are nebulous, and extant quantitative drought indicators suffer from short prediction horizons. One such indicator is the Normalized Vegetation Difference Index (NDVI), which measures photosynthetic activity, making it a strong proxy for vegetation density. Recent studies have identified chaotic attractors in satellite-derived NDVI time-series, suggesting a dynamical systems framework may be helpful for time-series prediction of NDVI. In this study, we compare the performance of a mechanistic model and two physics-informed machine learning methods (Sparse Identification of Nonlinear Dynamics [SINDy] and reservoir computing) on the prediction of NDVI time-series data in drought-prone regions of Kenya. We find that SINDy, a sparse polynomial modelling architecture, narrowly outperforms the other two methods with the use of precipitation data, while also retaining some of the interpretability of the mechanistic model. We also find that none of the methods perform as well in the regions in which the chaotic NDVI attractors were originally identified. We conclude by proposing more sophisticated extensions to the methods presented here, both with and without the availability of precipitation data, that draw on the existing dynamical systems and machine learning literature to enable better quantitative predictions of key drought indicators.

Identification of moment equations via data-driven approaches in nonlinear Schrödinger models

IntroductionThe moment quantities associated with the nonlinear Schrödinger equation offer important insights into the evolution dynamics of such dispersive wave partial differential equation (PDE) models. The effective dynamics of the moment quantities are amenable to both analytical and numerical treatments.MethodsIn this paper, we present a data-driven approach associated with the “Sparse Identification of Nonlinear Dynamics” (SINDy) to capture the evolution behaviors of such moment quantities numerically.Results and DiscussionOur method is applied first to some well-known closed systems of ordinary differential equations (ODEs) which describe the evolution dynamics of relevant moment quantities. Our examples are, progressively, of increasing complexity and our findings explore different choices within the SINDy library. We also consider the potential discovery of coordinate transformations that lead to moment system closure. Finally, we extend considerations to settings where a closed analytical form of the moment dynamics is not available.

An Online Data-Driven Method for Accurate Detection of Thermal Updrafts Using SINDy

Utilizing thermal updrafts shows potential for enabling long-endurance cruising of fixed-wing unmanned aerial vehicles without energy consumption. This article presents a novel online method based on sparse identification of nonlinear dynamics (SINDy) approach to achievement identification of thermal sources in the atmosphere. Initially, the algorithm is incorporated into the upper-level planning system, interacting with the lower-level controller. Then, experiments are conducted through software-in-the-loop simulations (SITL) to validate the implementation of the proposed algorithm. It is found that direct observation of thermal sources through measurements using SINDy is unfeasible during straight and circular flight modes. Nevertheless, simulation analysis of the proposed approach indicates that under unobservable conditions, a portion of the parameters can still be identified. By comparing results obtained using the particle filter algorithm, this method is shown to accurately estimate the parameters with negligible errors under observability conditions. The novelty of this approach lies in its significant improvement of the localization accuracy of the thermal source, without the need for parameter adjustments in the algorithm. Finally, the proposed methods are integrated into commonly used hardware platforms, and their online feasibility is verified through hardware-in-the-loop simulations.

Data-driven system identification and model predictive control of pneumatic conveying using nonlinear dynamics analysis for optimised energy consumption

Pneumatic conveying systems provide secure transportation of particulate material and a dust-free environment. These systems face high energy consumption, material degradation, and pipeline blockages. This research presents an innovative solution by integrating nonlinear dynamics analysis of electrostatic sensor data, including chaos and recurrence quantification analysis, sparse identification of nonlinear dynamics with control (SINDYc) and model predictive control (MPC). The Lyapunov exponent, approximate entropy and recurrence rate of electrostatic sensor data reveal the chaotic nature of gas-solid flows. MPC framework was tailored for real-time optimisation of a pneumatic conveying system. SINDYc system models were developed using data collected from an open-loop control pneumatic conveying process to conduct MPC simulations and select an appropriate model for real-time control. This research illustrates the potential of integrating nonlinear dynamics analysis, SINDYc integrated and MPC for enhanced system performance, showcased a significant reduction in energy consumption without compromising the system's efficiency or reliability.

Data-driven discovery of a model equation describing self-oscillations of direct current discharge

Data-driven techniques developed in recent years for the discovery of equations describing complex physical phenomena open unique opportunities for plasma physics. These methods allow getting insights into the processes difficult for analytical description. Since gas discharges can be represented as complex electrical circuits consisting of impedances and capacitances, it looks natural to use the data-driven techniques to study their complex dynamics. In the present paper, the sparse identification of nonlinear dynamics (SINDy) method is applied to analyze the self-oscillations of direct current discharge in argon. It is obtained that the third order polynomials describe best the oscillations of the discharge voltage and current. They allow an accurate capturing of the oscillations amplitudes as well as the harmonics of these oscillations. To understand the physical meaning of each term, an analytical model is presented which describes the discharge self-oscillations.

Nonlinear parametric models of viscoelastic fluid flows.

Reduced-order models (ROMs) have been widely adopted in fluid mechanics, particularly in the context of Newtonian fluid flows. These models offer the ability to predict complex dynamics, such as instabilities and oscillations, at a considerably reduced computational cost. In contrast, the reduced-order modelling of non-Newtonian viscoelastic fluid flows remains relatively unexplored. This work leverages the sparse identification of nonlinear dynamics (SINDy) algorithm to develop interpretable ROMs for viscoelastic flows. In particular, we explore a benchmark oscillatory viscoelastic flow on the four-roll mill geometry using the classical Oldroyd-B fluid. This flow exemplifies many canonical challenges associated with non-Newtonian flows, including transitions, asymmetries, instabilities, and bifurcations arising from the interplay of viscous and elastic forces, all of which require expensive computations in order to resolve the fast timescales and long transients characteristic of such flows. First, we demonstrate the effectiveness of our data-driven surrogate model to predict the transient evolution and accurately reconstruct the spatial flow field for fixed flow parameters. We then develop a fully parametric, nonlinear model capable of capturing the dynamic variations as a function of the Weissenberg number. While the training data are predominantly concentrated on a limit cycle regime for moderate , we show that the parametrized model can be used to extrapolate, accurately predicting the dominant dynamics in the case of high Weissenberg numbers. The proposed methodology represents an initial step in applying machine learning and reduced-order modelling techniques to viscoelastic flows.

Web-платформа для організації хмарних обчислень у нейрофізіологічних дослідженнях на основі даних айтрекінгу

The purpose of the work is to develop the architecture and web version of the software complex based on the proposed new concept of cloud computing organization, which allows to expand the diagnostic capabilities of the tools of model-oriented information technology for assessing the neurophysiological state of a person using methods of nonlinear dynamic identification of the oculomotor system based on eye tracking data. The concept of cloud computing is proposed, which is based on the combination of PaaS and SaaS services as part of the developed software complex, due to which the cross-platform nature of cloud computing is ensured, the productivity and efficiency of scientific research increases. The developed architecture allows you to easily expand the functionality of the software complex and adapt it to different application conditions. The key feature of the complex is its undemanding hardware on the client side thanks to the use of cloud computing and its modular structure, which allows for easy scaling of the complex, as well as the isolation of the script code execution process in the cloud computing environment to increase the level of security when interpreting the script code on the server . Compared to other similar services, the complex has several advantages: it provides effective work in research and educational applications, supports Python and JavaScript programming languages, and also allows the use of software-implemented identification methods through specially developed GUI interfaces. In addition, the complex offers social opportunities and a high level of abstraction, which allows to optimize the research process.

AI-Lorenz: A physics-data-driven framework for Black-Box and Gray-Box identification of chaotic systems with symbolic regression

Discovering mathematical models that characterize the observed behavior of dynamical systems remains a major challenge, especially for systems in a chaotic regime, due to their sensitive dependence on initial conditions and the complex non-linear interactions within the system. The challenge is even greater when the physics underlying such systems is not yet understood, and scientific inquiry must solely rely on empirical data. Despite advancements such as the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm, which has shown success in identifying chaotic systems with reasonable accuracy, challenges remain in dealing with noise, sparse data, and the need for models that generalize well across different chaotic systems. Driven by the need to fill this gap, we develop a framework named AI-Lorenz that learns mathematical expressions modeling complex dynamical behaviors by identifying differential equations from noisy and sparse observable data. We train a physics-informed neural network (PINN) with the eXtreme Theory of Functional Connections (X-TFC) algorithm by using data and known physics (when available) to learn the dynamics of a system, its rate of change in time, and missing model terms, which are used as input for a symbolic regression algorithm (PySR) to autonomously distill the explicit mathematical terms. This, in turn, enables us to predict the future evolution of the dynamical behavior. The performance of this framework is validated by recovering the right-hand sides and unknown terms of certain complex, chaotic systems, such as the well-known Lorenz system, a six-dimensional hyperchaotic system, the non-autonomous Sprott chaotic system, and the slow-fast Duffing system, and comparing them with their known analytical expressions and state-of-the-art regression and system identification methods.

Optimal charging of Li-ion batteries using sparse identification of nonlinear dynamics

Optimal charging of Li-ion batteries requires careful management of charge rates, as high rates can lead to accelerated degradation, while low rates significantly extend charging times. Traditional methods for determining charge rates often rely on rule-based approaches, which typically fail to effectively balance battery performance with charging duration. To address this, we introduce a novel optimization approach that directly integrates the dual objectives of minimizing charge time and maximizing battery lifetime into the optimization process. Unlike most existing charge optimization methods that do not directly track battery lifetime and charge time simultaneously, our method employs a data-driven model that facilitates direct and dynamic estimation of both battery lifetime and charge time at each step of the optimization process. Specifically, we utilize the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm to predict battery capacity and voltage dynamics, which informs the calculations of lifetime and charge time required to solve the optimization problem. This approach provides a balanced optimization strategy that enhances the effectiveness of battery’s performance while maintaining the efficiency of the charging process. We applied this method to a novel next-generation NMC811 battery, featuring a cathode comprised of 80% nickel, 10% manganese, and 10% cobalt, and a lithium metal foil anode — a combination not extensively studied previously. Experimental validation demonstrated that when optimized charge rates are applied every 10 cycles in a 100-cycle operation, the method leads to more stable cycling and improved capacity retention of approximately 7.4% over the nominal charge rate, demonstrating the potential of the developed approach.

Optimal operational planning of a bio-fuelled cogeneration plant: Integration of sparse nonlinear dynamics identification and deep reinforcement learning

This paper presents a novel data-driven approach for short-term operational planning of a cogeneration plant. The proposed methodology utilizes sparse identification of nonlinear dynamics (SINDy) to extract a dynamic model of heat generation from operational data. This model is then employed to simulate the plant dynamics during the training of a reinforcement learning (RL) agent, enabling online stochastic optimization of the production plan in real-time. The incorporation of SINDy enhances the accuracy of capturing the plant's nonlinear dynamics and significantly improves the computational speed of plant simulations, enabling efficient RL agent training within a reasonable timeframe. The performance of operational planning with the RL agent is compared to that of dynamic programming, a widely used method in the literature. The evaluation metric encompasses energy efficiency, unmet demands, and wasted heat. The comparison investigates the effectiveness of RL and dynamic programming under various scenarios with different qualities of energy demand forecasts. The RL agent exhibits robustness and notably improves the operational planning performance, particularly when faced with uncertain energy demands in the environment. Furthermore, the findings show that the RL agent, trained on a school building data, could successfully perform planning tasks for a hotel building, indicating the transferability of learned planning knowledge across different cogeneration use cases.

Investigation of the accuracy of integral models of the oculo-motor system

For mathematical modeling of the human oculomotor system (OMS), integral nonlinear models are used, which simultaneously take into account the nonlinear and inertial properties of the research object. Based on the data of experimental studies of the OMS "input-output", transient and diagonal intersections of transient functions of the second and third orders are determined. To obtain experimental data, an innovative eye tracking technology is used, which allows recording eye responses to test visual stimuli. Thus test signals are displayed on the computer monitor at different distances from the starting position in the horizontal direction. The aim of the research is to study the accuracy of OMS identification using eye-tracking data by evaluating the calculation errors of multidimensional transient functions when using methods of nonlinear dynamic identification based on models in the form of Volterra series and polynomials. The object of the study is the process of nonparametric identification of the OMS based on Volterra models in the time domain. The subject of the research is algorithmic and software tools for calculating the dynamic characteristics of OMS based on eye-tracking data, analyzing the accuracy of the obtained models using two identification methods: the approximation method and the least squares method (LSM). The means of nonlinear dynamic identification of the human OMS based on Volterra series and polynomials were developed in the Python programming environment. The accuracy estimates of various OMS (linear, quadratic, and cubic) models were obtained based on data from three responses to test signals of different amplitudes. For the same test signals, the same models in the form of the Volterra series and polynomials were obtained, as these models coincide in the region of convergence of the Volterra series. The analysis of errors in the assessment of the dynamic characteristics of the OMS demonstrated that the model in the form of an integral polynomial of the second degree, which was built using the LSM based on three responses, has an accuracy twice as high as the accuracy of similar models built using the data of two responses. Thus, in further studies of the psychophysiological state of a person based on nonlinear dynamic models of the OMS according to the data of three responses, it is advisable to use a model in the form of a quadratic Volterra polynomial. Figs.: 17. Tabl. 3. Refs.: 10 titles.

Nonlinear system identification via sparse Bayesian regression based on collaborative neurodynamic optimization

Abstract Sparse identification of nonlinear dynamics is a popular approach to system identification. In this approach system identification is reformulated as a sparse regression problem, and the use of a good sparse regression method is crucial. Sparse Bayesian learning based on collaborative neurodynamic optimization is a recent method that consistently produces high-quality solutions. In this article, we extensively assess how this method performs for ordinary differential equation identification. We find that it works very well compared with sparse regression algorithms currently used for this task in terms of the tradeoff between the approximation accuracy and the complexity of the identified system. We also propose a way to substantially reduce the computational complexity of this algorithm compared with its original implementation, thus making it even more practical.

Dynamical system identification, model selection, and model uncertainty quantification by Bayesian inference.

This study presents a Bayesian maximum a posteriori (MAP) framework for dynamical system identification from time-series data. This is shown to be equivalent to a generalized Tikhonov regularization, providing a rational justification for the choice of the residual and regularization terms, respectively, from the negative logarithms of the likelihood and prior distributions. In addition to the estimation of model coefficients, the Bayesian interpretation gives access to the full apparatus for Bayesian inference, including the ranking of models, the quantification of model uncertainties, and the estimation of unknown (nuisance) hyperparameters. Two Bayesian algorithms, joint MAP and variational Bayesian approximation, are compared to the least absolute shrinkage and selection operator (LASSO), ridge regression, and the sparse identification of nonlinear dynamics (SINDy) algorithms for sparse regression by application to several dynamical systems with added Gaussian or Laplace noise. For multivariate Gaussian likelihood and prior distributions, the Bayesian formulation gives Gaussian posterior and evidence distributions, in which the numerator terms can be expressed in terms of the Mahalanobis distance or "Gaussian norm" ||y-y^||M-12=(y-y^)⊤M-1(y-y^), where y is a vector variable, y^ is its estimator, and M is the covariance matrix. The posterior Gaussian norm is shown to provide a robust metric for quantitative model selection for the different systems and noise models examined.