Sparse Identification Of Nonlinear Dynamics Research Articles

In search of a data-driven symbolic multi-fluid ten-moment model closure

The inclusion of kinetic effects into fluid models has been a long standing problem in magnetic reconnection and plasma physics. Generally, the pressure tensor is reduced to a scalar which is an approximation used to aid in the modelling of large scale global systems such as the Earth's magnetosphere. This unfortunately omits important kinetic physics which have been shown to play a crucial role in collisionless regimes. The multi-fluid ten-moment model, however, retains the full symmetric pressure tensor. The ten-moment model is constructed by taking moments of the Vlasov equation up to second order, and includes the scalar density, the vector bulk-flow and the symmetric pressure tensor for a total of ten separate components. Use of the multi-fluid ten-moment model requires a closure which truncates the cascading system of equations. Here we look to leverage data-driven methodologies to seek a closure which may improve the physical fidelity of the ten-moment multi-fluid model in collisionless regimes. Specifically, we use the sparse identification of nonlinear dynamics (SINDy) method for symbolic equation discovery to seek the truncating closure from fully kinetic particle-in-cell simulation data, which inherently retains the relevant kinetic physics. We verify our method by reproducing the ten-moment model from the particle-in-cell (PIC) data and use the method to generate a closure truncating the ten-moment model which is analysed through the nonlinear phase of reconnection.

Construction of a reduced-order model of an electro-osmotic micromixer and discovery of attractors for petal structure

The chaotic state of microfluidic devices such as electroosmotic micromixers has received extensive attention. Its unsteady flow and multi-physics mask low-dimensional structure and potential attractors. Based on the dynamic mode decomposition and the sparse identification of nonlinear dynamics, this study aims to construct a manifold equation with the minimum degree of freedom, reveal the mixing mechanism of micromixers, and discover the evolution of chaotic states. The attenuation degree of freedom was introduced to force the modal coefficients to be pure oscillations. The six, four, and two-dimensional minimum reduced-order models (ROMs) were constructed under different mixing conditions. The nonlinear dynamics evolves on attractors resembling a six-petal structure based on the amplitude-phase method. The attractor periodicity and decay map the evolution of the periodic oscillation and limit cycle of the active modes and are related to the appearance of the low-energy dominant non-axisymmetric modes. These results emphasize the significance of ROM technology in revealing the low-dimensional structure and attractor of the electroosmotic micromixer.

Learning sparse nonlinear dynamics via mixed-integer optimization

Discovering governing equations of complex dynamical systems directly from data is a central problem in scientific machine learning. In recent years, the sparse identification of nonlinear dynamics (SINDy) framework, powered by heuristic sparse regression methods, has become a dominant tool for learning parsimonious models. We propose an exact formulation of the SINDy problem using mixed-integer optimization (MIO-SINDy) to solve the sparsity constrained regression problem to provable optimality in seconds. On a large number of canonical ordinary and partial differential equations, we illustrate the dramatic improvement in our approach in accurate model discovery while being more sample efficient, robust to noise, and flexible in accommodating physical constraints.

Study on Nonlinear Correlation in Modal Coefficients of the Bionic Airfoil

Applying bionic airfoils is essential in enlightening the design of rotating machinery and flow control. Dynamic mode decomposition was used to reveal the low dimensional flow structure of Riblets, Seagull, and Teal bionic airfoils at low Reynolds numbers 1 × 105 and is compared with NACA4412 airfoils. The attack angle of the two-dimensional airfoil is 19°, and the SST k-ω turbulence model and ANSYS fluent were used to obtain the transient flow field data. The sparse identification of nonlinear dynamics reveals the nonlinear correlation between modal coefficients and establishes manifold dynamics. The results show that the bionic airfoil and NACA4412 airfoil have the same type of nonlinear correlation, and the dimension and form of the minimum reduced-order model are consistent. The modal coefficients always appear in the manifold equation in pairs with a phase difference of 90°. The dimension of the manifold equation is two-dimensional, and the absolute value of the coefficient corresponds to the fundamental frequency of airfoil vortex shedding. The reconstructed flow field based on the manifold equation is highly consistent with the numerical simulation flow field, which reveals the accuracy of the manifold equation. The relevant conclusions of this study emphasize the unity of the nonlinear correlation of bionic airfoils.

Neural network and Sparse identification of Nonlinear Dynamics Integrated Algorithm for Digital Twin identification

Digital twins play a critical role in simulating industrial manufacturing systems to increase productivity and reduce time spent on troubleshooting. Owing to the complexity of real-world industrial systems, automatic sparse identification has emerged as an attractive approach to perform digital twin modelling. The sparse identification of nonlinear dynamics (SINDy) is a machine learning algorithm that performs feature engineering by generating a model term library and then solves a sparse regression problem between the objective outputs and the generated features. By solving a linear-in-parameter sparse regression problem, SINDy provides automatic discovery of system governing equations. However, the performance of SINDy-based algorithms may decline dramatically when applied to identify complex nonlinear relationships, such as implicit relationships. The substantial number of input variables for a real industrial process may further complicate the modelling procedure. We therefore propose the neural network and SINDy integrated algorithm to automatically select the critical features from a model term library and utilize the neural network to capture the process nonlinearity that cannot be captured by a linear-in-parameter model. SINDy performs feature generation considering both numerical methods and first-principles knowledge, making the proposed algorithm a hybrid system identification approach. A diesel hydrotreating unit case study with 37 input variables is analyzed in this paper to demonstrate the advantages of the proposed algorithm for nonlinear digital twin identification. By combining the advantages from both SINDy and neural networks, the proposed algorithm is able to improve the output prediction accuracy for all the three objectives.

SINDy for delay-differential equations: application to model bacterial zinc response

We extend the data-driven method of sparse identification of nonlinear dynamics (SINDy) developed by Brunton et al. , Proc. Natl Acad. Sci. USA 113 (2016) to the case of delay differential equations (DDEs). This is achieved in a bilevel optimization procedure by first applying SINDy for fixed delay and then subsequently optimizing the error of the reconstructed SINDy model over delay times. We test the SINDy-delay method on a noisy short dataset from a toy DDE and show excellent agreement. We then apply the method to experimental data of gene expressions in the bacterium Pseudomonas aeruginosa subject to the influence of zinc. The derived SINDy model suggests that the increase in zinc concentration mainly affects the time delay and not the strengths of the interactions between the different agents controlling the zinc export mechanism.

Comparative Analysis of Machine Learning Regression Models for Unknown Dynamics

System identification methods can enable scientists and engineers to model a system for analysis, estimation, or activity planning. Machine Learning (ML) regression algorithms are a useful data-driven system identification tool that can be used in many fields, such as signal identification, wireless communication, or dynamic modelling. However, researchers must select an appropriate algorithm depending on the system’s complexity. In this study, we evaluate the performance of three ML algorithms: Polynomial Fit, Artificial Neural Network (ANN), and Sparse Identification of Non-linear Dynamics (SINDy), to perform model identification in four different time-invariant dynamic environments. We trained each algorithm using 100 simulated data sets and validated them with ten different trajectories. We compare the results using an error distribution framework, demonstrating that ANN had the lowest prediction error, SINDy had comparable performance for three dynamic environments, but none of the algorithms reliably predicted the discontinuous accelerations. This study demonstrated that spacecraft control systems with continuous dynamics may benefit from ML methods.

Data-Enabled Identification of Nonlinear Dynamics of Water Systems using Sparse Regression Technique

The complex, multi-variable, highly nonlinear and strong coupling characteristics of water distribution systems (WDSs) has significantly limited the capability of model-based approaches for control purposes in such systems. With the emerging application of high-resolution metering devices and historical data, model-free identification of WDSs can facilitate the control design without tedious modeling complexities. This paper develops a data-driven framework to facilitate the identification of nonlinear models of WDSs using available data. A quadruple tank system that represents the nonlinear and strong coupling nature of WDSs is considered as the test system and sparse identification of nonlinear dynamics (SINDy) is utilized to identify the nonlinear dynamics from the data. Unlike existing modeling approaches that either heavily rely on knowing the detailed dynamics of the system (model-based) or designs that relay on large historical data and are not interpretable (data-driven approaches), the proposed model-free identification framework is parsimonious, which can accurately capture the dynamics of the quadruple tank process with available measurements suitable for control problems. The effectiveness of the proposed approach is validated using time-domain simulations in MATLAB.

Sparse Modeling of Energy Storage Systems in Presence of Noise

This paper presents a reduced-order modeling technique for energy storage systems such as Lithium-ion batteries (LiBs). Data-driven models offer a solution to represent the system dynamics without requiring in-situ measurements and proprietary information needed by mechanistic models. However, such models can have poor performance in unseen scenarios as they tend to overfit the training data. Here, we present a physics-inspired data-driven model to discover the governing equations of LiBs using only the excitation inputs and measured outputs. Instead of adding generic terms to discover the model, we seek physics-inspired reduced-order nonlinear models. The method is based on sparse identification of nonlinear dynamics with control, and the sparsification was achieved using sequentially thresholded ridge regression. Further, we extended the technique to work with noisy data using unscented Kalman filters, which update the terms for an enhanced state of charge (SOC) and voltage estimation. The trade-off between model accuracy and complexity is determined using threshold and regularization parameters. We formulated the problem to treat them as hyperparameters and adjust them using a training and a validation set. The model was trained on uniformly distributed electrical current signals with maximum amplitudes of 2C charge and 4C discharge rates. We used the US-highway profile as the validation set. The model's ability to predict unseen scenarios was assessed with urban dynamometer driving schedule data, where the identified model achieved the normalized root mean square error of <1.1e-3 for SOC and Voltage predictions.

Model Predictive Control Technique for Ducted Fan Aerial Vehicles Using Physics-Informed Machine Learning

This paper proposes a model predictive control (MPC) approach for ducted fan aerial robots using physics-informed machine learning (ML), where the task is to fully exploit the capabilities of the predictive control design with an accurate dynamic model by means of a hybrid modeling technique. For this purpose, an indigenously developed ducted fan miniature aerial vehicle with adequate flying capabilities is used. The physics-informed dynamical model is derived offline by considering the forces and moments acting on the platform. On the basis of the physics-informed model, a data-driven ML approach called adaptive sparse identification of nonlinear dynamics is utilized for model identification, estimation, and correction online. Thereafter, an MPC-based optimization problem is computed by updating the physics-informed states with the physics-informed ML model at each step, yielding an effective control performance. Closed-loop stability and recursive feasibility are ensured under sufficient conditions. Finally, a simulation study is conducted to concisely corroborate the efficacy of the presented framework.

Data-driven sparse identification of galloping model of iced quad bundle conductors

The galloping of iced conductors is a serious threat to the safe operation of power systems. Establishing an accurate galloping model of iced conductors has always been a difficult point in galloping research. Therefore, the sparse identification of nonlinear dynamics (SINDy) algorithm is used to identify the galloping model from noise measurement data. A theoretical model of galloping of iced quad bundle conductors is established. Meanwhile, the algorithm is used to identify the simulated data of the theoretical model. The parameter identification ability of the algorithm under noisy velocity measurement is analyzed. An excellent denoising method was selected for data preprocessing, and then the model identification effect of the algorithm after data preprocessing is studied. Besides, the accuracy of the prediction model based on this algorithm and the support vector regression (SVR) prediction model under different training data lengths are compared. The results show that the model identified by the SINDy algorithm in the noise measurement data after data preprocessing has high accuracy and robustness. Moreover, the amount of data used is small. The model identified by this algorithm plays an important role in the rapid investigation, prediction and early warning of galloping phenomena.

Dimensionally consistent learning with Buckingham Pi.

In the absence of governing equations, dimensional analysis is a robust technique for extracting insights and finding symmetries in physical systems. Given measurement variables and parameters, the Buckingham Pi theorem provides a procedure for finding a set of dimensionless groups that spans the solution space, although this set is not unique. We propose an automated approach using the symmetric and self-similar structure of available measurement data to discover the dimensionless groups that best collapse these data to a lower dimensional space according to an optimal fit. We develop three data-driven techniques that use the Buckingham Pi theorem as a constraint: (1) a constrained optimization problem with a non-parametric input-output fitting function, (2) a deep learning algorithm (BuckiNet) that projects the input parameter space to a lower dimension in the first layer and (3) a technique based on sparse identification of nonlinear dynamics to discover dimensionless equations whose coefficients parameterize the dynamics. We explore the accuracy, robustness and computational complexity of these methods and show that they successfully identify dimensionless groups in three example problems: a bead on a rotating hoop, a laminar boundary layer and Rayleigh-Bénard convection.

Modeling and Control of Nonlinear Processes Using Sparse Identification: Using Dropout to Handle Noisy Data

Sparse identification of nonlinear dynamics (SINDy) is a recent nonlinear modeling technique that has demonstrated superior performance in modeling complex time-series data in the form of first-order ordinary differential equations (ODEs), which are explicit and continuous in time. However, a crucial step in the SINDy algorithm involves estimating the time derivative of the states from the discrete, measured data. Therefore, the presence of noise can greatly deteriorate the performance if it is not carefully considered and accounted for. In this work, SINDy is used with ensemble learning, where multiple models are identified to improve the overall/final nonlinear model’s performance. Specifically, in the SINDy algorithm, a fraction of the library functions considered for the ODE model representation are randomly dropped out in each submodel to favor model sparsity and stability at the possible risk of lowering the model accuracy. This trade-off is controlled by manipulating the fraction of the library functions dropped out and the total number of models generated, both of which are considered as hyperparameters to be tuned in the proposed algorithm. Data from open-loop simulations of a large-scale chemical plant are generated using the well-known high-fidelity process simulator, Aspen Plus Dynamics, and corrupted with substantial sensor noise to be implemented in the newly proposed algorithm, dropout-SINDy. The dropout-SINDy models obtained from training with the noisy data are then tested in open-loop simulations to demonstrate accurate identification of the steady-state and reasonably close transient behavior under a variety of initial conditions and manipulated input values. Finally, the constructed models are used in a Lyapunov-based model predictive controller to control the large-scale Aspen process, meeting desired closed-loop stability and performance specifications.

Sparse dynamical system identification with simultaneous structural parameters and initial condition estimation

Sparse Identification of Nonlinear Dynamics (SINDy) has been shown to successfully recover governing equations from data; however, this approach assumes the initial condition to be exactly known in advance and is sensitive to noise. In this work we propose an integral SINDy (ISINDy) method to simultaneously identify model structure and parameters of nonlinear ordinary differential equations (ODEs) from noisy time-series observations. First, the states are estimated via penalized spline smoothing and then substituted into the integral-form numerical discretization solver, leading to a sparse pseudo linear regression. Then, the sequential threshold least squares is performed to extract the fewest active terms from the overdetermined set of candidate features, thereby estimating structural parameters and initial condition simultaneously and meanwhile, making the identified dynamics parsimonious and interpretable. Simulations detail the method’s recovery accuracy and robustness to noise. Examples include a logistic equation, Lotka–Volterra system, and Lorenz system.

Data-driven identification of coupling closure equations in vortex-induced vibrations phenomenological models

High-fidelity computational models for describing fluid–structure interaction used in vortex-induced vibrations applications, even providing fine and accurate descriptions of quantities of interest, might not be adequate when multiple code runs are required, like in exploring an input space or synthesizing a controller. Reduced order or simplified models play a crucial role in such context. Here, wake oscillator modeling is adopted, in which, as the interest is devoted to the solid response, the flow, more specifically the action of its vortex shedding, is described by a nonlinear oscillator. This leads to the need to introduce closure models for the fluid–structure interaction. Previous works use suggested functional forms and identify the associated parameters through inverse formulations and available data. A different approach is pursued here. Instead of estimating parameters, the missing term describing the action of the flow on the structure is cast as a model discrepancy to be discovered through the sparse identification of nonlinear dynamics, given a previously chosen library of functions, as identifying a form sheds light on the underlying physics and helps in building a predictive model. An example, dealing with multiple scenarios, is explored from different perspectives, in order to assess the performance of the proposed methodology.

Equation discovery from data: promise and pitfalls, from rabbits to Mars

The problem of equation discovery seeks to reconstruct the underlying dynamics of a time-varying system from observations of the system, and moreover to do so in an instructive way such that we may understand these underlying dynamics from the reconstruction.This article illustrates one type of modern equation discovery method (sparse identification of nonlinear dynamics, or SINDy) in the context of two classic problems. The presentation is in a tutorial style intended to be accessible to students, and could form a useful module in undergraduate or graduate courses in modelling, data analysis, or numerical methods. In this style we explore the strengths and limitations of these methods. We also demonstrate, through use of a carefully constructed example, a new result about the relationship between the reconstructed and true models when a na\"ive polynomial basis is used.

An improved sparse identification of nonlinear dynamics with Akaike information criterion and group sparsity.

A crucial challenge encountered in diverse areas of engineering applications involves speculating the governing equations based upon partial observations. On this basis, a variant of the sparse identification of nonlinear dynamics (SINDy) algorithm is developed. First, the Akaike information criterion (AIC) is integrated to enforce model selection by hierarchically ranking the most informative model from several manageable candidate models. This integration avoids restricting the number of candidate models, which is a disadvantage of the traditional methods for model selection. The subsequent procedure expands the structure of dynamics from ordinary differential equations (ODEs) to partial differential equations (PDEs), while group sparsity is employed to identify the nonconstant coefficients of partial differential equations. Of practical consideration within an integrated frame is data processing, which tends to treat noise separate from signals and tends to parametrize the noise probability distribution. In particular, the coefficients of a species of canonical ODEs and PDEs, such as the Van der Pol, Rössler, Burgers’ and Kuramoto–Sivashinsky equations, can be identified correctly with the introduction of noise. Furthermore, except for normal noise, the proposed approach is able to capture the distribution of uniform noise. In accordance with the results of the experiments, the computational speed is markedly advanced and possesses robustness.

Learning anisotropic interaction rules from individual trajectories in a heterogeneous cellular population

Interacting particle system (IPS) models have proven to be highly successful for describing the spatial movement of organisms. However, it is challenging to infer the interaction rules directly from data. In the field of equation discovery, the weak-form sparse identification of nonlinear dynamics (WSINDy) methodology has been shown to be computationally efficient for identifying the governing equations of complex systems from noisy data. Motivated by the success of IPS models to describe the spatial movement of organisms, we develop WSINDy for the second-order IPS to learn equations for communities of cells. Our approach learns the directional interaction rules for each individual cell that in aggregate govern the dynamics of a heterogeneous population of migrating cells. To sort a cell according to the active classes present in its model, we also develop a novel ad hoc classification scheme (which accounts for the fact that some cells do not have enough evidence to accurately infer a model). Aggregated models are then constructed hierarchically to simultaneously identify different species of cells present in the population and determine best-fit models for each species. We demonstrate the efficiency and proficiency of the method on several test scenarios, motivated by common cell migration experiments.

SINDy-SA framework: enhancing nonlinear system identification with sensitivity analysis

Machine learning methods have revolutionized studies in several areas of knowledge, helping to understand and extract information from experimental data. Recently, these data-driven methods have also been used to discover structures of mathematical models. The sparse identification of nonlinear dynamics (SINDy) method has been proposed with the aim of identifying nonlinear dynamical systems, assuming that the equations have only a few important terms that govern the dynamics. By defining a library of possible terms, the SINDy approach solves a sparse regression problem by eliminating terms whose coefficients are smaller than a threshold. However, the choice of this threshold is decisive for the correct identification of the model structure. In this work, we build on the SINDy method by integrating it with a global sensitivity analysis (SA) technique that allows to hierarchize terms according to their importance in relation to the desired quantity of interest, thus circumventing the need to define the SINDy threshold. The proposed SINDy-SA framework also includes the formulation of different experimental settings, recalibration of each identified model, and the use of model selection techniques to select the best and most parsimonious model. We investigate the use of the proposed SINDy-SA framework in a variety of applications. We also compare the results against the original SINDy method. The results demonstrate that the SINDy-SA framework is a promising methodology to accurately identify interpretable data-driven models.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11071-022-07755-2.

System Identification For Enhanced Flow Description From Multi-Pulse PIV

We propose a novel approach to obtain a time-resolved description from non-time-resolved PIV measurements. The method needs in input velocity and acceleration fields from ( at least) 3-pulse PIV data, without time r esolution. A sparse identification of nonlinear dynamics (based on the SINDy technique) is carried out to estimate the time history of coordinates in a low-dimensional space from Proper Orthogonal Decomposition (POD). The output is a reduced-order model of the dynamics which can be integrated to obtain time-resolved velocity fields using t he s napshots as i nitial c onditions. The t ime-resolved e volution of t he pressure fields is th en derived from the momentum equation. The time history of the velocity and pressure fields is obtained through a weighted Backward-Forward Integration (BFI) of the identified s ystem. Time super-sampling can be achieved by integration between consecutive non-time-resolved realizations. In alternative, simple one-directional time integration can be used for short-time prediction starting from individual snapshots. The algorithm is validated using a synthetic test case from 2D Direct Numerical Simulation of the wake of a fluidic pinball at a Reynolds number Re equal to 130, and PIV measurements of the wake of a NACA 0018 wing profile with Re = 4 800. We show that the method is able to reconstruct the flow dynamics for horizons of several convective times, provided that the basis for the data reduction is sufficiently c ompact. Our results suggest that manifold learning and data assimilation can be combined to obtain an enhanced flow description.