Identification Of Nonlinear Dynamic Systems Research Articles

Boosting the Model Discovery of Hybrid Dynamical Systems in an Informed Sparse Regression Approach

Abstract We present an efficient data-driven sparse identification of dynamical systems. The work aims at reconstructing the different sets of governing equations and identifying discontinuity surfaces in hybrid systems when the number of discontinuities is known a priori. In a two-stage approach, we first locate the switches between separate vector fields. Then, the dynamics among the manifolds are regressed, in this case by making use of the existing algorithm of Brunton et al. (2016, “Discovering Governing Equations From Data by Sparse Identification of Nonlinear Dynamical Systems,” Proc. Natl. Acad. Sci., 113(15), pp. 3932–3937). The reconstruction of the discontinuity surfaces comes as the outcome of a statistical analysis implemented via symbolic regression with small clusters (microclusters) and a rigid library of models. These allow to classify all the feasible discontinuities that are clustered and to reduce them into the actual discontinuity surfaces. The performances of the sparse regression hybrid model discovery are tested on two numerical examples, namely, a canonical spring-mass hopper and a free/impact electromagnetic energy harvester (FIEH), engineering archetypes characterized by the presence of a single and double discontinuity, respectively. Results show that a supervised approach, i.e., where the number of discontinuities is pre-assigned, is computationally efficient and it determines accurately both discontinuities and set of governing equations. A large improvement in the time of computation is found with the maximum achievable reliability. Informed regression-based identification offers the prospect to outperform existing data-driven identification approaches for hybrid systems at the expense of instructing the algorithm for expected discontinuities.

Chebyshev Functional Link Spline Neural Filter for Nonlinear Dynamic System Identification

Chebyshev Functional Link Spline Neural Filter for Nonlinear Dynamic System Identification

Sparse identification of nonlinear dynamical systems via non-convex penalty least squares.

This paper proposes a non-convex penalty regression method to identify governing equations of nonlinear dynamical systems from noisy state measurements. The idea to connect the non-convex penalty function instead of the l1- norm with least squares is due to the fact that the l1- norm excessively penalizes large coefficients and may incur estimation bias. The purpose of this work is to improve the accuracy and robustness in regression tasks. A threshold non-convex penalty sparse least squares optimization algorithm is developed, wherein the threshold parameter is selected using the L-curve criterion. With two examples of nonlinear dynamical systems, we illustrate the accuracy and robustness of the non-convex penalty least squares on noisy state measurements, indicating the validity of our method in a wide range of potential applications.

Uncertainty Analysis and Experimental Validation of Identifying the Governing Equation of an Oscillator Using Sparse Regression

In recent years, the rapid growth of computing technology has enabled identifying mathematical models for vibration systems using measurement data instead of domain knowledge. Within this category, the method Sparse Identification of Nonlinear Dynamical Systems (SINDy) shows potential for interpretable identification. Therefore, in this work, a procedure of system identification based on the SINDy framework is developed and validated on a single-mass oscillator. To estimate the parameters in the SINDy model, two sparse regression methods are discussed. Compared with the Least Squares method with Sequential Threshold (LSST), which is the original estimation method from SINDy, the Least Squares method Post-LASSO (LSPL) shows better performance in numerical Monte Carlo Simulations (MCSs) of a single-mass oscillator in terms of sparseness, convergence, identified eigenfrequency, and coefficient of determination. Furthermore, the developed method SINDy-LSPL was successfully implemented with real measurement data of a single-mass oscillator with known theoretical parameters. The identified parameters using a sweep signal as excitation are more consistent and accurate than those identified using impulse excitation. In both cases, there exists a dependency of the identified parameter on the excitation amplitude that should be investigated in further research.

Deriving a thermal model from data by machine learning based on physical laws

We propose a method for creating a reduced-order model (ROM) for temperature prediction that is applicable to nonlinear time-variant systems, able to be created using a very small amount of data, and easy to interpret. Our nonlinear time-variant ROM is an extension of sparse identification of nonlinear dynamical systems, which was first proposed in 2016. Three machine learning methods are developed for automatically deriving a thermal network model from time-series data. Link relationships between temperature nodes and parameter settings, such as thermal resistance and heat capacity, are automatically inferred by machine learning. The effectiveness of the proposed method is demonstrated using the results of thermo-fluid analysis for a natural air-cooled power module in which thermal resistance is a function of temperature. The ROM was created using the results of 15 thermo-fluid analyses, with is fewer than the 25 independent variables in the ROM. The computational efficiency of the created ROM is very high, and unknown data, including extrapolated data, were predicted with error of less than 1 K. By using the proposed method, it is expected that design parameters can be fine-tuned and actual loads can be taken into account, and also that condition-based maintenance can be realized through real-time simulation.

Application of nonlinear system identification for EEG modelling using VMD-based deep random vector functional link network

In this paper, the EEG signal is considered for the development of the model. As the signal is nonlinear and non-stationary, the model is designed accordingly which is similar to nonlinear dynamic system identification. Initially, the signal is decomposed by a robust variational mode decomposition method for which the basic noise components are eliminated. Later, a kurtosis index method is applied to choose the best band-limited intrinsic mode functions (BLIMFs) based on their clean coefficient the model is developed using a random vector functional link neural network (RVFLN) for identification. The use of deep RVFLN provides better results as compared to simple RVFLN as explained in the result section. For verification of the system's robustness, three different epileptic signals known as pre-ictal, inter-ictal and ictal are experienced in this piece of work.

Identification and Stochastic Optimizing the UAV Motion Control in Turbulent Atmosphere

In a class of diffusion Markov processes, we formulate a problem of identification of nonlinear stochastic dynamic systems with random parameters, multiplicative and additive noises, control functions, and the state vector at a final time moment. For such systems, the identifiability conditions are being studied, and necessary conditions are formulated in terms of the general theory of extreme problems. The developed engineering methods for identification and optimizing nonlinear stochastic systems are presented as well as their application for unmanned aerial vehicles under wind disturbances caused by atmospheric turbulence, namely, for optimizing the autopilot parameters during a rotary maneuver of an unmanned aerial vehicle in translational motion, taking into account the identification of its angular velocities.

Modeling Melburnians—Using the Koopman operator to gain insight into crowd dynamics

Describing and forecasting city traffic is challenging, given the array of factors influencing the movement of pedestrians and vehicles. Faced with this complexity, research has focused on machine learning as a way to capture spatio-temporal traffic patterns, based on past sensor data. While the methods can accurately forecast high-dimensional observations, it is often hard to explain the models and their predictions. In many cases, only the predictive performance of an otherwise black box model is evaluated. An accurate but explainable alternative is provided by the Koopman operator, an emerging methodology for data-driven identification of high-dimensional and nonlinear dynamical systems. In this study, we showcase the method by analyzing pedestrian traffic data from Melbourne, Australia. We formulate our computations in the Extended Dynamic Mode Decomposition framework, where we approximate the Koopman operator in a function basis computed with time delay embedding and the Diffusion Map algorithm. The model captures the distinct temporal patterns of 11 traffic sensors simultaneously. Because the dynamics become linear in the operator-theoretic perspective, we can decompose the model into its spectral components. Importantly, these components facilitate interpretation and can increase scientific understanding of the underlying system and the model’s operations. For the Melbourne data, we show that the spectral components connect to the underlying state space geometry and indicate the model’s stability over a prediction horizon. Our study showcases how the Koopman operator framework offers explainable and accurate data-driven predictions in a real-world traffic system. The results can easily be transferred to other traffic systems.

A novel rule-based evolving Fuzzy System applied to the thermal modeling of power transformers

Big Data advancements motivate researchers to develop and improve intelligent models to deal efficiently and effectively with data. In this scenario, time series forecasting obtains even more attention. The literature demonstrated the better performance of such models in this subject. Forecasting is widely used in strategic planning to support decision-making, providing competitive differential to organizations. In this paper, a novel rule-based evolving Fuzzy System is proposed for time series forecasting. This is a robust model able to develop and update its structure in unknown environments, capture dynamics and changes of streams, and produce accurate results even when dealing with complex data. The introduced model implements the distance correlation to improve the rules’ quality by reducing their standard deviation. The model is evaluated using two synthetic datasets: the Mackey–Glass time-series and the nonlinear dynamic system identification. And finally, the introduced system is implemented to predict the hot spot temperature using three datasets from a real power transformer. Hot spot monitoring is necessary to maximize the load capacity and the lifespan of power transformers. The proposed method is evaluated in terms of root-mean-square error, non-dimensional index error, mean absolute error, runtime, and the number of final rules. The results are compared with traditional forecasting models and with some related state-of-the-art rule-based evolving Fuzzy Systems. The new evolving Fuzzy System outperformed the compared models for the Mackey–Glass time-series and the power transformers datasets concerning the errors. A statistical test comprised the superior performance of the introduced model. The algorithm also obtained a competitive execution time and number of final rules. The results demonstrate the high level of autonomy and adaptation of the model to predict accurately complex and non-stationary data. Seeing the importance of accurate models to deal with data to support decision-making, the results suggest the model’s implementation as a forecasting tool in strategic planning.

Modeling and prediction of the transmission dynamics of COVID-19 based on the SINDy-LM method.

The transmission dynamics of COVID-19 is investigated in this study. A SINDy-LM modeling method that can effectively balance model complexity and prediction accuracy is proposed based on data-driven technique. First, the Sparse Identification of Nonlinear Dynamical systems (SINDy) method is used to discover and describe the nonlinear functional relationship between the dynamic terms in the model in accordance with the observation data of the COVID-19 epidemic. Moreover, the Levenberg–Marquardt (LM) algorithm is utilized to optimize the obtained model for improving the accuracy of the SINDy algorithm. Second, the obtained model, which is consistent with the logistic model in mathematical form with small errors and high robustness, is leveraged to review the epidemic situation in China. Otherwise, the evolution of the epidemic in Australia and Egypt is predicted, which demonstrates that this method has universality for constructing the global COVID-19 model. The proposed model is also compared with the extreme learning machine (ELM), which shows that the prediction accuracy of the SINDy-LM method outperforms that of the ELM method and the generated model has higher sparsity.

Identification of autonomous nonlinear dynamical system based on discrete-time multiscale wavelet neural network

Identification of autonomous nonlinear dynamical system based on discrete-time multiscale wavelet neural network

Structural identification with physics-informed neural ordinary differential equations

This paper exploits a new direction of structural identification by means of Neural Ordinary Differential Equations (Neural ODEs), particularly constrained by domain knowledge, such as structural dynamics, thus forming Physics-informed Neural ODEs, aiming at governing equations discovery/approximation. Structural identification problems often entail complex setups featuring high-dimensionality, or stiff ODEs, which pose difficulties in the training and learning of conventional data-driven algorithms who seek to unveil the governing dynamics of a system of interest. In this work, Neural ODEs are re-casted as a two-level representation involving a physics-informed term, that stems from possible prior knowledge of a dynamical system, and a discrepancy term, captured by means of a feed-forward neural network. The re-casted format is highly adaptive and flexible to structural monitoring problems, such as linear/nonlinear structural identification, model updating, structural damage detection, driving force identification, etc. As an added step, for inferring an explainable model, we propose the adoption of sparse identification of nonlinear dynamical systems as an additional tool to distill closed-form expressions for the trained nets, that embed a more straightforward engineering interpretation. We demonstrate the framework on a series of numerical and experimental examples, with the latter pertaining to a structural system featuring highly nonlinear behavior, which is successfully learned by the proposed framework. The proposed structural identification with Physics-informed Neural ODEs comes with the benefits of direct approximation of the governing dynamics, and a versatile and flexible framework for discrepancy modeling in structural identification problems.

Nonlinear dynamical system identification using the sparse regression and separable least squares methods

This paper proposes a novel nonlinear dynamical system identification method based on the sparse regression algorithm and the separable least squares method. To effectively avoid solving the second derivative of the displacement signal and reduce the effect of noise, the Duhamel's integral is adopted to represent the dynamic relationship between the system input and output. In the expression form of Duhamel's integral, nonlinear dynamical system identification can be cast as a separable least squares problem. Thus, the separable least squares method is leveraged to separately identify the parameters of the linear subsystem and the coefficients corresponding to nonlinearities among the nonlinear dynamical system. During the identification process of nonlinear restoring forces, one complete set of nonlinear basis functions are used to represent the nonlinear restoring forces. Not all the candidate nonlinear terms are contributing, however, thus the sparse regression algorithm is adopted to select the actual contributing nonlinear components in the candidate nonlinear terms and eliminate the non-contributing nonlinear components, and then the corresponding parameters of contributing nonlinear components are estimated by the unbiased least squares method. Finally, one RKHS (Reproducing Kernel Hilbert Space)-based non-parametric de-noise method is further proposed to reduce the noise in the vibration displacement and obtain the noise-reduced velocity from the displacement signal. The numerical simulation about the identification of the rotating blade-casing system and the dynamic experiment of the HSLDS (high-static-low-dynamic stiffness) isolator system verify the effectiveness of the new identification method for nonlinear dynamical systems proposed in this paper.

Identification of nonlinear dynamical systems with time delay

This paper proposes a technique to identify nonlinear dynamical systems with time delay. The sparse optimization algorithm is extended to nonlinear systems with time delay. The proposed algorithm combines cross-validation techniques from machine learning for automatic model selection and an algebraic operation for preprocessing signals to filter the noise and for removing the dependence on initial conditions. We further integrate the bootstrapping resampling technique with the sparse regression to obtain the statistical properties of estimation. We use Taylor expansion to parameterize time delay. The proposed algorithm in this paper is computationally efficient and robust to noise. A nonlinear Duffing oscillator is simulated to demonstrate the efficiency and accuracy of the proposed technique. An experimental example of a nonlinear rotary flexible joint is presented to further validate the proposed method.

Sparse identification of nonlinear dynamical systems via reweighted [formula omitted]-regularized least squares

This work proposes an iterative sparse-regularized regression method to recover governing equations of nonlinear dynamical systems from noisy state measurements. The method is inspired by the Sparse Identification of Nonlinear Dynamics (SINDy) approach of Brunton et al. (2016), which relies on two main assumptions: the state variables are known a priori and the governing equations lend themselves to sparse, linear expansions in a (nonlinear) basis of the state variables. The aim of this work is to improve the accuracy and robustness of SINDy in the presence of state measurement noise. To this end, a reweighted ℓ1-regularized least squares solver is developed, wherein the regularization parameter is selected from the corner point of a Pareto curve. The idea behind using weighted ℓ1-norm for regularization – instead of the standard ℓ1-norm – is to better promote sparsity in the recovery of the governing equations and, in turn, mitigate the effect of noise in the state variables. We also present a method to recover single physical constraints from state measurements. Through several examples of well-known nonlinear dynamical systems, we demonstrate empirically the accuracy and robustness of the reweighted ℓ1-regularized least squares strategy with respect to state measurement noise, thus illustrating its viability for a wide range of potential applications.

Gray-Box system modeling using symbolic regression and nonlinear model predictive control of a semibatch polymerization

A nonlinear model, predictive control approach, based on a gray-box model is introduced in this contribution. The gray-box model is built by combining white-box and black-box modeling elements. The white-box elements consist of relevant mass and energy balances, including algebraic relations that describe different phenomena of the system behavior (e.g., reaction kinetics, heat transfer phenomena, thermodynamic assumptions). To fill in or compensate for modeling gaps (e.g., missing kinetics information, heat transfer efficiency, inaccurate thermodynamic assumptions), black-box model elements trained from historical process data are introduced into the model. An algorithm for Sparse Identification of Nonlinear Dynamical Systems (SINDy) that identifies the most promising regressors from a pool of possible explanatory functions is used to generate the model’s black-box part. The proposed gray-box modeling approach and its application to model predictive control were tested for performance and robustness using a benchmark problem, and the approach shows promising results.

Identification of nonlinear dynamic system using machine learning techniques

Identification of nonlinear systems finds extensive applications in control design and stability analysis. To identify complex nonlinear systems, the neural network has drawn the attention of many researchers due to its broad application area. In this paper, an improved identification method based on robust regularised exponentially extended random vector functional link network (RERVFLN) has been proposed for nonlinear system identification. The input is extended using trigonometric expansion which increases the accuracy of the algorithm. To verify the accuracy of the proposed model, some benchmark Monte Carlo simulations are carried out through simulation study and the obtained results are compared with some established techniques such as original RVFLN, ELM, and LMS. Prediction accuracy of the proposed method RERVFLN is higher than the normal RVFLN for different nonlinear systems which is clear from the performance evaluation section.

Identification of Nonlinear Dynamical System Based on Raised-Cosine Radial Basis Function Neural Networks

In this paper, we present and investigate a new type of radial basis function (RBF) neural networks mechanism using raised-cosine (RC) function to identify nonlinear dynamic system. In this design, the RBF neural networks mechanism utilizes RC function to replace Gaussian function, which is called RCRBF. An N-dimensional RC function has the constant interpolation property, which is benefit for the function approximating errors analysis in the neural networks. Based on multivariable RC function approximation theory, we develop how to select the updated parameters and the distance of adjacent nodes in lattice points. Therefore, the proposed networks can uniformly approximate nonlinear dynamical function. As persistency excitation (PE) plays an important part in neural networks learning system, how does PE condition behave in input sequences is formulated by RC function analysis. The weights updating and errors convergence are concluded by Lyapunov function analysis. To illustrate the effectiveness of the proposed RCRBF method, Van Der Pol and Rossler dynamical system are used as test examples, in comparison with GRBF mechanism. The results show that the proposed method has better accurate identification and approximating effect than that of GRBF mechanism.

Identification of nonlinear dynamic system using machine learning techniques

Identification of nonlinear dynamic system using machine learning techniques

Non-Autoregressive vs Autoregressive Neural Networks for System Identification

The application of neural networks to non-linear dynamic system identification tasks has a long history, which consists mostly of autoregressive approaches. Autoregression, the usage of the model outputs of previous time steps, is a method of transferring a system state between time steps, which is not necessary for modeling dynamic systems with modern neural network structures, such as gated recurrent units (GRUs) and Temporal Convolutional Networks (TCNs). We compare the accuracy and execution performance of autoregressive and non-autoregressive implementations of a GRU and TCN on the simulation task of three publicly available system identification benchmarks. Our results show, that the non-autoregressive neural networks are significantly faster and at least as accurate as their autoregressive counterparts. Comparisons with other state-of-the-art black-box system identification methods show, that our implementation of the non-autoregressive GRU is the best performing neural network-based system identification method, and in the benchmarks without extrapolation, the best performing black-box method.