Identification Of Nonlinear Dynamic Systems Research Articles

Data-driven reconstruction of chaotic dynamical equations: The Hénon–Heiles type system

In this study, the classical two-dimensional potential VN=12mω2r2+1NrNsin(Nθ), N∈Z+, is considered. At N=1,2, the system is superintegrable and integrable, respectively, whereas for N>2 it exhibits a richer chaotic dynamics. For instance, at N=3 it coincides with the Hénon–Heiles system. The periodic, quasi-periodic and chaotic motions are systematically characterized employing time series, Poincaré sections, symmetry lines and the largest Lyapunov exponent as a function of the energy E and the parameter N. Concrete results for the lowest cases N=3,4 are presented in complete detail. This model is used as a benchmark system to estimate the accuracy of the Sparse Identification of Nonlinear Dynamical Systems (SINDy) method, a data-driven algorithm which reconstructs the underlying governing dynamical equations. We pay special attention at the transition from regular motion to chaos and how this influences the precision of the algorithm. In particular, it is shown that SINDy is a robust and stable tool possessing the ability to generate non-trivial approximate analytical expressions for periodic trajectories as well.

Identification of nonlinear dynamical system based on adaptive radial basis function neural networks

Identification of nonlinear dynamical system based on adaptive radial basis function neural networks

Recurrent context layered radial basis function neural network for the identification of nonlinear dynamical systems

This paper proposes a novel recurrent context layered radial basis function neural network (RCLRBFNN) for the identification of nonlinear dynamical systems. The proposed model consists of an additional context layer in which the nodes represent the unit-delayed outputs of the hidden layer radial centers. These delayed outputs undergo a nonlinear transformation by applying a tangent hyperbolic function. These transformed signals connect to the output layer neuron through the adjustable context layered weights. To tune the parameters of the proposed model the update equations are derived using the dynamic back-propagation algorithm. Further, an adaptive learning rate scheme is proposed to improve the performance of the learning algorithm. In the simulation experiment, a total of two examples are considered to test the efficacy and performance of the proposed model. The performance comparison is made with the conventional structure of the radial basis function neural network (RBFNN), Jordan Recurrent neural network (JRNN), and the feed-forward neural network (FFNN) (which is nothing but a single-layered multi-layered perceptron). Both the disturbance signal as well as system’s uncertainty scenarios are considered to test the robustness shown by the proposed model. The results showed that the proposed model has delivered a better identification accuracy as compared to the other neural models.

A data-driven implicit deep adaptive neuro-fuzzy inference system capable of manifold learning for function approximation

Fuzzy Neural Networks (FNN) have the ability of decision-making based on constructing semi-ellipsoidal clusters in the input space as the antecedent parts of their fuzzy rules. To determine the output value for each input instance, FNNs consider its membership degree to different sub-regions of the input space. However, forming such meaningful sub-regions is not possible in all applications due to the nonlinear interactions among input variables and their low information gain. Indeed, the samples could be distributed on a manifold in the input space. Therefore, to cover the input space, we need lots of rules, each representing a small region of input space. This issue decreases the generalization ability of the model along with its explainability. Consequently, to efficiently form fuzzy rules, first, it is necessary to unfold the manifold by mapping the samples to an appropriate embedding space. Next, the fuzzy rules in the form of semi-ellipsoidal regions should be constructed in this extracted feature space. Deep Fuzzy Neural Networks address this problem by representation learning through stacking multiple cascade mapping layers. In this paper, we propose a novel approach for nonlinear function approximation and time-series prediction problems, based on using the kernel trick to implicitly learn the mapping function to the new feature space. Moreover, to initialize the fuzzy rules, a KNN-based method using the kernel trick is proposed. A hierarchical Levenberg–Marquardt approach is applied to learn the model’s parameters. The performance and structure of the proposed method are studied and compared with some other relevant methods in synthetic and real-world benchmarks. Based on these experiments, the proposed method has the best performance with the most parsimonious architecture. According to these experiments, the test RMSE of the proposed method is 0.002 for Mc-Glass chaotic time-series prediction, 0.015 for a Nonlinear dynamic system identification, 0.0345 for Box–Jenkins nonlinear system identification, 0.0609 for Fuel consumption prediction of automobiles, 10.24 for Sydney stock price tracking, and 0.595 for California housing.

A kernel framework for learning differential equations and their solution operators

This article presents a three-step kernel framework for regression of the functional form of differential equations (DEs) and learning their solution operators. Given a training set consisting of pairs of noisy DE solutions and source/boundary terms on a mesh: (i) kernel smoothing is utilized to denoise the data and approximate derivatives of the solution; (ii) This information is then used in a kernel regression model to learn the functional form of the DE; (iii) The learned DE is then used within a numerical solver to approximate the solution of the DE with a new source term or initial data, thereby constituting an operator learning framework. Numerical experiments compare the method to state-of-the-art algorithms. In DE learning our framework matches the performance of Sparse Identification of nonlinear Dynamical Systems (SINDy) while in operator learning the method has superior performance compared to well-established neural network methods in low training data regimes.

A New Method for Dynamical System Identification by Optimizing the Control Parameters of Legendre Multiwavelet Neural Network

Wavelet neural networks have been widely applied to dynamical system identification fields. The most difficult issue lies in selecting the optimal control parameters (the wavelet base type and corresponding resolution level) of the network structure. This paper utilizes the advantages of Legendre multiwavelet (LW) bases to construct a Legendre multiwavelet neural network (LWNN), whose simple structure consists of an input layer, hidden layer, and output layer. It is noted that the activation functions in the hidden layer are adopted as LW bases. This selection if based on the its rich properties of LW bases, such as piecewise polynomials, orthogonality, various regularities, and more. These properties contribute to making LWNNs more effective in approximating the complex characteristics exhibited by uncertainties, step, nonlinear, and ramp in the dynamical systems compared to traditional wavelet neural networks. Then, the number of selection LW bases and the corresponding resolution level are effectively optimized by the simple Genetic Algorithm, and the improved gradient descent algorithm is implemented to learn the weight coefficients of LWNN. Finally, four nonlinear dynamical system identification problems are applied to validate the efficiency and feasibility of the proposed LWNN-GA method. The experiment results indicate that the LWNN-GA method achieves better identification accuracy with a simpler network structure than other existing methods.

A recurrent neural network-based identification of complex nonlinear dynamical systems: a novel structure, stability analysis and a comparative study

A recurrent neural network-based identification of complex nonlinear dynamical systems: a novel structure, stability analysis and a comparative study

Exploiting Spike-and-Slab Prior for Variational Estimation of Nonlinear Systems

Identification of nonlinear dynamic systems remains challenging nowadays. Although the nonlinear auto-regressive with exogenous input (NARX) model is flexible to describe complex nonlinear behaviors, it is critical to select appropriate model terms to obtain a parsimonious description of the system. In this study, a variational Bayesian (VB) approach to the estimation of NARX systems is developed. A sparsity inducing prior is introduced for model parameters, and the sparseness can be automatically determined by the weighting factor of such prior. The Bayesian model for the identification problem is constructed, and an iterative model pruning strategy is formulated to remove redundant terms and address the structure selection problem. Instead of the single-point estimation, the model parameters with their uncertainties are jointly estimated under VB framework. Finally, one numerical example and several benchmark datasets are adopted to illustrate that the developed algorithm can work promisingly.

On adaptive identification of systems having multiple nonlinearities

Objectives. The solution to the relevant problem of identifying systems with multiple nonlinearities depends on such factors as feedback, ways of connecting nonlinear links, and signal properties. The specifics of nonlinear systems affect control systems design methods. As a rule, the basis for the development of a mathematical model involves the linearization of a system. Under conditions of uncertainty, the identification problem becomes even more relevant. Therefore, the present work sets out to develop an approach to the identification of nonlinear dynamical systems under conditions of uncertainty. In order to obtain a solution to the problem, an adaptive identification method is developed by decomposing the system into subsystems.Methods. Methods applied include the adaptive identification method, implicit identified representation, S-synchronization of a nonlinear system, and the Lyapunov vector function method.Results. A generalization of the excitation constancy condition based on fulfilling the S-synchronizability for a nonlinear system is proposed along with a method for decomposing the system in the output space. Adaptive algorithms are obtained on the basis of the second Lyapunov method. The boundedness of the adaptive system trajectories in parametric and coordinate spaces is demonstrated. Approaches for self-oscillation generation and nonlinear correction of a nonlinear system are considered along with obtained exponential stability conditions for the adaptive system.Conclusions. Simulation results confirm the possibility of applying the proposed approach to solving the problems of adaptive identification while taking the estimation of the structural identifiability (S-synchronization) of the system nonlinear part into account. The influence of the structure and relations of the system on the quality of the obtained parametric estimates is investigated. The proposed methods can be used in developing identification and control systems for complex dynamic systems.

Design of a novel robust recurrent neural network for the identification of complex nonlinear dynamical systems

Design of a novel robust recurrent neural network for the identification of complex nonlinear dynamical systems

Physics-Informed Machine Learning for Surrogate Modeling of Heat Transfer Phenomena

Abstract In this paper, we propose a sparse modeling method for automatically creating a surrogate model for nonlinear time-variant systems from a very small number of time series data with nonconstant time steps. We developed three machine learning methods, namely, (1) a data preprocessing method for considering the correlation between errors, (2) a sequential thresholded non-negative least-squares method based on term size criteria, and (3) a solution space search method involving similarity model classification—to apply sparse identification of nonlinear dynamical systems, as first proposed in 2016, to temperature prediction simulations. The proposed method has the potential for wide application to fields where the concept of equivalent circuits can be applied. The effectiveness of the proposed method was verified using time series data obtained by thermofluid analysis of a power module. Two types of cooling systems were verified: forced air cooling and natural air cooling. The model created from the thermofluid analysis results with fewer than the number of input parameters, predicted multiple test data, including extrapolation, with a mean error of less than 1 K. Because the proposed method can be applied using a very small number of data, has a high extrapolation accuracy, and is easy to interpret, it is expected not only that design parameter can be fine-tuned and actual loads can be taken into account, but also that condition-based maintenance can be realized through real-time simulation.

Discovering governing equations in discrete systems using PINNs

Sparse identification of nonlinear dynamical systems is a topic of continuously increasing significance in the dynamical systems community. Here we explore it at the level of lattice nonlinear dynamical systems of many degrees of freedom. We illustrate the ability of a suitable adaptation of Physics-Informed Neural Networks (PINNs) to solve the inverse problem of parameter identification in such discrete, high-dimensional systems inspired by physical applications. The methodology is illustrated in a diverse array of examples including real-field ones (ϕ4 and sine-Gordon), as well as complex-field (discrete nonlinear Schrödinger equation) and going beyond Hamiltonian to dissipative cases (the discrete complex Ginzburg–Landau equation). Both the successes, as well as some limitations of the method are discussed along the way.

Discrete identification of continuous non-linear andnon-stationary dynamical systems that is insensitive to noise correlation and measurement outliers

Discrete identification of continuous non-linear andnon-stationary dynamical systems that is insensitive to noise correlation and measurement outliers

Prediction and identification of nonlinear dynamical systems using machine learning approaches

Nonlinear dynamical systems are widely implemented in many areas. The prediction and identification of these dynamical systems purely based on observational data are of great significance for practical applications. In the work, we develop a machine learning based approach called Runge–Kutta guided next-generation reservoir computing (RKNG-RC). The proposed scheme can process data information generated by the most complicated nonlinear dynamical systems such as chaotic Lorenz63 system even with noise, and experimental systems such as chaotic Chua’s electronic circuit, showing an outstanding ability for prediction tasks. More importantly, the RKNG-RC is found to have distinctive interpretability that from the trained weights the ordinary differential equation governing the observable data can be deduced, which is beyond the processing capacities of traditional approaches. The work provides an efficient platform for processing information generated by various dynamical systems.

Data-informed reservoir computing for efficient time-series prediction

We propose a new approach to dynamical system forecasting called data-informed-reservoir computing (DI-RC) that, while solely being based on data, yields increased accuracy, reduced computational cost, and mitigates tedious hyper-parameter optimization of the reservoir computer (RC). Our DI-RC approach is based on the recently proposed hybrid setup where a knowledge-based model is combined with a machine learning prediction system, but it replaces the knowledge-based component by a data-driven model discovery technique. As a result, our approach can be chosen when a suitable knowledge-based model is not available. We demonstrate our approach using a delay-based RC as the machine learning component in conjunction with sparse identification of nonlinear dynamical systems for the data-driven model component. We test the performance on two example systems: the Lorenz system and the Kuramoto–Sivashinsky system. Our results indicate that our proposed technique can yield an improvement in the time-series forecasting capabilities compared with both approaches applied individually, while remaining computationally cheap. The benefit of our proposed approach, compared with pure RC, is most pronounced when the reservoir parameters are not optimized, thereby reducing the need for hyperparameter optimization.

Recurrent general type-2 fuzzy neural networks for nonlinear dynamic systems identification

This paper introduces a recurrent general type-2 Takagi–Sugeno–Kang fuzzy neural network (RGT2-TSKFNN) for the identification of nonlinear systems. In the proposed structure, the general type-2 fuzzy set (GT2FS) and a recurrent fuzzy neural network (RFNN) are combined to obviate the data uncertainties. The fuzzy firing strengths in the developed structure are returned to the network input as internal variables. In the proposed structure, GT2FS is utilized to characterize the antecedent parts while the consequent parts are performed using TSK type. The issues of constructing a RGT2-TSKFNN involve type reduction, structure learning as well as parameter learning. An efficient strategy is developed by utilizing alpha-cuts to decompose a GT2FS into several interval type-2 fuzzy sets (IT2FSs). In order to solve the computation time of the type-reduction issue, a direct defuzzification method is used instead of iterative nature of Karnik–Mendel​ (KM) algorithm. Type-2 fuzzy clustering and Lyapunov criteria are utilized for online structure learning as well as the antecedent and consequent parameters, respectively for reducing the number of rules and guaranteeing the stability of the proposed RGT2-TSKFNN. The reported comparative analysis of the simulation results is utilized to estimate the performance of the proposed RGT2-TSKFNN with respect to other popular type-2 FNNs (T2FNNs) methodologies.

The identification of piecewise non-linear dynamical system without understanding the mechanism.

This paper designs an algorithm to distill the piecewise non-linear dynamical system from the data without prior knowledge. The system to be identified does not have to be written as a known model term or be thoroughly understood. We exploit the fact that an unknown piecewise non-linear system can be decomposed into the Fourier series as long as its equations of motion are Riemann integrable. Based on this property, we reduce the challenge of finding the correct model to discovering the Fourier series approximation. However, the Fourier series approximation of the piecewise function is inaccurate. The new method takes advantage of this weakness to determine whether the model has piecewise features and to find a way to discover the discontinuity set. Then, the dynamical system on each segment is identified as a pure Fourier series. Identification of intricate models can be achieved in simple steps. The results show that the method can accurately discover the equation of motion and precisely capture the non-smooth characteristic. Next, the prediction and further detailed analysis can be carried out.

A novel neural emulator identification of nonlinear dynamical systems using Lyapunov stability theory

This paper deals with a new weight-updating algorithm using Lyapunov stability theory (LST) for the training of a neural emulator (NE), of nonlinear systems, connected by an autonomous algorithm inspired from the real-time recurrent learning (RTRL). The proposed method is formulated by an inequality-constraint optimization problem where the Lagrange multiplier theory is used as the optimization tool. The contribution of this paper is the integration of the LST into the Lagrange constraint function to synthesize a new analytical adaptation gain rate satisfying the asymptotic stability of the NE and providing good emulation performances. To confirm the good performances and the convergence ability of the proposed adaptation algorithm, a numerical example and an experimental validation on a chemical reactor are proposed.

A study on data-driven identification and representation of nonlinear dynamical systems with a physics-integrated deep learning approach: Koopman operators and nonlinear normal modes

Identifying the intrinsic coordinates or modes of dynamical systems is essential to understand, analyze, and characterize the underlying complex dynamical behaviors. This is a critical challenge for nonlinear dynamical systems because the modal transformation, which is universal for linear systems, no longer applies. Koopman operators and nonlinear normal modes (NNMs) are two main frameworks aiming to provide a general representation of nonlinear dynamics. In this study, we investigate their abilities on representing nonlinear dynamical systems. Addressing the challenge that the closed-form models or equations of dynamical systems are generally unknown realistically, we first present a physics-integrated deep learning-based data-driven framework to identify the eigenfunction of Koopman operators and the nonlinear modal transformation function of NNMs, respectively. We then evaluate their representation abilities by their reconstruction and predictive accuracy of the nonlinear dynamical response using the identified Koopman modes and NNMs. We conduct numerical experiments on Duffing systems with varying nonlinearity levels and dimensions, and observe that NNMs achieve higher representation accuracy and better computational efficiency than Koopman operators using the same dimension of intrinsic coordinates or modes.

A regularised fast recursive algorithm for fraction model identification of nonlinear dynamic systems

The fraction model has been widely used to represent a range of engineering systems. To accurately identify the fraction model is however challenging, and this paper presents a regularised fast recursive algorithm (RFRA) to identify both the true fraction model structure and the associated unknown model parameters. This is achieved first by transforming the fraction form to a linear combination of nonlinear model terms. Then the terms in the denominator are used to form a regularisation term in the cost function to offset the bias induced by the linear transformation. According to the structural risk minimisation principle based on the new cost function, the model terms are selected based on their contributions to the cost function and the coefficients are then identified recursively without explicitly solving the inverse matrix. The proposed method is proved to have low computational complexity. Simulation results confirm the efficacy of the method in fast identification of the true fraction models for the targeted nonlinear systems.