Nonlinear System Identification Research Articles

Online Identification of Nonlinear Systems With Separable Structure.

Separable nonlinear models (SNLMs) are of great importance in system modeling, signal processing, and machine learning because of their flexible structure and excellent description of nonlinear behaviors. The online identification of such models is quite challenging, and previous related work usually ignores the special structure where the estimated parameters can be partitioned into a linear and a nonlinear part. In this brief, we propose an efficient first-order recursive algorithm for SNLMs by introducing the variable projection (VP) step. The proposed algorithm utilizes the recursive least-squares method to eliminate the linear parameters, resulting in a reduced function. Then, the stochastic gradient descent (SGD) algorithm is employed to update the parameters of the reduced function. By considering the tight coupling relationship between linear parameters and nonlinear parameters, the proposed first-order VP algorithm is more efficient and robust than the traditional SGD algorithm and alternating optimization algorithm. More importantly, since the proposed algorithm just uses the first-order information, it is easier to apply it to large-scale models. Numerical results on examples of different sizes confirm the effectiveness and efficiency of the proposed algorithm.

Identification of nonlinear dynamical system based on adaptive radial basis function neural networks

Identification of nonlinear dynamical system based on adaptive radial basis function neural networks

Validating a data-driven framework for vehicular traffic modeling

This study presents a data-driven framework for modeling complex systems, with a specific emphasis on traffic modeling. Traditional methods in traffic modeling often rely on assumptions regarding vehicle interactions. Our approach comprises two steps: first, utilizing information- theoretic (IT) tools to identify interaction directions and candidate variables thus eliminating assumptions, and second, employing the sparse identification of nonlinear systems (SINDy) tool to establish functional relationships. We validate the framework’s efficacy using synthetic data from two distinct traffic models, while considering measurement noise. Results show that IT tools can reliably detect directions of interaction as well as instances of no interaction. SINDy proves instrumental in creating precise functional relationships and determining coefficients in tested models. The innovation of our framework lies in its ability to use data-driven approach to model traffic dynamics without relying on assumptions, thus offering applications in various complex systems beyond traffic.

Towards optimizing canal system operations: Extremum-seeking controller design via a frequency-based control approach

This paper addresses strategies for enhancing the operational management of canal systems, aiming to reduce water losses resulting from overflows that arise due to inadequate water level control during canal operations. The study focuses on nonlinear modeling and system identification of a canal system, presenting a detailed analysis, design, and practical implementation of a frequency-based compensator. The mathematical model of the experimental canal system is derived through real-time experiments. The operational management of this experimental canal system is achieved using a compensator structure based on Bernstein polynomials. The proposed compensator parameters are refined using the extremum-seeking control algorithm, which is employed for optimizing a frequency-based cost function. The efficiency and performance of the proposed compensator compared to the cases where classical PI and two-degree-of-freedom PI with feed-forward controllers are used demonstrated through time domain and harmonic analysis plots related to the water level dynamics.

Identification of High-Order Nonlinear Coupled Systems Using a Data-Driven Approach

Most works related to the identification of mathematical nonlinear systems suggest that such approaches can always be directly applied to any nonlinear system. This misconception is greatly discouraging when the obtained results are not expected. Thus, the current work hypothesizes that the more information one has about the mathematical structure of the model, the most precise the identification result. Therefore, a variant of the Sparse Identification of Nonlinear Dynamics (SINDY) approach is presented to obtain the full mathematical nonlinear model of a high-order system with coupled dynamics, namely, a commercial quadcopter. Furthermore, due to its high sensitivity to inputs, a control system is devised using the identified model to stabilize the quadcopter. This illustrates the effectiveness of the proposed identification method.

State Observer-Based Conditioned Reverse-Path Method for Nonlinear System Identification

In light of the complex behavior of vibrating structures, their reliable modeling plays a crucial role in the analysis and system design for vibration control. In this paper, the reverse-path (RP) method is revisited, further developed, and applied to modeling a nonlinear system, particularly with respect to the identification of the frequency response function for a nominal underlying linear system and the determination of the structural nonlinearities. The present approach aims to overcome the requirement for measuring all nonlinear system states all the time during operation. Especially in large-scale systems, this might be a tedious task and often practically infeasible since it would require having individual sensors assigned for each state involved in the design process. In addition, the proper placement and simultaneous operation of a large number of transducers would represent further difficulty. To overcome those issues, we have proposed state estimation in light of the observability criteria, which significantly reduces the number of required sensor elements. To this end, relying on the optimal sensor placement problem, the state estimation process reduces to the solution of Kalman filtering. On this ground, the problem of nonlinear system identification for large-scale systems can be addressed using the observer-based conditioned RP method (OBCRP) proposed in this paper. In contrast to the classical RP method, the current one can potentially handle local and distributed nonlinearities. Moreover, in addition to the state estimation and in comparison to the orthogonal RP method, a new frequency-dependent weighting is introduced in this paper, which results in superior nonlinear system identification performances. Implementation of the method is demonstrated on a multi-degree-of-freedom discretized lumped mass system, representing a substitute model of a physical counterpart used for the identification of the model parameters.

Study of Various Neural Network Architecture for Nonlinear System Identification

Study of Various Neural Network Architecture for Nonlinear System Identification

Likelihood-based generalization of Markov parameter estimation and multiple shooting objectives in system identification

This paper considers system identification (ID) of linear and nonlinear non-autonomous systems from noisy and sparse data. We analyze an optimization objective derived from Bayesian inference for the dynamics of hidden Markov models. We then relate this objective to that used in several state-of-the-art approaches for both linear and nonlinear system ID. In the former, we analyze least squares approaches for Markov parameter estimation, and in the latter, we analyze the multiple shooting approach. We demonstrate the limitations of the optimization problems posed by these existing methods by showing that they can be seen as special cases of the proposed optimization objective under certain simplifying assumptions: conditional independence of data and zero model error. Furthermore, we observe that the proposed approach has improved smoothness and inherent regularization that make it well-suited for system ID and provide mathematical explanations for these characteristics’ origins. Finally, numerical simulations demonstrate a mean squared error over 8.7 times lower compared to multiple shooting when data are noisy and/or sparse. Moreover, the proposed approach identifies accurate and generalizable models even when there are more parameters than data or when the system exhibits chaotic behavior.

A Grey-Box Model of a DC/DC Boost Converter for PV Energy Systems

This paper presents a grey-box model of a DC/DC boost converter for PV energy systems. The proposed model contains a white-box model part and a black-box model part together to prepare a better model for the PV boost converter. The white-box model part is used for knowledge of the circuit by mathematical equations since the black-box model part is used for unknown parameters such as temperature and electromagnetic interference. The black-box part of the proposed model is created by a nonlinear system identification of a real boost converter circuit with an artificial neural network. The precision of the mathematical model and the advantages of the fast prediction ability of the artificial neural network were used together. The proposed grey-box model is compared with the existing state-space and black-box models and experimental results. The results of the study showed that the average correlation between the proposed grey-box model output and the experimental results is 97.52%. Therefore, the proposed model can be used for analyzing DC/DC boost converter output characteristics before field applications.

Data-Driven Bayesian Inference for Stochastic Model Identification of Nonlinear Aeroelastic Systems

The objective of this work is to propose a data-driven Bayesian inference framework to efficiently identify parameters and select models of nonlinear aeroelastic systems. The framework consists of the use of Bayesian theory together with advanced kriging surrogate models to effectively represent the limit cycle oscillation response of nonlinear aeroelastic systems. Three types of sampling methods, namely, Markov chain Monte Carlo, transitional Markov chain Monte Carlo, and the sequential Monte Carlo sampler, are implemented into Bayesian model updating. The framework has been demonstrated using a nonlinear wing flutter test rig. It is modeled by a two-degree-of-freedom aeroelastic system and solved by the harmonic balance methods. The experimental data of the flutter wing is obtained using control-based continuation techniques. The proposed methodology provided up to a 20% improvement in accuracy compared to conventional deterministic methods and significantly increased computational efficiency in the updating and uncertainty quantification processes. Transitional Markov chain Monte Carlo was identified as the optimal choice of sampling method for stochastic model identification. In selecting alternative nonlinear models, multimodal solutions were identified that provided a closer representation of the physical behavior of the complex aeroelastic system than a single solution.

Z-number based neural network structured inference system

Z-number based Neural Network structured Inference System (ZNIS) with rule-base consisting of linguistic Z-terms trainable with Differential Evolution with Constraints (DEC) optimization algorithm is suggested. The inference mechanism of the multi-layered ZNIS consists of a fuzzifier, fuzzy rule base, inference engine, and output processor. Due to the use of extended fuzzy terms, each processing layer implements appropriate extended fuzzy operations, including computation of fuzzy valued rule firing strengths, fuzzy Level-2 aggregate outputs, and two consecutive Center of Gravity (COG) defuzzification procedures. The experiments with different versions of ZNIS have demonstrated that it is a universal modeling tool suitable for dealing with both approximate reasoning and functional mapping tasks. Random experiments on benchmark examples (among which are simple functional mapping, Parkinson disease, and non-linear system identification) have shown that ZNIS performance is equivalent to or better than FLS Type 2 and far superior to FLS Type 1, showing on average 2–3 times lower MSE. Along with this, the main advantages of ZNIS over other inference systems are better semantic expressing power, higher degree of perception and interpretability of the linguistic rules by humans, and a higher confidence in the reliability of achieved decision due to the transparency of the underlying decision-making mechanism.

Correction to: Nonlinear system identification in coherence with nonlinearity measure for dynamic physical systems—case studies

Correction to: Nonlinear system identification in coherence with nonlinearity measure for dynamic physical systems—case studies

Recurrent context layered radial basis function neural network for the identification of nonlinear dynamical systems

This paper proposes a novel recurrent context layered radial basis function neural network (RCLRBFNN) for the identification of nonlinear dynamical systems. The proposed model consists of an additional context layer in which the nodes represent the unit-delayed outputs of the hidden layer radial centers. These delayed outputs undergo a nonlinear transformation by applying a tangent hyperbolic function. These transformed signals connect to the output layer neuron through the adjustable context layered weights. To tune the parameters of the proposed model the update equations are derived using the dynamic back-propagation algorithm. Further, an adaptive learning rate scheme is proposed to improve the performance of the learning algorithm. In the simulation experiment, a total of two examples are considered to test the efficacy and performance of the proposed model. The performance comparison is made with the conventional structure of the radial basis function neural network (RBFNN), Jordan Recurrent neural network (JRNN), and the feed-forward neural network (FFNN) (which is nothing but a single-layered multi-layered perceptron). Both the disturbance signal as well as system’s uncertainty scenarios are considered to test the robustness shown by the proposed model. The results showed that the proposed model has delivered a better identification accuracy as compared to the other neural models.

A data-driven implicit deep adaptive neuro-fuzzy inference system capable of manifold learning for function approximation

Fuzzy Neural Networks (FNN) have the ability of decision-making based on constructing semi-ellipsoidal clusters in the input space as the antecedent parts of their fuzzy rules. To determine the output value for each input instance, FNNs consider its membership degree to different sub-regions of the input space. However, forming such meaningful sub-regions is not possible in all applications due to the nonlinear interactions among input variables and their low information gain. Indeed, the samples could be distributed on a manifold in the input space. Therefore, to cover the input space, we need lots of rules, each representing a small region of input space. This issue decreases the generalization ability of the model along with its explainability. Consequently, to efficiently form fuzzy rules, first, it is necessary to unfold the manifold by mapping the samples to an appropriate embedding space. Next, the fuzzy rules in the form of semi-ellipsoidal regions should be constructed in this extracted feature space. Deep Fuzzy Neural Networks address this problem by representation learning through stacking multiple cascade mapping layers. In this paper, we propose a novel approach for nonlinear function approximation and time-series prediction problems, based on using the kernel trick to implicitly learn the mapping function to the new feature space. Moreover, to initialize the fuzzy rules, a KNN-based method using the kernel trick is proposed. A hierarchical Levenberg–Marquardt approach is applied to learn the model’s parameters. The performance and structure of the proposed method are studied and compared with some other relevant methods in synthetic and real-world benchmarks. Based on these experiments, the proposed method has the best performance with the most parsimonious architecture. According to these experiments, the test RMSE of the proposed method is 0.002 for Mc-Glass chaotic time-series prediction, 0.015 for a Nonlinear dynamic system identification, 0.0345 for Box–Jenkins nonlinear system identification, 0.0609 for Fuel consumption prediction of automobiles, 10.24 for Sydney stock price tracking, and 0.595 for California housing.

Application of nonlinear system identification and feature extraction in fault diagnosis of the cracked cantilever beam

The cantilever beam structure is extensively applied in civil, transportation, and mechanical fields. It is generally prone to structural damage like cracks arising from material defects, operation conditions, and environmental factors. Cracks in a cantilever beam generally induce nonlinear behavior, potentially compromising structural durability and even posing threats to production safety. The present study primarily focuses on online and quantitative fault diagnosis of cracks in the cantilever beam. A simulation model is established for the cantilever beam with varying cracks, where responses corresponding to excitation signals are collected. A Nonlinear Autoregressive with Exogenous inputs (NARX) model for the cantilever beam is identified, and Nonlinear Output Frequency Response Functions weighted contribution rate (WNOFRFs) are evaluated from the established NARX model. A WNOFRFs-associated index is extracted to quantify the severity of breathing cracks. Simulation results indicate that this approach can online diagnose and quantify the severity of crack faults to a certain extent. The applicability of this approach is further verified through experiments on a cracked cantilever beam.

Nonlinear system identification using modified variational autoencoders

This research proposes a methodology for identifying nonlinear systems using input/output data and deep learning generative models. Our framework integrates Variational Autoencoders (VAE) with Nonlinear Autoregressive with exogenous input (NARX) in a unified identification structure to address overfitting in nonlinear system identification using NARX structures. Specifically, we modify a variational autoencoder by replacing the decoder module with a NARX model using the latent space information captured from the VAE encoder module as one of the exogenous inputs. Following the training phase, the decoder module can be used as a nonlinear model of the system. We evaluate the efficacy of our approach by performing open-loop prediction tests on data from four nonlinear benchmark systems: Cascaded tanks, Gas furnace, Silverbox, and Wiener-Hammerstein. The proposed VAE-NARX method reported Root Mean Squared Error (RMSE) of 8.23×10−3, 16.69×10−3, 0.002×10−3 and 0.037×10−3 respectively. Our results demonstrate that our proposed method achieves similar and outperforms prediction performances to standard identification techniques and can enhance the performance of traditional nonlinear system identification methods based on multi-layer perceptron models.

Identification of nonlinear system with time delay based on wavelet packet decomposition and Gaussian kernel GMDH network

This article addresses the identification problem of the input nonlinear closed-loop system with unknown time delay (INCTD). An identification algorithm of the INCTD system which combines the wavelet packet decomposition (WPD), the variance Gaussian kernel function (VGKF) and the grouped method of data handling (GMDH) network is proposed. The data collected by the sensor is decomposed into intrinsic mode function with different nonlinear characteristics using WPD. The GMDH network is applied to mine bidirectional strong nonlinear relationship between input and output data. The VGKF criterion selects the optimal complexity model to enhance the extrapolation performance, and further improves the identification accuracy. Thus, the WPD-GMDH-VGKF identification algorithm is derived. The numerical simulation result verifies that the WPD-GMDH-VGKF identification method can effectively identify the INCTD system, and it is superior to GMDH and WPD-GMDH in terms of identification accuracy. The application case verifies the feasibility of applying the WPD-GMDH-VGKF identification method to the battery management system (BMS).

A kernel framework for learning differential equations and their solution operators

This article presents a three-step kernel framework for regression of the functional form of differential equations (DEs) and learning their solution operators. Given a training set consisting of pairs of noisy DE solutions and source/boundary terms on a mesh: (i) kernel smoothing is utilized to denoise the data and approximate derivatives of the solution; (ii) This information is then used in a kernel regression model to learn the functional form of the DE; (iii) The learned DE is then used within a numerical solver to approximate the solution of the DE with a new source term or initial data, thereby constituting an operator learning framework. Numerical experiments compare the method to state-of-the-art algorithms. In DE learning our framework matches the performance of Sparse Identification of nonlinear Dynamical Systems (SINDy) while in operator learning the method has superior performance compared to well-established neural network methods in low training data regimes.

Nonlinear system identification in coherence with nonlinearity measure for dynamic physical systems—case studies

Nonlinear system identification in coherence with nonlinearity measure for dynamic physical systems—case studies

Data-driven identification for nonlinear dynamic systems

Data-driven identification for nonlinear dynamic systems