Nonlinear System Identification Research Articles

Design of experiments for nonlinear system identification: A set membership approach

Design of Experiments (DoE) is an important step in system identification. Regardless of the chosen model structure and identification method, the DoE quality determines an upper bound on the accuracy of the identified models. One of the greatest challenges in this context is to design an experiment which gives the maximum information about the dynamics of the system of interest. In this paper, a novel DoE algorithm for input-constrained MISO nonlinear systems, based on set membership identification, is proposed. The DoE algorithm is aimed to minimize the so-called radius of information, a quantity giving the worst-case model error. Two numerical examples are presented, showing the effectiveness of the approach and its potential in view of real-world applications.

Identification and parameter estimation of cubic nonlinear damping using harmonic probing and volterra series

Dynamic systems, such as vibration isolators, rotor-bearing systems, are inherently nonlinear and their dynamic behaviour often cannot be sufficiently explained or predicted by simple linear models. Presence of nonlinearity leads to certain characteristic behaviours in the response such as jump phenomenon, limit cycle, super-harmonic resonances and such behaviours can be accurately predicted only if the nonlinearity structure and related parameters are properly known. This emphasises the recently growing importance of nonlinear system identification. A majority of the identification works is based on a-priori knowledge of nonlinearity structure and most of them consider only stiffness nonlinearities, such as Duffing’s oscillator and bilinear oscillator. Not much work has been reported on nonlinearity structure identification for systems with damping nonlinearities. This paper, first discusses a systematic classification of nonlinearity structures based on first, second and third harmonic response amplitudes under harmonic excitation. Characteristics response for individual nonlinearity class is explained by Volterra series response formulation with higher order Frequency Response Functions. In the second part, a typical cubic damping nonlinearity is identified from cubic stiffness nonlinearity and an algorithm for estimating the nonlinear and linear damping parameters is developed. A new term called measurability ratio is introduced to show how it can help in deciding the most appropriate excitation frequency. Effect of truncating the Volterra series response on parameter estimation error is also studied for different excitation frequencies and varying excitation levels. It is shown that, with recursive iteration in computation of third harmonic amplitude, estimation accuracy can be further improved.

PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data

Scientists have long quantified empirical observations by developing mathematical models that characterize the observations, have some measure of interpretability, and are capable of making predictions. Dynamical systems models in particular have been widely used to study, explain, and predict system behavior in a wide range of application areas, with examples ranging from Newton’s laws of classical mechanics to the Michaelis-Menten kinetics for modeling enzyme kinetics. While governing laws and equations were traditionally derived by hand, the current growth of available measurement data and resulting emphasis on data-driven modeling motivates algorithmic approaches for model discovery. A number of such approaches have been developed in recent years and have generated widespread interest, including Eureqa (Schmidt & Lipson, 2009), sure independence screening and sparsifying operator (Ouyang, Curtarolo, Ahmetcik, Scheffler, & Ghiringhelli, 2018), and the sparse identification of nonlinear dynamics (SINDy) (Brunton, Proctor, & Kutz, 2016). Maximizing the impact of these model discovery methods requires tools to make them widely accessible to scientists across domains and at various levels of mathematical expertise.

Uncertain nonlinear system identification using Jaya-based adaptive neural network

Uncertain nonlinear system identification using Jaya-based adaptive neural network

NNARX Networks on Didactic Level System Identification

This work has as main objective to propose the identification of a small scale non-linear system through the Neural Network AutoRegressive with eXternal input. The use of this network requires an adequate methodology for its configuration and, consequently, a good training set. Then, it is proposed that the main definitions of the network parameters be obtained through the analysis of nonintrusive performance indices. Additionally, using a database based on the system’s response, excited by the Pseudo-Random Binary Sequence signal. The methodology will be applied in two specific open-loop identification situations: numerical simulation of a fourth order polynomial system (Case 01), and an experimental system that controls a nonlinear water tank level (Case 02). The results of the identified models were able to represent the system dynamics with high fidelity, presenting an average identification error of less than 0:14 and 0:34% for Case 1 and 2, respectively. Also, it is observed that the learning and generalization evidence could represent the process intrinsic nonlinearities satisfactorily. Besides, it will be possible to find the potentiality and usefulness of the developed network in nonlinear system identification.

Design of quaternion-valued second-order Volterra adaptive filters for nonlinear 3-D and 4-D signals

In this paper, the quaternion-valued second-order Volterra adaptive filters (QSOVAFs) are designed for the processing of nonlinear three-dimensional (3-D) and four-dimensional (4-D) signals. In the proposed frameworks, the structure of the strictly nonlinear (SNL), semi-widely nonlinear (SWNL), and widely nonlinear (WNL) QSOVAFs are primarily constructed in the quaternion domain. Then, their loss functions defined by the instantaneous error signals are minimized in the quaternion domain by using the recent generalized Hamilton-real (GHR) calculus. Thus, novel weight update equations are obtained for training the proposed SNL-QSOVAF, SWNL-QSOVAF, and WNL-QSOVAF. Furthermore, the stability bounds for each quaternion-valued kernel functions of them are derived from the convergence in the mean analysis. The comprehensive simulations on the nonlinear system identification and one-step-ahead prediction experiments support that the proposed SWNL-QSOVAF and WNL-QSOVAF can be effectively used in the processing of nonlinear noncircular quaternion signals, whereas that their SNL version can produce optimal results for nonlinear circular quaternion signals.

Fast and improved backpropagation learning of multi‐layer artificial neural network using adaptive activation function

AbstractIn the conventional backpropagation (BP) learning algorithm used for the training of the connecting weights of the artificial neural network (ANN), a fixed slope−based sigmoidal activation function is used. This limitation leads to slower training of the network because only the weights of different layers are adjusted using the conventional BP algorithm. To accelerate the rate of convergence during the training phase of the ANN, in addition to updates of weights, the slope of the sigmoid function associated with artificial neuron can also be adjusted by using a newly developed learning rule. To achieve this objective, in this paper, new BP learning rules for slope adjustment of the activation function associated with the neurons have been derived. The combined rules both for connecting weights and slopes of sigmoid functions are then applied to the ANN structure to achieve faster training. In addition, two benchmark problems: classification and nonlinear system identification are solved using the trained ANN. The results of simulation‐based experiments demonstrate that, in general, the proposed new BP learning rules for slope and weight adjustments of ANN provide superior convergence performance during the training phase as well as improved performance in terms of root mean square error and mean absolute deviation for classification and nonlinear system identification problems.

On Treed Gaussian Processes and piecewise-linear NARX modelling

In the scope of nonlinear system identification, traditional parametric models are widely adopted as simplifying approaches to modelling the complexity of nonlinearity. However, many high-order parametric models are disadvantaged due to their inherent demand for model detection and their tendency to overfit in the absence of additional validation processes. Nonparametric models, such as the Gaussian Process (GP), though being naturally exempt from model detection, can involve expensive procedures of model optimisation. This article presents a Linear Kernel Chipman-based Treed Gaussian Processes (LK-CTGP), which is essentially an assembly of simple linear parametric models using a decision tree framework, to model nonlinear systems. The piecewise-linear structure of the LK-CTGP offers a natural geometric solution to modelling nonlinear systems, where no model detection is required. The essence of simplicity from the traditional parametric model is also completely retained within each of the submodels. The effectiveness of the LK-CTGP is illustrated here via a number of case studies from simple synthetic data to experimental data, on which Nonlinear Autoregressive eXogenous (NARX) systems will built from the data for in-depth study.

A new multimodel approach by Laguerre filters on sliding window for nonlinear system identification and control

This paper proposes an online identification procedure on sliding window with the synthesis of a nonlinear adaptive predictive control for a new nonlinear model resulting in multimodel approach. Such a model is entitled ARX-Laguerre multimodel obtained by expanding the conventional ARX multimodel on independent Laguerre orthonormal bases. It allows a significant parameter number reduction as well as a simple recursive representation compared with the ARX-multimodel. This parametric reduction is provided from an optimal iteratif identification approach of the Laguerre poles presented in Adaily et al. (2013). We propose to combine and carry out this identification approach on a sliding window to achieve an online identification procedure of the ARX-Laguerre multimodel for real time procedure depending on Fourier coefficients that are identified by a regularized square error. This property allows to synthese a new nonlinear adaptive predictive control on sliding window. We develop the general form of a new predictor and so, we propose an optimization algorithm formulated as a quadratic programming (QP) under linear constraints for an adaptive control law. The performances of the proposed online identification procedure and the developed nonlinear adaptive control algorithm are illustrated on a benchmark system as the continuous stirred tank reactor system (CSTR) with respect to the process parameter uncertainties.

Intelligent early structural health prognosis with nonlinear system identification for RFID signal analysis

The mechanical state of non-linear processes includes these characteristics such as multivariable, strong coupling, multiple vibration sources, large signal noise, and various random factors. The intrinsic relationship and overall consistency optimization of the characteristic factor direction paths of multi-source matrix signals are a new research hotspot in multi-source dynamic characteristic signal identification. A intelligent fault diagnosis and prognosis method based on NARMAX-FRF and PCA is proposed in this paper. This method can be widely used in industrial system fault diagnosis. This system solves many basic key problems, including identifying a non-linear model from a detected system, accurately solving the frequency response function, extracting a representative frequency domain from the frequency response function, and applying extracted system frequency domain features for large-scale structural health assessment. In order to verify the performance of the NARMAX_FRF and PCA method for nonlinear defect signal analysis, the experiment of intelligent RFID system of corrosion monitoring and the TOFD experimental system are analyzed for the structural health monitoring in this paper. The set of samples in the experiment of intelligent RFID system of corrosion monitoring consists of coated and uncoated mild steel plates, which have a patch that has been exposed to the environment for different durations to create different levels of corrosion. The results show the effectiveness and robustness of the proposed method.

Simplified Analysis for Multiple Input Systems: A Toolbox Study Illustrated on F-16 Measurements

This paper introduces a nonparametric, nonlinear system identification toolbox called SAMI (simplified analysis for multiple input systems) developed for industrial measurements of vibro-acoustic systems with multiple inputs. It addresses the questions related to the user-friendly (semi-)automatic processing of multiple-input, multiple-output measurements with respect to the design of an experiment and the analysis of the measured data. When the proposed toolbox is used, with minimal user interaction, it is easily possible (a) to decide whether the underlying system is linear or not, (b) to decide whether the linear framework is still adequate to be used, and (c) to tell an inexperienced user how much can be gained using an advanced nonlinear framework. The toolbox is illustrated on openly accessible F-16 ground vibration testing measurements.

Heuristic Order Reduction of NARX-OBF models Applied to Nonlinear System Identification

Nonlinear system identification concerns the determination of the modelstructure and its parameters. Although the designers often seek the bestmodel for each system, it can be tricky to determine, at the same time, thebest structure and the parameters which optimize the model performance.This paper proposes the use of a Genetic Algorithm, GA, and the Levenberg-Marquardt, LM, method to obtain the model parameters, as well asperform the order reduction of the model. In order to validate the proposedmethodology, the identification of a magnetic levitator, operating in closedloop, was performed. The class NARX-OBF, Nonlinear Auto Regressivewith eXogenous input-Orthonormal Basis Function, was used. The use ofOBF functions aims to reduce the number of terms in NARX models. Oncethe model is found, the order reduction is performed using GA and LM, ina hybrid application, capable of determining the model parameters and reducing the original model order, simultaneously. The results show, considering the inherent trade-of between accuracy and computational effort, theproposed methodology provided an implementation with good mean squareerror, when compared with the full NARX-OBF model.

Convergence Analysis of Adaptive Exponential Functional Link Network.

The adaptive exponential functional link network (AEFLN) is a recently introduced novel linear-in-the-parameters nonlinear filter and is used in numerous nonlinear applications, including system identification, active noise control, and echo cancellation. The improved modeling accuracy offered by AEFLN for different nonlinear applications can be attributed to the exponentially varying sinusoidal basis functions used for nonlinear expansion. Even though AEFLN has been widely used for the identification of nonlinear systems, no theoretical analysis of AEFLN is available in the literature. Hence, in this article, a theoretical performance analysis of AEFLN trained using an adaptive exponential least mean square (AELMS) algorithm under the Gaussian input assumption is discussed. Expressions describing the mean as well as mean square behavior of the weight vector and adaptive exponential parameter are derived. Computer simulations are carried out, and the derived theoretical expressions show a close correspondence with simulation results.

Nonlinear System Identification via Tensor Completion

Function approximation from input and output data pairs constitutes a fundamental problem in supervised learning. Deep neural networks are currently the most popular method for learning to mimic the input-output relationship of a general nonlinear system, as they have proven to be very effective in approximating complex highly nonlinear functions. In this work, we show that identifying a general nonlinear function y = ƒ(x1,…,xN) from input-output examples can be formulated as a tensor completion problem and under certain conditions provably correct nonlinear system identification is possible. Specifically, we model the interactions between the N input variables and the scalar output of a system by a single N-way tensor, and setup a weighted low-rank tensor completion problem with smoothness regularization which we tackle using a block coordinate descent algorithm. We extend our method to the multi-output setting and the case of partially observed data, which cannot be readily handled by neural networks. Finally, we demonstrate the effectiveness of the approach using several regression tasks including some standard benchmarks and a challenging student grade prediction task.

Model‐based method with nonlinear ultrasonic system identification for mechanical structural health assessment

AbstractConventional methods for the structural nondestructive testing (NDT) such as pulsed eddy current (PEC), ultrasonic detection (UT), and so on are based on the output signal interpretation technique of the NDT system to extract the features to assess the structural health. In this article, a novel NDT method for the structural health monitoring from the different perspective is proposed, which is based on the structural system model between the exciting input and output response. Nonlinear autoregressive moving average with exogenous inputs (NARMAX) is used to establish the time domain model between the input signal and the output response. Frequency response (FRF) of NARMAX model from the time domain data is proposed to obtain the indicators to assess the structural health. The effectiveness and advantage of the proposed method are tested with two sets of experimental data by comparison with the existing method.

Adaptive Fast Orthogonal Search (FOS) algorithm for forecasting streamflow

Data-driven models for streamflow forecasting have attracted considerable attention, as they are independent of physical system features. The physical features of the river basin are extremely hard to collect, especially for large rivers. Empirical data-driven models, such as stochastic and regression models, have been widely used in the field of streamflow forecasting. However, they suffered limited accuracy in predicting extreme streamflow. They also required raw data pre-processing prior to the modeling process, especially for lengthy data records and for large time-scale increments (e.g. monthly resolution). To overcome these challenges, data-driven forecasting models based on Artificial Intelligence (AI) have been widely used and resulted in enhancing the forecasting accuracy. Nevertheless, AI-based models required augmentation with proper optimization schemes to adjust the model parameters for optimal accuracy. Furthermore, in some cases, due to unsuitability of the optimization model, there is high possibility for overfitting of the AI model, which might cause poor prediction of input patterns that were not adequately mimicked. This study introduces a new approach to streamflow forecasting based on nonlinear system identification. The proposed technique employs Fast Orthogonal Search (FOS) to develop a nonlinear model of stream flow. The main advantage of using FOS is eliminating the requirement of raw data pre-processing and the need for an optimization scheme for model parameter adjustment since the FOS algorithm takes this into account while building the model. In addition, the FOS algorithm includes a pole-zero cancellation procedure that can detect and avoid the over-fitted models. The FOS-based nonlinear modeling approach was adopted in this research for the development of a streamflow forecasting model at Aswan High Dam using monthly basis natural streamflow records for 130 years. The results indicated that the proposed FOS algorithm outperformed the previously developed AI models of streamflow forecasting for large data records and for large time-scale increment (monthly resolution).

Nonlinear System Identification Using Exact and Approximate Improved Adaptive Exponential Functional Link Networks

Adaptive exponential functional link network (AEFLN) is a recently introduced linear-in-the-parameters nonlinear filter. In an attempt to improve the performance of AEFLN, an improved AEFLN (IAEFLN) which employs independent decay rates for each exponentially varying sinusoidal basis function, has been proposed in this brief. The update rules for the filter weights as well as the decay parameter vector has been derived. To further reduce the computational complexity of the proposed network, without sacrificing performance, two approximate versions of IAEFLN, namely approximate 1 IAEFLN (Apx1-IAEFLN) and approximate 2 IAEFLN (Apx2-IAEFLN) has been developed and their corresponding update rules have been derived. Bounds on learning rates have also been derived and simulation study shows improved convergence behaviour of the proposed IAEFLN over AEFLN. The approximate versions achieve similar convergence performance as that of IAEFLN, at a lower computational load.

Experimental assessment of polynomial nonlinear state-space and nonlinear-mode models for near-resonant vibrations

In the present paper, two existing nonlinear system identification methodologies are used to identify data-driven models. The first methodology focuses on identifying the system using steady-state excitations. To accomplish this, a phase-locked loop controller is implemented to acquire periodic oscillations near resonance and construct a nonlinear-mode model. This model is based on amplitude-dependent modal properties, i.e. does not require nonlinear basis functions. The second methodology exploits uncontrolled experiments with broadband random inputs to build polynomial nonlinear state-space models using advanced system identification tools. The methods are applied to two experimental test rigs, a magnetic cantilever beam and a free-free beam with a lap joint. The respective models obtained by either method for both specimens are then challenged to predict dynamic, near-resonant behavior observed under different sine and sine-sweep excitations. The vibration prediction of the nonlinear-mode and state-space models clearly highlight capabilities and limitations. The nonlinear-mode model, by design, yields a perfect match at resonance peaks and high accuracy in close vicinity. However, it is limited to well-spaced modes and sinusoidal excitation. The state-space model covers a wider dynamic range, including transient excitations. However, the real-life nonlinearities considered in this study can only be approximated by polynomial basis functions. Consequently, the identified state-space models are found to be highly input-dependent, in particular for sinusoidal excitations where they are found to lead to a low predictive capability.

Efficient hinging hyperplanes neural network and its application in nonlinear system identification

In this paper, the efficient hinging hyperplanes (EHH) neural network is proposed, which is basically a single hidden layer neural network. Different from the dominant single hidden layer neural networks, the hidden layer in the EHH neural network can be seen as a directed acyclic graph (DAG) and all the nodes in the DAG contribute to the output. It is proved that for every EHH neural network, there is an equivalent adaptive hinging hyperplanes (AHH) tree, the model of which was proposed based on the hinging hyperplanes (HH) model and finds good applications in system identification. Analog to the proof for the AHH model, the universal approximation ability of the EHH neural network is provided. Different from other neural networks, the EHH neural network has interpretability, which can be easily obtained through its ANOVA decomposition (or interaction matrix). The interpretability can then be used as an indication for the importance of the input variables. The construction of the EHH neural network includes initial network generation and parameter optimization (including the structure and weights parameter optimization). A descent algorithm for searching the locally optimal EHH neural network is proposed and the worst-case complexity of the algorithm is also provided. The EHH neural network is applied in nonlinear system identification, the simulation results show that satisfactory accuracy can be achieved with relatively low computational cost, and at the same time, some insights into the importance of the regressors and the interactions among the regressors can be revealed.

Nonlinear system modeling and application based on restricted Boltzmann machine and improved BP neural network

Aiming at the complexity, nonlinearity and difficulty in modeling of nonlinear system. In this paper, an improved back-propagation(BP) neural network based on restricted boltzmann machine(RBM-IBPNN) is proposed for nonlinear systems modeling. First, the structure of BP neural network(BPNN) is optimized by using sensitivity analysis(SA) and mutual information(MI) of the hidden neurons. Namely when the SA value and the MI value of the hidden neurons satisfy the set standard, the corresponding neurons will be pruned, split or merged. second, the restricted boltzmann machine(RBM) is employed to perform parameters initialization of training on the IBPNN. Finally, the proposed RBM-IBPNN is evaluated on nonlinear system identification, lorenz chaotic time series prediction and the total phosphorus prediction problems. The experimental results demonstrate that the proposed RBM-IBPNN not only has faster convergence speed and higher prediction accuracy, but also realizes a more compact network structure.