Nonlinear System Identification Problem Research Articles

State Observer-Based Conditioned Reverse-Path Method for Nonlinear System Identification

In light of the complex behavior of vibrating structures, their reliable modeling plays a crucial role in the analysis and system design for vibration control. In this paper, the reverse-path (RP) method is revisited, further developed, and applied to modeling a nonlinear system, particularly with respect to the identification of the frequency response function for a nominal underlying linear system and the determination of the structural nonlinearities. The present approach aims to overcome the requirement for measuring all nonlinear system states all the time during operation. Especially in large-scale systems, this might be a tedious task and often practically infeasible since it would require having individual sensors assigned for each state involved in the design process. In addition, the proper placement and simultaneous operation of a large number of transducers would represent further difficulty. To overcome those issues, we have proposed state estimation in light of the observability criteria, which significantly reduces the number of required sensor elements. To this end, relying on the optimal sensor placement problem, the state estimation process reduces to the solution of Kalman filtering. On this ground, the problem of nonlinear system identification for large-scale systems can be addressed using the observer-based conditioned RP method (OBCRP) proposed in this paper. In contrast to the classical RP method, the current one can potentially handle local and distributed nonlinearities. Moreover, in addition to the state estimation and in comparison to the orthogonal RP method, a new frequency-dependent weighting is introduced in this paper, which results in superior nonlinear system identification performances. Implementation of the method is demonstrated on a multi-degree-of-freedom discretized lumped mass system, representing a substitute model of a physical counterpart used for the identification of the model parameters.

Identification of nonlinear system with time delay based on wavelet packet decomposition and Gaussian kernel GMDH network

This article addresses the identification problem of the input nonlinear closed-loop system with unknown time delay (INCTD). An identification algorithm of the INCTD system which combines the wavelet packet decomposition (WPD), the variance Gaussian kernel function (VGKF) and the grouped method of data handling (GMDH) network is proposed. The data collected by the sensor is decomposed into intrinsic mode function with different nonlinear characteristics using WPD. The GMDH network is applied to mine bidirectional strong nonlinear relationship between input and output data. The VGKF criterion selects the optimal complexity model to enhance the extrapolation performance, and further improves the identification accuracy. Thus, the WPD-GMDH-VGKF identification algorithm is derived. The numerical simulation result verifies that the WPD-GMDH-VGKF identification method can effectively identify the INCTD system, and it is superior to GMDH and WPD-GMDH in terms of identification accuracy. The application case verifies the feasibility of applying the WPD-GMDH-VGKF identification method to the battery management system (BMS).

A New Method for Dynamical System Identification by Optimizing the Control Parameters of Legendre Multiwavelet Neural Network

Wavelet neural networks have been widely applied to dynamical system identification fields. The most difficult issue lies in selecting the optimal control parameters (the wavelet base type and corresponding resolution level) of the network structure. This paper utilizes the advantages of Legendre multiwavelet (LW) bases to construct a Legendre multiwavelet neural network (LWNN), whose simple structure consists of an input layer, hidden layer, and output layer. It is noted that the activation functions in the hidden layer are adopted as LW bases. This selection if based on the its rich properties of LW bases, such as piecewise polynomials, orthogonality, various regularities, and more. These properties contribute to making LWNNs more effective in approximating the complex characteristics exhibited by uncertainties, step, nonlinear, and ramp in the dynamical systems compared to traditional wavelet neural networks. Then, the number of selection LW bases and the corresponding resolution level are effectively optimized by the simple Genetic Algorithm, and the improved gradient descent algorithm is implemented to learn the weight coefficients of LWNN. Finally, four nonlinear dynamical system identification problems are applied to validate the efficiency and feasibility of the proposed LWNN-GA method. The experiment results indicate that the LWNN-GA method achieves better identification accuracy with a simpler network structure than other existing methods.

Auxiliary model‐based interval‐varying maximum likelihood estimation for nonlinear systems with missing data

AbstractThe identification problem of nonlinear system with missing data is focused in this article. In order to overcome the system unavailable outputs, an auxiliary model‐based interval‐varying recursive identification method is derived by changing the sampling interval and substituting the missing output with the output of an auxiliary model. Based on the maximum likelihood principle and the least‐squares method, a maximum likelihood‐based interval‐varying recursive least‐squares method is investigated. The validity of the proposed maximum likelihood method is tested by a numerical simulation example and a practical continuous stirred tank reactor (CSTR) process.

Training recurrent neural networks by sequential least squares and the alternating direction method of multipliers

This paper proposes a novel algorithm for training recurrent neural network models of nonlinear dynamical systems from an input/output training dataset. Arbitrary convex and twice-differentiable loss functions and regularization terms are handled by sequential least squares and either a line-search (LS) or a trust-region method of Levenberg–Marquardt (LM) type for ensuring convergence. In addition, to handle non-smooth regularization terms such as ℓ1, ℓ0, and group-Lasso regularizers, as well as to impose possibly non-convex constraints such as integer and mixed-integer constraints, we combine sequential least squares with the alternating direction method of multipliers (ADMM). We call the resulting algorithm NAILS (nonconvex ADMM iterations and least squares) in the case line search (LS) is used, or NAILM if a trust-region method (LM) is employed instead. The training method, which is also applicable to feedforward neural networks as a special case, is tested in three nonlinear system identification problems.

Parameter estimation of multiple‐input single‐output Hammerstein controlled autoregressive system based on improved adaptive moment estimation algorithm

AbstractIn this article, the problem of parameter estimation for a multiple‐input single‐output Hammerstein controlled autoregressive (MISO‐HCAR) model is solved. After establishing the identification model of MISO‐HCAR system, an improved adaptive moment estimation algorithm with decreasing weight (IADAM) is proposed. The improved algorithm transforms the nonlinear system identification problem into a parameter space optimization problem, according to the input and output data obtained from the system, and uses the parallel search ability of the IADAM algorithm to estimate all the parameters of the system simultaneously. Two numerical examples and the case study example of chemical continuously stirred tank reactor (CSTR) system show that the proposed IADAM algorithm with decreasing weight has better identification effect on the MISO‐HCAR model and CSTR system than the stochastic gradient descent (SGD) algorithm and the basic adaptive moment estimation (ADAM) algorithm, and the estimation accuracy and convergence speed are greatly improved. Thus, the IADAM can estimate the parameters of the MISO‐HCAR system and the CSTR system more effectively.

Interval type-2 fuzzy neural networks with asymmetric MFs based on the twice optimization algorithm for nonlinear system identification

This paper proposes a novel algorithm twice optimization for interval type-2 fuzzy neural networks with asymmetric membership functions (TOIT2FNN-AMF), for nonlinear system identification problems. The proposed TOIT2FNN-AMF uses an asymmetric Gaussian interval type-2 membership function to enhance the network’s ability to describe and solve nonlinear and uncertain problems. The twice optimization algorithm consists of structure learning and parameter learning. Firstly, this paper proposes a multi-strategy adaptive differential evolution (MSADE) algorithm as the first optimization algorithm, which is used to determine the structure and the initial values of the parameters of the TOIT2FNN-AMF. It applies the root mean square error (RMSE) of the TOIT2FNN-AMF as the fitness function to determine the structure (number of rules) and initial parameters of the IT2FNN by searching for the RMSE values under different structures. When the fitness value reaches the minimum, that is, the RMSE value of the TOIT2FNN-AMF, the corresponding number will become the optimal one of fuzzy rules of the TOIT2FNN-AMF. Then, the second optimization algorithm of the TOIT2FNN-AMF turns into a hybrid optimization algorithm composed of an adaptive moment estimation (Adam) algorithm and recursive least squares (RLS) algorithm. Adam is used to optimize the antecedent parameters of TOIT2FNN-AMF rules, so as to maintain rapid convergence without generating oscillation during the training process; RLS is used to optimize the consequent parameters of TOIT2FNN-AMF rules, so that the network parameters can be optimized rapidly. In this way, the problems of excessive parameters to be adjusted and excessive slow convergence of the network can be solved. Finally, this paper evaluates the proposed TOIT2FNN-AMF by testing on problems of nonlinear system identification and chaotic time-series prediction. The simulation results are compared with those of similar methods in the existing literatures, which demonstrates that the proposed TOIT2FNN-AMF model yields a lower RMSE value and a simpler network structure than the other type-2 fuzzy neural networks (T2FNNs).

Nonlinear System Identification Based on an Online SCFNN With Applications in IoTs

In this paper, an online self-constructing fuzzy neural network (SCFNN) is proposed to solve four kinds of nonlinear dynamic system identification (NDSI) problems in the internet of things (IoTs). The SCFNN is capable of constructing a simple network without the need for knowledge of the NDSI. Thus, carefully setting conditions for the increased demands for fuzzy rules will make the architecture of the constructed SCFNN fairly simple. The applications of neural networks in IoTs are discussed. The authors also propose a new identification model for NDSI. Through an experimental example, it is proved that online learning can arrange membership functions in a more appropriate vector space. The performance of the online SCFNN is compared with both MLP and RBF through four extensive simulations. The comparison terms are convergence rate, training root mean square error (RMSE), test RMSE, and prediction accuracy (PA). The simulation results show that SCFNN is superior to MLP and RBF in NDSI problems.

Nonlinear Dynamic System Identification of ARX Model for Speech Signal Identification

System Identification becomes very crucial in the field of nonlinear and dynamic systems or practical systems. As most practical systems don’t have prior information about the system behaviour thus, mathematical modelling is required. The authors have proposed a stacked Bidirectional Long-Short Term Memory (Bi-LSTM) model to handle the problem of nonlinear dynamic system identification in this paper. The proposed model has the ability of faster learning and accurate modelling as it can be trained in both forward and backward directions. The main advantage of Bi-LSTM over other algorithms is that it processes inputs in two ways: one from the past to the future, and the other from the future to the past. In this proposed model a backward-running Long-Short Term Memory (LSTM) can store information from the future along with application of two hidden states together allows for storing information from the past and future at any moment in time. The proposed model is tested with a recorded speech signal to prove its superiority with the performance being evaluated through Mean Square Error (MSE) and Root Means Square Error (RMSE). The RMSE and MSE performances obtained by the proposed model are found to be 0.0218 and 0.0162 respectively for 500 Epochs. The comparison of results and further analysis illustrates that the proposed model achieves better performance over other models and can obtain higher prediction accuracy along with faster convergence speed.

Novel Fractional Swarming with Key Term Separation for Input Nonlinear Control Autoregressive Systems

In recent decades, fractional order calculus has become an important mathematical tool for effectively solving complex problems through better modeling with the introduction of fractional differential/integral operators; fractional order swarming heuristics are also introduced and applied for better performance in different optimization tasks. This study investigates the nonlinear system identification problem of the input nonlinear control autoregressive (IN-CAR) model through the novel implementation of fractional order particle swarm optimization (FO-PSO) heuristics; further, the key term separation technique (KTST) is introduced in the FO-PSO to solve the over-parameterization issue involved in the parameter estimation of the IN-CAR model. The proposed KTST-based FO-PSO, i.e., KTST-FOPSO accurately estimates the parameters of an unknown IN-CAR system with robust performance in cases of different noise scenarios. The performance of the KTST-FOPSO is investigated exhaustively for different fractional orders as well as in comparison with the standard counterpart. The results of statistical indices through Monte Carlo simulations endorse the reliability and stability of the KTST-FOPSO for IN-CAR identification.

Design of fractional hierarchical gradient descent algorithm for parameter estimation of nonlinear control autoregressive systems

The trend of developing fractional gradient based iterative adaptive strategies is evolved in the recent years through effectively exploring the fractional and fractal dynamics. In this study, fractional hierarchical gradient descent (FHGD) is proposed by generalizing the standard hierarchical gradient descent (HGD) to fractional order for effectively solving nonlinear system identification problem. The FHGD is effectively to applied to estimate the parameters of nonlinear control autoregressive (NCAR) systems under different fractional order and noise conditions. The fractional order greater than 1 provides faster convergence speed, less than 1 gives better steady state performance and equal to 1 reduces the FHGD to HGD. The accurate estimation of NCAR system parameters representing electrically stimulated muscle model validates the efficacy and robustness of the proposed FHGD in comparison with the standard HGD.

M-estimate affine projection spline adaptive filtering algorithm: Analysis and implementation

This paper investigates the M-estimate affine projection spline adaptive filtering (MAPSAF) algorithm, which utilizes a modified Huber function with robustness against impulsive interference, and employs historical regression data to update the weight and the knot vector estimates for nonlinear filtering tasks. The detailed convergence and steady-state analyses of MAPSAF are also carried out in the mean and mean-square senses. In addition, an improved MAPSAF by exploiting the combined step sizes, called the CSS-MAPSAF algorithm, is derived to speed up the convergence on the premise of low steady-state misalignment. Numerical experiments in nonlinear system identification and nonlinear acoustic echo cancellation problems corroborate the theoretical performance analysis and the superiority of the proposed algorithms.

Zero-Attracting Kernel Maximum Versoria Criterion Algorithm for Nonlinear Sparse System Identification

Sparsity-induced kernel adaptive filters have emerged as a promising candidate for a nonlinear sparse system identification (SSI) problem. The existing zero-attracting kernel least mean square (ZA-KLMS) algorithm relies on minimum mean square error criterion, which considers only second order statistics of error, thereby resulting in suboptimal performance in the presence of non-Gaussian/impulsive distortions. In this letter, we propose a novel random Fourier features (RFF) based ZA kernel maximum Versoria criterion (ZA-KMVC) algorithm, and their variants, which are robust for nonlinear SSI in the presence of non-Gaussian distortions over both stationary and time-varying environments. Furthermore, the mean-square convergence analysis of the proposed RFF-ZA-KMVC algorithm is performed. It has been observed from the simulation results that the proposed algorithm delivers better convergence performance as compared to the existing state-of-art approaches.

Online Interval Type-2 Fuzzy Extreme Learning Machine applied to 3D path following for Remotely Operated Underwater Vehicles

In marine missions that involve 3D path following tasks, the overall goal of Underwater Vehicles (UVs) is the successful completion of a path previously specified by the operator. This implies that the path must be followed by the UV as closely as possible and arrive at a location for collection by a vessel. In this paper, an Online Interval Type-2 Fuzzy Extreme Learning Machine (OIT2-FELM) is suggested to achieve a robust following behaviour along a predefined 3D path using a Remotely Operated Underwater Vehicle (ROV). The proposed machine is a fast sequential learning scheme to the training of a more generalised model of TSK Interval Type-2 Fuzzy Inference Systems (TSK IT2 FISs) equivalent to Single Layer Feedforward Neural Networks (SLFNs). Learning new input data in the OIT2-FELM can be done one-by-one or chunk-by-chunk with a fixed or varying size. The OIT2-FELM is implemented in a hierarchical navigation strategy (HNS) as the main guidance mechanism to infer local control motions and to provide the ROV with the necessary autonomy to complete a predefined 3D path. For local path-planning, the OIT2-FELM performs signal classification for obstacle avoidance and target detection based on data collected by an on-board scan sonar. To evaluate the performance of the proposed OIT2-FELM, two different experiments are suggested. First, a number of benchmark problems in the field of non-linear system identification, regression and classification problems are used. Secondly, a number of experiments to the completion of a predefined 3D path using an ROV is implemented. Compared to other fuzzy strategies, the OIT2-FELM offered two significant capabilities. On the one hand, the OIT2-FELM provides a better treatment of uncertainty and noisy signals in underwater environments while improving the ROV’s performance. Secondly, online learning in OIT2-FELM allows continuous knowledge discovery from survey data to infer the surroundings of the ROV. Experiment results to the completion of 3D paths show the effectiveness of the proposed approach to handle uncertainty and produce reasonable classification predictions (∼90.5% accuracy in testing data).

Auxiliary Model-Based Multi-Innovation Fractional Stochastic Gradient Algorithm for Hammerstein Output-Error Systems

This paper focuses on the nonlinear system identification problem, which is a basic premise of control and fault diagnosis. For Hammerstein output-error nonlinear systems, we propose an auxiliary model-based multi-innovation fractional stochastic gradient method. The scalar innovation is extended to the innovation vector for increasing the data use based on the multi-innovation identification theory. By establishing appropriate auxiliary models, the unknown variables are estimated and the improvement in the performance of parameter estimation is achieved owing to the fractional-order calculus theory. Compared with the conventional multi-innovation stochastic gradient algorithm, the proposed method is validated to obtain better estimation accuracy by the simulation results.

Multiparameter Real-World System Identification Using Iterative Residual Tuning

AbstractIn this work, we consider the problem of nonlinear system identification using data to learn multiple and often coupled parameters that allow a simulator to more accurately model a physical system or mechanism and close the so-called reality gap for more accurate robot control. Our approach uses iterative residual tuning (IRT), a recently developed derivative-free system identification technique that utilizes neural networks and visual observation to estimate parameter differences between a proposed model and a target model. We develop several modifications to the basic IRT approach and apply it to the system identification of a five-parameter model of a marble rolling in a robot-controlled labyrinth game mechanism. We validate our technique both in simulation—where we outperform two baselines—and on a real system, where we achieve marble tracking error of 4% after just five optimization iterations.

Dynamical machine learning volumetric reconstruction of objects\u2019 interiors from limited angular views

Limited-angle tomography of an interior volume is a challenging, highly ill-posed problem with practical implications in medical and biological imaging, manufacturing, automation, and environmental and food security. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. Recently, it was shown that one effective way to learn the priors for strongly scattering yet highly structured 3D objects, e.g. layered and Manhattan, is by a static neural network [Goy et al. Proc. Natl. Acad. Sci. 116, 19848–19856 (2019)]. Here, we present a radically different approach where the collection of raw images from multiple angles is viewed analogously to a dynamical system driven by the object-dependent forward scattering operator. The sequence index in the angle of illumination plays the role of discrete time in the dynamical system analogy. Thus, the imaging problem turns into a problem of nonlinear system identification, which also suggests dynamical learning as a better fit to regularize the reconstructions. We devised a Recurrent Neural Network (RNN) architecture with a novel Separable-Convolution Gated Recurrent Unit (SC-GRU) as the fundamental building block. Through a comprehensive comparison of several quantitative metrics, we show that the dynamic method is suitable for a generic interior-volumetric reconstruction under a limited-angle scheme. We show that this approach accurately reconstructs volume interiors under two conditions: weak scattering, when the Radon transform approximation is applicable and the forward operator well defined; and strong scattering, which is nonlinear with respect to the 3D refractive index distribution and includes uncertainty in the forward operator.

A discrete repetitive sliding mode multicontrol for non-linear systems

The identification and the design of an accurate control are still an active researches of nonlinear systems subject to harmonic disturbances. In this paper, we propose two techniques to deal with nonlinear system identification and control problems. First, a new identification scheme based only on input/output measurements is proposed. In this scheme, an adaptive filter is used to extract the periodic disturbance and the non-perturbed system dynamics from the real output. Then, a multimodel approach is adopted to obtain the models base of the real non-perturbed system. Secondly,the models base is used to develop a multi-model sliding mode control (MM-SMC). Thereafter, the structure of plug-in repetitive control is incorporated into the MM-SMC to ensure good closed loop performances in terms of harmonic disturbance rejection and tracking in the case of nonlinear systems. Simulation results of two numerical examples shows the effectiveness of the proposed identification and control strategies in spite the presence of nonlinearities and harmonic disturbances.

Synthesis analysis for multi-UAVs formation anomaly detection

PurposeThe purpose of this paper is to deal with the anomaly detection problem in multi-unmanned aerial vehicles (UAVs) formation that can be transformed to identify some unknown parameters; a more general nonlinear dynamical model for each UAV is considered to include two terms. Due to an unknown parameter corresponding to the normal or abnormal state for each UAV, the bias-compensated approach is proposed to obtain the unbiased parameter estimation. Meanwhile, the biased error and accuracy analysis are also given in case of strict statistical description of the uncertainty or white noise. To relax this strict statistical description on external noise, an analytic center approach is proposed to identify the unknown parameters in presence of bounded noise, such that two inner and outer ellipsoidal approximations are constructed to include the membership set. To be precise, this paper is regarded as one extension and summary for the author’s previous research on the anomaly detection in multi-UAV formation. Finally, one simulation example is given to confirm the theoretical results.Design/methodology/approachFirstly, one extended nonlinear relation is constructed to embody the mutual relationship of UAVs. Secondly, to obtain the unbiased parameter estimations, the bias-compensated approach is applied to achieve it under the condition of white noise. Thirdly, in case of unknown but bounded noise, an analytic center approach is proposed to deal with this special case. Without loss of generality, the author thinks this paper can be used as one extension and summary for research on multi-UAVs formation anomaly detection.FindingsAn anomaly detection problem in multi-UAVs formation can be transformed into a problem of nonlinear system identification, and in modeling the nonlinear dynamical model for each UAV, two terms are considered simultaneously to embody the mutual relationships with other nearest UAV.Originality/valueTo the best knowledge of the authors, this problem of the anomaly detection problem in multi-UAVs formation is proposed by the authors’ previous work, and the problem of multi-UAVs formation anomaly detection can be transferred into one problem of parameter identification. In case of unknown but bounded noise, an analytic center approach is proposed to identify the unknown parameters, which correspond to achieve the goal of the anomaly detection.

Multi-gene genetic programming to building up fuzzy rule-base in Neo-Fuzzy-Neuron networks

This paper introduces a new approach to build the rule-base on Neo-Fuzzy-Neuron (NFN) Networks. The NFN is a Neuro-Fuzzy network composed by a set of n decoupled zero-order Takagi-Sugeno models, one for each input variable, each one containing m rules. Employing Multi-Gene Genetic Programming (MG-GP) to create and adjust Gaussian membership functions and a Gradient-based method to update the network parameters, the proposed model is dubbed NFN-MG-GP. In the proposed model, each individual of MG-GP represents a complete rule-base of NFN. The rule-base is adjusted by genetic operators (Crossover, Reproduction, Mutation), and the consequent parameters are updated by a predetermined number of Gradient method epochs, every generation. The algorithm uses Elitism to ensure that the best rule-base is not lost between generations. The performance of the NFN-MG-GP is evaluated using instances of time series forecasting and non-linear system identification problems. Computational experiments and comparisons against state-of-the-art alternative models show that the proposed algorithms are efficient and competitive. Furthermore, experimental results show that it is possible to obtain models with good accuracy applying Multi-Gene Genetic Programming to construct the rule-base on NFN Networks.