Nonlinear System Identification Problem Research Articles

Novel sparse LSSVR models in primal weight space for robust system identification with outliers

Recent demands from big data applications have strongly motivated a successful sparse formulation of the least squares support vector regression (LSSVR) model in primal weight space. Such an approach, which has been called fixed-size LSSVR (FS-LSSVR), is built upon an approximation of the nonlinear feature mapping based on Nyström method to overcome the usual memory constraints and high computational costs of the standard non-sparse LSSVR model. Despite this advance, an important modeling issue remains unaddressed by the FS-LSSVR model. As in the standard LSSVR model, the performance of FS-LSSVR model is greatly degraded when estimation data is corrupted with non-Gaussian noise or outliers. Bearing this major issue in mind, we introduce two robust variants of the FS-LSSVR model based on M-estimation framework and the weighted least squares method. The proposed approaches, henceforth called Robust FS-LSSVR (RFS-LSSVR) and Reweighted Robust FS-LSSVR (R2FS-LSSVR) models, produce solutions that are simultaneously robust to outliers and sparse, making use of only a small sample of training patterns as prototype vectors. We evaluate the performances of both algorithms in benchmarking nonlinear system identification problems with synthetic and real-world datasets (including a large scale dataset) corresponding to SISO and MIMO systems, whose estimation outputs are contaminated with outliers. The obtained results indicate that our proposed approaches consistently outperforms existing robust models designed in dual space (e.g. W-LSSVR and IR-LSSVR models), specially as the amount of outliers in the data increases.

Adaptive infinite impulse response system identification using modified-interior search algorithm with Lèvy flight

In this paper, a new meta-heuristic optimization technique, called interior search algorithm (ISA) with Lèvy flight is proposed and applied to determine the optimal parameters of an unknown infinite impulse response (IIR) system for the system identification problem. ISA is based on aesthetics, which is commonly used in interior design and decoration processes. In ISA, composition phase and mirror phase are applied for addressing the nonlinear and multimodal system identification problems. System identification using modified-ISA (M-ISA) based method involves faster convergence, single parameter tuning and does not require derivative information because it uses a stochastic random search using the concepts of Lèvy flight. A proper tuning of control parameter has been performed in order to achieve a balance between intensification and diversification phases. In order to evaluate the performance of the proposed method, mean square error (MSE), computation time and percentage improvement are considered as the performance measure. To validate the performance of M-ISA based method, simulations has been carried out for three benchmarked IIR systems using same order and reduced order system. Genetic algorithm (GA), particle swarm optimization (PSO), cat swarm optimization (CSO), cuckoo search algorithm (CSA), differential evolution using wavelet mutation (DEWM), firefly algorithm (FFA), craziness based particle swarm optimization (CRPSO), harmony search (HS) algorithm, opposition based harmony search (OHS) algorithm, hybrid particle swarm optimization-gravitational search algorithm (HPSO-GSA) and ISA are also used to model the same examples and simulation results are compared. Obtained results confirm the efficiency of the proposed method.

Combined nonlinear filtering architectures involving sparse functional link adaptive filters

Sparsity phenomena in learning processes have been extensively studied, since their detection allows to derive suited regularized optimization algorithms capable of improving the overall learning performance. In this paper, we investigate the sparsity behavior that may occur in nonlinear adaptive filtering problems and how to leverage it and develop enhanced algorithms. In particular, we focus on a particular class of linear-in-the-parameters nonlinear adaptive filters, whose nonlinear transformation is based on a functional link expansion. The analysis of the sparsity in functional links leads us to derive a family of adaptive combined filtering architectures that is capable of exploiting any sparseness degree in the nonlinear filtering. We propose two different filtering schemes based on a new block-based combined approach, well suited for sparse adaptive algorithms. Moreover, a hierarchical architecture is also proposed that generalizes the different combined schemes and does not need any a priori information about the nature of the nonlinearity to be modeled. Experimental results prove the effectiveness of the proposed combined architectures in exploiting any sparseness degree and improving the modeling performance in nonlinear system identification problems.

Extracting T-S Fuzzy Models Using the Cuckoo Search Algorithm.

A new method called cuckoo search (CS) is used to extract and learn the Takagi–Sugeno (T–S) fuzzy model. In the proposed method, the particle or cuckoo of CS is formed by the structure of rules in terms of number and selected rules, the antecedent, and consequent parameters of the T–S fuzzy model. These parameters are learned simultaneously. The optimized T–S fuzzy model is validated by using three examples: the first a nonlinear plant modelling problem, the second a Box–Jenkins nonlinear system identification problem, and the third identification of nonlinear system, comparing the obtained results with other existing results of other methods. The proposed CS method gives an optimal T–S fuzzy model with fewer numbers of rules.

Training ANFIS Using Genetic Algorithm for Dynamic Systems Identification

In this study, the premise and consequent parameters of ANFIS are optimized using Genetic Algorithm (GA) based on a population algorithm. The proposed approach is applied to the nonlinear dynamic system identification problem. The simulation results of the method are compared with the Backpropagation (BP) algorithm and the results of other methods that are available in the literature. With this study it was observed that the optimisation of ANFIS parameters using GA is more successful than the other methods.

Design of momentum LMS adaptive strategy for parameter estimation of Hammerstein controlled autoregressive systems

In the present work, strength of momentum least mean square (MLMS) algorithm is exploited for nonlinear system identification problems represented with Hammerstein model. The MLMS algorithm uses the previous gradient information to estimate the current weights instead of using only current value of gradient; thus, it is faster in the convergence and less probable to trap in local minima. The perfection of the design scheme is certified through effective parameter estimation of nonlinear Hammerstein control autoregressive models. The robustness of the scheme is established by examining the performance for different levels of noise variance. The performance comparison through mean square error and Nash–Sutcliffe efficiency parameters, calculated for sufficiently large number of multiple runs, proves the intrinsic worth of MLMS algorithm in system identification.

Gradient-Based Recursive Identification Methods for Input Nonlinear Equation Error Closed-Loop Systems

The identification problem of closed-loop or feedback nonlinear systems is a hot topic. Based on the hierarchical identification principle, this paper presents a hierarchical stochastic gradient algorithm and a hierarchical multi-innovation stochastic gradient algorithm for feedback nonlinear systems. The simulation results show that the hierarchical multi-innovation stochastic gradient can more effectively estimate the parameters of the feedback nonlinear systems than the hierarchical stochastic gradient algorithm.

Network-based Type-2 Fuzzy System with Water Flow Like Algorithm for System Identification and Signal Processing

This paper introduces a network-based interval type-2 fuzzy inference system (NT2FIS) with a dynamic solution agent algorithm water flow like algorithm (WFA), for nonlinear system identification and blind source separation (BSS) problem. The NT2FIS consists of interval type-2 asymmetric fuzzy membership functions and TSK-type consequent parts to enhance the performance. The proposed scheme is optimized by a new heuristic learning algorithm, WFA, with dynamic solution agents. The proposed WFA is inspired by the natural behavior of water flow. Splitting, moving, merging, evaporation, and precipitation have all been introduced for optimization. Some modifications, including new moving strategies, such as the application of tabu searching and gradient-descent techniques, are proposed to enhance the performance of the WFA in training the NT2FIS systems. Simulation and comparison results for nonlinear system identification and blind signal separation are presented to illustrate the performance and effectiveness of the proposed approach.

Nonlinear process identification in the presence of multiple correlated hidden scheduling variables with missing data

Identification of nonlinear processes in the presence of noise corrupted and correlated multiple scheduling variables with missing data is concerned. The dynamics of the hidden scheduling variables are represented by a state‐space model with unknown parameters. To assure generality, it is assumed that the multiple correlated scheduling variables are corrupted with unknown disturbances and the identification dataset is incomplete with missing data. A multiple model approach is proposed to formulate the identification problem of nonlinear systems under the framework of the expectation‐maximization algorithm. The parameters of the local process models and scheduling variable models as well as the hyperparameters of the weighting function are simultaneously estimated. The particle smoothing technique is adopted to handle the computation of expectation functions. The efficiency of the proposed method is demonstrated through several simulated examples. Through an experimental study on a pilot‐scale multitank system, the practical advantages are further illustrated. © 2015 American Institute of Chemical Engineers AIChE J, 61: 3270–3287, 2015

Recursive parameter and state estimation for an input nonlinear state space system using the hierarchical identification principle

This paper focuses on the identification problem of an input nonlinear state space system with colored noise. Based on the observability canonical form, an identification model is derived and a state observer is designed. By using the hierarchical identification principle, a state observer based hierarchical stochastic gradient algorithm is presented for estimating the parameter vectors and states jointly. Furthermore, by using the multi-innovation identification theory, a state observer based hierarchical multi-innovation stochastic gradient algorithm is proposed for improving the convergence rate. The analysis indicates that the parameter estimates given by the proposed algorithms converge to the true values under persistent excitation conditions. Two numerical examples are offered to demonstrate the effectiveness of the proposed algorithms.

Learning deep dynamical models from image pixels

Modeling dynamical systems is important in many disciplines, such as control, robotics, or neurotechnology. Commonly the state of these systems is not directly observed, but only available through noisy and potentially high-dimensional observations. In these cases, system identification, i.e., finding the measurement mapping and the transition mapping (system dynamics) in latent space can be challenging. For linear system dynamics and measurement mappings efficient solutions for system identification are available. However, in practical applications, the linearity assumptions does not hold, requiring nonlinear system identification techniques. If additionally the observations are high-dimensional (e.g., images), nonlinear system identification is inherently hard. To address the problem of nonlinear system identification from high-dimensional observations, we combine recent advances in deep learning and system identification. In particular, we jointly learn a low-dimensional embedding of the observation by means of deep auto-encoders and a predictive transition model in this low-dimensional space. We demonstrate that our model enables learning good predictive models of dynamical systems from pixel information only.

Multiple Model LPV Approach to Identification of Nonlinear Dual-rate System with Random Time Delay

This paper deals with the problem of nonlinear dual-rate system identification with random time delay. The proposed approach adopts the multiple modeling framework, and the global LPV model is represented by a combination of various local models weighted by a probability function. The considered structure of the local model is in a state space form, and the process has fast rate inputs and slow rate outputs with random time delay. The expectation maximization algorithm is utilized to formulate and solve the problem of interest. The parameters of the local models and the weighting functions are estimated simultaneously. The particle smoothing technique is adopted to handle the computation of expectation functions. The effectiveness of the proposed approach is further illustrated through a simulation example.

Network-based Type-2 Fuzzy System with Water Flow Like Algorithm for System Identification and Signal Processing

This paper introduces a network-based interval type-2 fuzzy inference system (NT2FIS) with a dynamic solution agent algorithm water flow like algorithm (WFA), for nonlinear system identification and blind source separation (BSS) problem. The NT2FIS consists of interval type-2 asymmetric fuzzy membership functions and TSK-type consequent parts to enhance the performance. The proposed scheme is optimized by a new heuristic learning algorithm, WFA, with dynamic solution agents. The proposed WFA is inspired by the natural behavior of water flow. Splitting, moving, merging, evaporation, and precipitation have all been introduced for optimization. Some modifications, including new moving strategies, such as the application of tabu searching and gradient-descent techniques, are proposed to enhance the performance of the WFA in training the NT2FIS systems. Simulation and comparison results for nonlinear system identification and blind signal separation are presented to illustrate the performance and effectiveness of the proposed approach.

Sequential Monte Carlo Methods for System Identification

One of the key challenges in identifying nonlinear and possibly non-Gaussian state space models (SSMs) is the intractability of estimating the system state. Sequential Monte Carlo (SMC) methods, such as the particle filter (introduced more than two decades ago), provide numerical solutions to the nonlinear state estimation problems arising in SSMs. When combined with additional identification techniques, these algorithms provide solid solutions to the nonlinear system identification problem. We describe two general strategies for creating such combinations and discuss why SMC is a natural tool for implementing these strategies.

A New Structure Exploiting Derivation of Recursive Direct Weight Optimization

The recursive direct weight optimization method is used to solve challenging nonlinear system identification problems. This note provides a new derivation and a new interpretation of the method. The key underlying the note is to acknowledge and exploit a certain structure inherent in the problem.

Control modeling of ash wood drying using process neural networks

For the control and system identification problems of the deceleration phase of the ash wood drying process, we propose a deceleration phase modeling method of ash wood drying using process neural networks with double hidden layers. This method applies time-varying characteristics of process neural networks and the ability to extract time-space cumulative effects. The time-varying characteristics of wood drying deceleration phase modeling under time series background are directly incorporated into the model. By comparison with traditional neural network modeling results, we prove that the model of process neural networks has better control accuracy, providing an idea to solve control and nonlinear system identification problems under a time series background.

Benchmark problems for nonlinear system identification and control using Soft Computing methods: Need and overview

Using benchmark problems to demonstrate and compare novel methods to the work of others could be more widely adopted by the Soft Computing community. This article contains a collection of several benchmark problems in nonlinear control and system identification, which are presented in a standardized format. Each problem is augmented by examples where it has been adopted for comparison. The selected examples range from component to plant level problems and originate mainly from the areas of mechatronics/drives and process systems. The authors hope that this overview contributes to a better adoption of benchmarking in method development, test and demonstration.

Volterra filter modeling of a nonlinear discrete-time system based on a ranked differential evolution algorithm

This paper presents a ranked differential evolution (RDE) algorithm for solving the identification problem of non-linear discrete-time systems based on a Volterra filter model. In the improved method, a scale factor, generated by combining a sine function and randomness, effectively keeps a balance between the global search and the local search. Also, the mutation operation is modified after ranking all candidate solutions of the population to help avoid the occurrence of premature convergence. Finally, two examples including a highly nonlinear discrete-time rational system and a real heat exchanger are used to evaluate the performance of the RDE algorithm and five other approaches. Numerical experiments and comparisons demonstrate that the RDE algorithm performs better than the other approaches in most cases.

Identification of System Properties in a Square Frame Undergoing Large Deformations: Numerical and Experimental Investigations

The aim of this paper is to highlight and identify the influencing parameters of the nonlinear behavior of highly deformable structures. Therefore, as an example, a large deformable square frame consisting of four slender members of equal length has been investigated experimentally. Based on highly resolving measurements using the digital image correlation method (DIC), the inverse problem of nonlinear system identification has been solved by an automatic parameter identification algorithm. For this purpose, a numerical model is set up with a beam finite element model using the absolute nodal coordinate formulation (ANCF), which enables the modeling of geometrical and possible material nonlinearities. The influencing parameters as well as the system properties have been determined by using a genetic optimization algorithm. The impact of the main influencing parameter is carved out by an included sensitivity study. The final model with automatically identified parameters shows high agreement with the experimental setup. With this approach the influences and nonlinearities, e.g. material parameters, rigid behavior, real boundary conditions, etc., come up to surface leading to a deeper understanding of the structural behavior of the system itself. Therefore, the present approach can be utilized for further investigations of nonstandard structures undergoing large deformations.

Adaptive Polynomial Filtering using Generalized Variable Step-Size Least Mean pth Power (LMP) Algorithm

This correspondence presents the adaptive polynomial filtering using the generalized variable step-size least mean $$p$$ p th power (GVSS-LMP) algorithm for the nonlinear Volterra system identification, under the $$\alpha $$ ? -stable impulsive noise environment. Due to the lack of finite second-order statistics of the impulse noise, we espouse the minimum error dispersion criterion as an appropriate metric for the estimation error, instead of the conventional minimum mean square error criterion. For the convergence of LMP algorithm, the adaptive weights are updated by adjusting $$p\ge 1$$ p ? 1 in the presence of impulsive noise characterized by $$1<\alpha <2$$ 1 < ? < 2 . In many practical applications, the autocorrelation matrix of input signal has the larger eigenvalue spread in the case of nonlinear Volterra filter than in the case of linear finite impulse response filter. In such cases, the time-varying step-size is an appropriate option to mitigate the adverse effects of eigenvalue spread on the convergence of LMP adaptive algorithm. In this paper, the GVSS updating criterion is proposed in combination with the LMP algorithm, to identify the slowly time-varying Volterra kernels, under the non-Gaussian $$\alpha $$ ? -stable impulsive noise scenario. The simulation results are presented to demonstrate that the proposed GVSS-LMP algorithm is more robust to the impulsive noise in comparison to the conventional techniques, when the input signal is correlated or uncorrelated Gaussian sequence, while keeping $$1<p<\alpha <2$$ 1 < p < ? < 2 . It also exhibits flexible design to tackle the slowly time-varying nonlinear system identification problem.