Nonlinear System Identification Problem Research Articles

Constrained multisine input signals for plant-friendly identification of chemical process systems

This paper considers the use of constrained minimum crest factor multisine signals as inputs for plant-friendly identification testing of chemical process systems. The methodology presented here effectively integrates operating restrictions, information-theoretic requirements, and state-of-the-art optimization techniques to design minimum crest factor multisine signals meeting important user-specified time and frequency domain properties. A series of optimization problem formulations relevant to problems in linear, nonlinear, and multivariable system identification are presented; these culminate with their application to the modeling of the Weischedel–McAvoy high-purity distillation column problem, a demanding nonlinear and highly interactive system. The effectiveness of these signals for modeling for control purposes and the ability to incorporate a priori nonlinear models in the signal design procedure are demonstrated in this distillation system case study.

Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

Grammatical Immune System Evolution for reverse engineering nonlinear dynamic Bayesian models.

An artificial immune system algorithm is introduced in which nonlinear dynamic models are evolved to fit time series of interacting biomolecules. This grammar-based machine learning method learns the structure and parameters of the underlying dynamic model. In silico immunogenetic mechanisms for the generation of model-structure diversity are implemented with the aid of a grammar, which also enforces semantic constraints of the evolved models. The grammar acts as a DNA repair polymerase that can identify recombination and hypermutation signals in the antibody (model) genome. These signals contain information interpretable by the grammar to maintain model context. Grammatical Immune System Evolution (GISE) is applied to a nonlinear system identification problem in which a generalized (nonlinear) dynamic Bayesian model is evolved to fit biologically motivated artificial time-series data. From experimental data, we use GISE to infer an improved kinetic model for the oxidative metabolism of 17beta-estradiol (E(2)), the parent hormone of the estrogen metabolism pathway.

Application of evolving Takagi–Sugeno fuzzy model to nonlinear system identification

In this paper, a new encoding scheme is presented for learning the Takagi–Sugeno (T–S) fuzzy model from data by genetic algorithms (GAs). In the proposed encoding scheme, the rule structure (selection of rules and number of rules), the input structure (selection of inputs and number of inputs), and the antecedent membership function (MF) parameters of the T–S fuzzy model are all represented in one chromosome and evolved together such that the optimisation of rule structure, input structure, and MF parameters can be achieved simultaneously. The performance of the developed evolving T–S fuzzy model is first validated by studying the benchmark Box–Jenkins nonlinear system identification problem and nonlinear plant modelling problem, and comparing the obtained results with other existing results. Then, it is applied to approximate the forward and inverse dynamic behaviours of a magneto-rheological (MR) damper of which identification problem is significantly difficult due to its inherently hysteretic and highly nonlinear dynamics. It is shown by the validation applications that the developed evolving T–S fuzzy model can identify the nonlinear system satisfactorily with acceptable number of rules and appropriate inputs.

Nonlinear System Identification using a New Sliding-Window Kernel RLS Algorithm

In this paper we discuss in detail a recently proposed kernel-based version of the recursive least-squares (RLS) algorithm for fast adaptive nonlinear filtering. Unlike other previous approaches, the studied method combines a sliding-window approach (to fix the dimensions of the kernel matrix) with conventional ridge regression (to improve generalization). The resulting kernel RLS algorithm is applied to several nonlinear system identification problems. Experiments show that the proposed algorithm is able to operate in a time-varying environment and to adjust to abrupt changes in either the linear filter or the nonlinearity.

Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification

Mixture of experts (ME) models comprise a family of modular neural network architectures aiming at distilling complex problems into simple subtasks. This is done by deploying a separate gating module for softly dividing the input space into overlapping regions to be each assigned to one or more expert networks. Conversely, support vector machines (SVMs) refer to kernel-based methods, neural-network-alike models that constitute an approximate implementation of the structural risk minimization principle. Such learning machines follow the simple, but powerful idea of nonlinearly mapping input data into high-dimensional feature spaces wherein a linear decision surface discriminating different regions is properly designed. In this work, we formally characterize and empirically evaluate a novel approach, named as Mixture of Support Vector Machine Experts (MSVME), whose main purpose is to combine the complementary properties of both SVM and ME models. In the formal characterization, an algorithm based on a maximum likelihood criterion is considered for the MSVME training, and we demonstrate that it is possible to train each expert based on an SVM perspective. Regarding the empirical evaluation, simulation results involving nonlinear dynamic system identification problems are reported, contrasting the performance shown by the MSVME approach with that exhibited by conventional SVM and ME models.

USING MANIFOLD LEARNING FOR NONLINEAR SYSTEM IDENTIFICATION

A high-dimensional regression space usually causes problems in nonlinear system identification. However, if the regression data are contained in (or spread tightly around) some manifold, the dimensionality can be reduced. This paper presents a use of dimension reduction techniques to compose a two-step identification scheme suitable for high-dimensional identification problems with manifold-valued regression data. Illustrating examples are also given.

Sparse Model Identification Using a Forward Orthogonal Regression Algorithm Aided by Mutual Information

A sparse representation, with satisfactory approximation accuracy, is usually desirable in any nonlinear system identification and signal processing problem. A new forward orthogonal regression algorithm, with mutual information interference, is proposed for sparse model selection and parameter estimation. The new algorithm can be used to construct parsimonious linear-in-the-parameters models.

Introducing a new learning method for fuzzy descriptor systems with the aid of spectral analysis to forecast solar activity

In the last two decades, researches indicate that the physical precursor and solar dynamo techniques are preferred as practical tools for long term prediction of solar activity. But, why more than 23 cycles of solar activity history should be omitted and just use the empirical methods or simple autoregressive methods on the basis of observations for the latest eight cycles? In this article, a method based on fuzzy descriptor systems (as a generalization of ordinary Takagi Sugeno (T–S) neuro-fuzzy models), developed by the authors to yield a satisfactory solution to the unresolved problem of nonlinear descriptor system identification, and singular spectrum analysis (SSA) as one of the spectral analysis is proposed to forecast some of solar activity's indexes in the way that, a fuzzy descriptor model is optimized for each of the principal components obtained from SSA, and the multi step predicted values are recombined to make the disturbance storm time (DST) and proton flux indexes. The proposed method is used for forecasting hourly DST index in 2001 and daily average of the DST index from 1957 to 2005 and proton flux index in 2001. The results are remarkably good in the predictions of DST and proton flux indexes.

Reinforcement genetic approach to coefficient estimation for multivariable nonlinear discrete-time dynamical systems

In this paper, we propose a novel genetic algorithm (GA) with a multi-crossover fashion to estimate the associated coefficients for a class of nonlinear discrete-time multivariable dynamical systems. Unlike the traditional crossover method of using two chromosomes, the proposed method uses three chromosomes to achieve a crossover. According to the adjusting direction by crossing three chromosomes, more excellent offspring can be produced. To solve the identification problem of multivariable nonlinear discrete-time systems, each of estimated system coefficients represents a gene, and a collection of genes is referred to as a chromosome in the view of GA. The chromosomes in the population are then evolved using the proposed multi-crossover method. An illustrative example of multivariable nonlinear systems is given to demonstrate the effectiveness, as compared with the traditional crossover method, of the proposed method.

Fuzzy rules extraction using a hybrid of base‐n‐number‐coded GA and SVD‐QR

A hybrid of a base‐n‐number‐coded genetic algorithm (base‐n‐number‐coded GA) and an SVD‐QR is proposed to construct a fuzzy system directly from some gathered input‐output data of the identified system. Each individual in the base‐n‐number‐coded GA is applied to determine the fuzzy sets in each input variable. However, the grid‐type fuzzy partition by the fuzzy sets associated with each input variable may generate some redundant fuzzy subspaces. Therefore, an SVD‐QR method is applied to remove the redundant fuzzy subspaces to efficiently describe the behavior of the identified system so that the premise part of the fuzzy system is determined. Then, the recursive least‐squares method is used to determine the consequent part of the fuzzy system. Subsequently, a fitness function is defined such that it can guide the search procedure to select an appropriate fuzzy system that not only maintains a good performance but also has relevant fuzzy rules. Finally, two nonlinear system identification problems are used to illustrate the efficiency of the proposed method.

Nonlinear Model Structure Identification of Complex Biomedical Data Using a Genetic- Programming-Based Technique

In this contribution, a genetic programming (GP)-based technique, which combines the ability of GP to explore both automatically and effectively, the whole set of candidate model structures and the robustness of evolutionary multimodel partitioning filters, is presented. The method is applied to the nonlinear system identification problem of complex biomedical data. Simulation results show that the algorithm identifies the true model and the true values of the unknown parameters for each different model structure, thus assisting the GP technique to converge more quickly to the (near) optimal model structure. The method has all the known advantages of the evolutionary multimodel partitioning filters, that is, it is not restricted to the Gaussian case; it is applicable to on-line/adaptive operation and is computationally efficient. Furthermore, it can be realized in a parallel processing fashion, a fact which makes it amenable to very large scale integration implementation.

ONLINE NONLINEAR SYSTEM IDENTIFICATION USING LINEAR MODEL TREES

A linear model tree is a decision tree with a linear functional model in each leaf that can represent piecewise linear systems. This paper extends recent work in the machine learning literature on the online learning of linear model trees. Both the leaf splitting rule and the pruning rule are improved to take into account more information. The algorithm is demonstrated on two nonlinear system identification problems, including the modelling of complex high dimensional flight dynamics. Results compare favourably with existing online approaches, including an alternative linear model tree algorithm.

ON THE USE OF PARTIALLY LINEAR MODEL IN IDENTIFICATION OF ARCING-FAULT LOCATION ON OVERHEAD HIGH-VOLTAGE TRANSMISSION LINES

The task of locating an arcing-fault on overhead line using sampled measurements obtained at a single line terminal could be classified as a practical nonlinear system identification problem. The practical reasons impose the requirement that the solution should be with maximum possible precision. Dynamic behavior of an arc in open air is influenced by the environmental conditions that are changing randomly, and therefore the useful practically application of parametric modeling is out of question. The requirement to identify only one parameter is yet another specific of this problem. The parameter we need is the one that linearly correlates the voltage samples with the current derivative samples (inductance). The correlation between the voltage samples and the current samples depends on the unpredictable arc dynamic behavior. Therefore this correlation is reconstructed using nonparametric regression. A partially linear model combines both, parametric and nonparametric parts in one model. The fit of this model is noniterative, and provides an efficient way to identify (pull out) a single linear correlation from the nonlinear time series.

Modified recursive least squares (RLS) algorithm for neural networks using piecewise linear function

The recursive least squares (RLS) learning algorithm for multilayer feedforward neural networks uses a sigmoid nonlinearity at node outputs. It is shown that by using a piecewise linear function at node outputs, the algorithm becomes faster. The modified algorithm improves computational efficiency and by preserving matrix symmetry it is possible to avoid explosive divergence, which is normally seen in the conventional RLS algorithm due to the finite precision effects. Also the use of this piecewise linear function avoids the approximation, which is otherwise necessary in the derivation of the conventional algorithm with sigmoid nonlinearity. Simulation results on the XOR problem, 4–2–4 encoder and function approximation problem indicate that the modified algorithm reduces the occurrence of local minima and improves the convergence speed compared to the conventional RLS algorithm. A nonlinear system identification and control problem is considered to demonstrate the application of the algorithm to complex problems.

Identification of nonlinear systems via piecewise general orthogonal polynomials operator

Based on a hybrid orthogonal function, a new linear, continuous and bounded operator, piecewise general orthogonal polynomials operator (PGOPO), is proposed. Its main properties and operational rules have been strictly constructed based on the theory of convergence in the mean square, and are useful to discuss the orthogonal polynomial approach in the proper mathematics frame. Then applying the PGOPO method to solve the identification problem of nonlinear systems, the convergence analysis of PGOPO approximation method is given. Finally, using the PGOPO method to solve the parameter identification of two kinds of time-varying systems, the simulation studies show that the algorithm is simple and more effective than that of general orthogonal polynomials.

Nonlinear dynamic system identification using Chebyshev functional link artificial neural networks

A computationally efficient artificial neural network (ANN) for the purpose of dynamic nonlinear system identification is proposed. The major drawback of feedforward neural networks, such as multilayer perceptrons (MLPs) trained with the backpropagation (BP) algorithm, is that they require a large amount of computation for learning. We propose a single-layer functional-link ANN (FLANN) in which the need for a hidden layer is eliminated by expanding the input pattern by Chebyshev polynomials. The novelty of this network is that it requires much less computation than that of a MLP. We have shown its effectiveness in the problem of nonlinear dynamic system identification. In the presence of additive Gaussian noise, the performance of the proposed network is found to be similar or superior to that of a MLP. A performance comparison in terms of computational complexity has also been carried out.

Modelling of nonlinear process dynamics using Kohonen's neural networks, fuzzy systems and Chebyshev series

This paper introduces a new systematic methodology to the problem of nonlinear system identification with the aid of neural networks, fuzzy systems and truncated Chebyshev series. The proposed methodology is of general use and results in both a linguistic and an analytical model of the system under study. The method was successfully tested in the identification of certain operating regions in a Continuous Stirred Tank Reactor (CSTR) exhibiting various types of nonlinear behaviour, such as limit cycles and multiple steady states. The performance of the methodology was evaluated via a comparison with two different identification schemes, namely a feedforward neural network and an approach based on the normal form theory.

SYSTEM IDENTIFICATION USING DYNAMIC NEURAL NETWORKS: TRAINING AND INITIALIZATION ASPECTS

This paper explores training and initialization aspects of dynamic neural networks when applied to the nonlinear system identification problem. A well known dynamic neural network structure contains both output states and hidden states. Output states are related to the outputs of the system represented by the network. Hidden states are particularly important in allowing dynamic neural networks to approximate complex nonlinear dynamics. An optimisation based method is proposed in this paper for properly initialising the hidden states of a dynamic neural network, so as to avoid the introduction of bias in the network parameters as a result of incorrect hidden state initialisation. Furthermore, a simple optimisation based method is proposed to initialise the hidden states once the network has been trained. The methods are illustrated with experimental data taken from a laboratory scale pressure plant.

Adaptive identification of nonlinear systems with application to chaotic communications

This paper studies the nonlinear system identification problem in a noisy environment using an adaptive algorithm. In particular, nonlinear systems of the polynomial type are considered here. An improved least squares (ILS) objective function is used to reduce the estimation bias caused by measurement noise. Based on this ILS criterion, a novel adaptive filter is developed to track a time-varying polynomial system. Numerical simulations showed that the proposed adaptive algorithm was superior to the conventional identification technique. We applied this new adaptive filter to demodulate the signals of transmission in a chaotic multiuser spread spectrum (SS) communication system. It was observed that the new approach was effective in demodulating a SS signal, even at low signal-to-noise ratios (SNR's).