New techniques for nonlinear system identification : a rapprochement between neural networks and linear systems

Abstract

System identification is an area of research that has important implications for many problems. Techniques for system identification have been developed over a long period of time, with significant contributions made by Gauss (1890), Mener (1942), Kahnan (1956), and many others since. It is known however, that many physical processes are intrinsically nonlinear, and cannot be adequately approximated by linear models. Various methods have been devised to overcome this, including a generalization of the linear transfer function model to the nonlinear case, resulting in the Volterra series. Describing Functions, and linearization techniques using the extended Kalman filter have also been popular. There are disadvantages with these models however: the Volterra series has the problem of requiring a large number of terms, and in many cases a nonlinear model is required rather than a linearization of the system about some operating point. A number of other techniques have been proposed which include the State-Dependent Model, Bilinear model. Threshold Autoregressive model, and Exponential Autoregressive model. These models in one way or another attempt to generalize the linear transfer function description to the nonlinear case. More recently, neural networks have undergone rapid development and become recognized as powerful nonlinear approximation methods. Specifically, the proof that there exists a network, within the class of two-layer multilayer perceptrons, that is capable of approximating any nonlinear function to an arbitrary degree of accuracy has been a major milestone. The usefulness of this model has raised the question of whether it is possible to extend its modelling capabilities to time-dependent systems. Methods which have been proposed to do this, include the use of sampled raw input data fed to several input units, transforms of the input data, thereby producing a static representation of the input data, (that is, at least for the duration of the window, as in a Short-Time Fourier Transform (STFT)). Other techniques have included the use of feedback connections in the network, and time-delays in various pathways. In spite of the desire to produce time-depedent models, there has been little use made of the existing principles of adaptive filter theory, and system identification. This thesis develops a new set of nonlinear system identification techiuques, biased on a rational approach to combining the powerful nonlinear approximation capabilities of multilayer perceptrons, with the known methods of adaptive filtering and system identification. It is shown that this approach is successful, through the introduction of new nonlinear models with useful properties which extend the capabilities of existing linear adaptive filters, and neural networks models for time-dependent data.Keywords: Neural Networks, Multilayer Perceptrons, Nonlinear System Identification, Adaptive Filters.