Class Of Recurrent Neural Networks Research Articles

Computability with low-dimensional dynamical systems

It has been known for a short time that a class of recurrent neural networks has universal computational abilities. These networks can be viewed as iterated piecewise-linear maps in a high-dimensional space. In this paper, we show that similar systems in dimension two are also capable of universal computations. On the contrary, it is necessary to resort to more complex systems (e.g., iterated piecewise-monotone maps) in order to retain this capability in dimension one.

Feature-based model identification of nonlinear biotechnological processes

Abstract Modelling ill-defined systems requires powerful tools to attain a quantitatively description of the studied systems. In this paper, a modelling concept is presented that tackles the problems inherent to processes for which the necessary a priori knowledge for deductive analysis is lacking. The approach can be summarised as follows. By means of relationship detectors, such as SAPS or fractional factorials, the existence of a causal structure can be deduced qualitatively. In a next step of the modelling task, the goal is to find the quantitative description of this relationship. This step can be subdivided in two phases, i.e. model structure characterisation and finally parameter estimation. In this paper new techniques are proposed (and validated on real-life experimental results) to achieve the latter steps. In order to separate the structure characterisation from the parameter estimation task, an approach is taken in which parameter-invariant features are extracted from the data. The properties of these features are chosen in such a way that a classifier can select a specific model description. The decomposition in Zernike features and the recurrent neural network classifier, introduced by Sudharsanan, constitute the implementation of this concept. To check the feasibility of this modelling approach, a biotechnological application is chosen as a test case. Due to the changing nature of the wastewater treatment process, reflected in a set of mathematical descriptions applicable at different time instances, the aforementioned methodology will give the possibility to develop more efficient adaptive controllers.

A Class of Recurrent Neural Networks for Adaptive Control of Nonlinear Systems

In this paper, recurrent neural networks (RNN) are used to construct nonlinear learning systems for a class of unknown nonlinear systems. We describe the RNN controllers with reference to adaptive control theory and put emphasis on their flexible structure to express nonlinear systems. The proposed control scheme employs two types of RNN: The first network learns the unknown dynamics of the nonlinear process to be controlled. The second utilises the information given by the first network about the process, to learn the inverse dynamics of the neural network process model such that the error between the system and the desired output is nullified. We also present a generalized criterion for training the RNN which gives rise to new control schemes. The training methodology that is used for the weights adaptation is much faster than classical backpropagation algorithm. We investigate the effects the type of nonlinear function, present in the generalized criterion, have on performance of the learning algorithm of unknown nonlinear system dynamics.

Recurrent neural networks and robust time series prediction

We propose a robust learning algorithm and apply it to recurrent neural networks. This algorithm is based on filtering outliers from the data and then estimating parameters from the filtered data. The filtering removes outliers from both the target function and the inputs of the neural network. The filtering is soft in that some outliers are neither completely rejected nor accepted. To show the need for robust recurrent networks, we compare the predictive ability of least squares estimated recurrent networks on synthetic data and on the Puget Power Electric Demand time series. These investigations result in a class of recurrent neural networks, NARMA(p,q), which show advantages over feedforward neural networks for time series with a moving average component. Conventional least squares methods of fitting NARMA(p,q) neural network models are shown to suffer a lack of robustness towards outliers. This sensitivity to outliers is demonstrated on both the synthetic and real data sets. Filtering the Puget Power Electric Demand time series is shown to automatically remove the outliers due to holidays. Neural networks trained on filtered data are then shown to give better predictions than neural networks trained on unfiltered time series.

Recurrent neural networks for linear programming: Analysis and design principles

This paper explores the potential role of recurrent neural networks for solving linear programs. The emphases of the paper are on analyzing the asymptotic properties of recurrent neural networks that are relevant to linear programming and on developing general principles for designing such neural networks. A class of recurrent neural networks with monotonically increasing penalty variables is presented for solving linear programming problems. The proposed recurrent neural networks are asymptotically stable and able to generate optimal solutions to linear programming problems. The asymptotic properties of the proposed recurrent neural networks for linear programming are analyzed theoretically, and the design principles for synthesizing the recurrent networks are discussed based on the results of analysis. Some illustrative examples are also presented to demonstrate the performance behavior and operational characteristics of the recurrent neural networks.