Dynamical properties of learning process of weakly nonlinear and nonlinear neurons

Abstract

In recent years, arti cial neural networks have been studied very intensively. There are a lot of papers describing applications of neural networks to the solution of problems in control theory (see [14, Section 6:3; 16, Section 7:5; 21, Section 4:4; 24, Chapter 5; 28, Section 8:3]), robotics [28, Section 8:4], speech recognition [14, Section 6:3; 26, Section 10:1], pattern recognition [14, Section 6:3; 21, Section 4:1; 28, Section 8:2], data compression [14, Section 6:3; 21, Section 4:2], expert systems [28, Section 8:3], and many others. Since problems emerge with the learning process of neural networks, mathematical models have also been introduced (see [22],[29]). Dynamical systems theory has been used to analyse behaviour of recurrent networks [14, Section 2:2, 16, Chapter 4] and it seems to be a suitable tool for investigation layer neural networks learning process [26, Chapter 9; 4]. On the other hand, recently, several papers have been devoted to studying the qualitative properties of discrete-time dynamical systems obtained via discretization methods. The basic question is whether the qualitative properties of continuous-time systems are preserved under discretization. Various concepts of di erentiable dynamics were investigated. Results on stability and attraction properties [15], the saddle-point structure about equilibria [1,2], invariant manifolds [3,6], averagings [7] and algebraic-topological invariants [19,25] can be mentioned as examples. Numerical applications have been