Linear systems equivalent to artificial neural networks via Lie theory

Abstract

In this paper we study continuous time nonlinear artificial neural networks of Hopfield type using geometrical and topological methods. A particularly useful geometrical concept, the Lie algebra of analytical vector fields, is employed. Under certain conditions we are able to prove that a Hopfield type network with a time-varying synaptic weight matrix is equivalent (in the sense of locally diffeomorphic) to a linear system of differential equations. Our main intention with this work is to structure the synaptic weight matrix in a way that enables us to develop analytical solutions to the nonlinear network equations. Therefore we do not resort to training in order to calculate the synaptic weights. In this respect our work provides a different point of view of a Hopfield network, where we are directly concerned with the network state trajectories rather than simply the initial and final states.