System Modeling using Composition Networks

Abstract

We present a new approach to the modeling of continuous-time system observed in discrete time. The rationale behind this approach is that discrete-time models of continuous-time systems ought to be invertible; this property implying that we should investigate group theoretic methods - using compositions of invertible basis transformations to approximate the (invertible) map of a dynamical system. The appropriate framework for our approach is Lie algebra theory and, more specifically, the use of the Baker-Campbell-Hausdorff formula to develop methods of integration for ordinary differential equations using compositions. This leads to the development of architectures, which we call composition networks and recurrent composition networks, for dynamical system prediction and nonlinear system identification. These techniques incorporate a priori knowledge about the system (grey-box models) and therefore favorably compare with multi-layer perceptrons, if a model of the system is available. For the case where no a priori knowledge is available, we merge the composition network and the multi-layer perceptron to produce a“MLP in dynamics space”, which is an architecture having the universal approximation property for dynamical systems.Key WordsDynamical system approximationnonlinear system identificationLie algebra theoryBaker-Campbell-Hausdorff formula