Short-term dependency of a class of nonlinear continuous time dynamic systems

Abstract

Dynamic systems usually have long-term dependency. Recent work, however, shows that the long-term memory models like recurrent neural networks and short-term memory models like feed-forward neural networks have comparable performance in approximating the behavior of dynamic systems. While the pioneering researchers try to understand the short-term dependency in discrete time dynamic systems, this paper focuses on a class of continuous time dynamic systems characterized by ordinary differential equations. In this class of continuous time dynamic systems, only the variable can be observed, and its derivatives with respect to time cannot be measured directly. By analyzing the observability of the continuous time dynamic systems and the sampled systems, we show that the current output only depends on finite steps of history information when sampling fast. If the sampling is fast enough such that the Euler approximation can approximate the sampled system well, the current output only relies on the most recent n steps of history information. Here, n is the order of the ordinary differential equation. Later, we verified our results with the NARMAX method.