On information transfer in discrete dynamical systems

Abstract

In this paper, we propose a new definition of information transfer in a discrete dynamical system. The information transfer is based on how much entropy (uncertainty) is transferred from state x to state y, as the dynamical system evolves in time. In our previous work, we had provided an axiomatic definition of information transfer where we had considered absolute entropy to define the transfer. Our new definition can be viewed as a natural generalization of directed information from information theory to dynamical systems and is based on transfer of conditional entropy. This new definition also satisfies the properties of a) zero transfer, b) transfer asymmetry and c) information conservation. We start with a definition of one-step information transfer and then generalize this definition to n-step information transfer and average information transfer over infinite time step. We also provide analytical expressions for information transfer for linear systems. Some basic examples are provided to understand the physical meaning of information transfer and how this can be used to infer causality and influence in dynamical system setting.