Rényi information transfer: Partial rényi transfer entropy and partial rényi mutual information

Abstract

Shannon and Renyi information theory have been applied to coupling estimation in complex systems using time series of their dynamical states. By analysing how information is transferred between constituent parts of a complex system, it is possible to infer the coupling parameters of the system. To this end, we introduce the partial Renyi transfer entropy and we give an alternative derivation of the partial Renyi mutual information, using the conditional Renyi α-divergence. We prove that, in the limit α → 1, this divergence tends to the conditional Kullback-Leibler divergence from Shannon information theory. As a result, when α → 1, we obtain the partial transfer entropy and the partial mutual information from their Renyi equivalents. Using these Renyi information-theoretic functionals, we identify the coupling direction and delay between two processes in an autoregressive system of order 1.