Can Transfer Entropy Infer Information Flow in Neuronal Circuits for Cognitive Processing?

Ali Tehrani-Saleh,Christoph Adami

doi:10.3390/e22040385

Abstract

How cognitive neural systems process information is largely unknown, in part because of how difficult it is to accurately follow the flow of information from sensors via neurons to actuators. Measuring the flow of information is different from measuring correlations between firing neurons, for which several measures are available, foremost among them the Shannon information, which is an undirected measure. Several information-theoretic notions of “directed information” have been used to successfully detect the flow of information in some systems, in particular in the neuroscience community. However, recent work has shown that directed information measures such as transfer entropy can sometimes inadequately estimate information flow, or even fail to identify manifest directed influences, especially if neurons contribute in a cryptographic manner to influence the effector neuron. Because it is unclear how often such cryptic influences emerge in cognitive systems, the usefulness of transfer entropy measures to reconstruct information flow is unknown. Here, we test how often cryptographic logic emerges in an evolutionary process that generates artificial neural circuits for two fundamental cognitive tasks (motion detection and sound localization). Besides counting the frequency of problematic logic gates, we also test whether transfer entropy applied to an activity time-series recorded from behaving digital brains can infer information flow, compared to a ground-truth model of direct influence constructed from connectivity and circuit logic. Our results suggest that transfer entropy will sometimes fail to infer directed information when it exists, and sometimes suggest a causal connection when there is none. However, the extent of incorrect inference strongly depends on the cognitive task considered. These results emphasize the importance of understanding the fundamental logic processes that contribute to information flow in cognitive processing, and quantifying their relevance in any given nervous system.

Highlights

When searching for common foundations of cortical computation, more and more emphasis is being placed on information-theoretic descriptions of cognitive processing [1,2,3,4,5]
We sequentially eliminated each logic gate, where we eliminate all the input and output connections of that gate, and re-measured the mutant Brain’s fitness, allowing us to estimate which gates were essential to the motion detection function and which gates were redundant to the motion detection function
Our results imply that using pairwise transfer entropy has its limitations in accurately estimating the information flow, and its accuracy may depend on the type of network or cognitive task it is applied to, as well as the type of data that is used to construct the measure

Summary

Introduction

When searching for common foundations of cortical computation, more and more emphasis is being placed on information-theoretic descriptions of cognitive processing [1,2,3,4,5]. One of the core tasks in the analysis of cognitive processing is to follow the flow of information within the nervous system, by finding cause-effect components. Understanding causal relationships is considered to be fundamental to all natural sciences [6]. Inferring causal relationships and separating them from mere correlations is difficult, and the subject of ongoing research [7,8,9,10,11]. Entropy 2020, 22, 385 interactions among components or processes of a system. Schreiber [12] described Granger causality in terms of information theory by introducing the concept of transfer entropy (TE). The main idea is that if a process X is influencing process Y, an observer can predict the future state of Y more (k)

Methods

Results

Discussion

Conclusion